Image sensor, shooting module and electronic equipment

Abstract

The invention discloses an image sensor, a shooting module and electronic equipment, and belongs to the technical field of electronic equipment. The image sensor comprises a pixel array which comprises a plurality of pixel units, each pixel unit comprises a pixel circuit used for converting an optical signal into an electric signal, and a probe circuit which is connected with the pixel circuit and is used for monitoring the optical flow change of the pixel circuit and outputting an optical flow signal in the exposure period of the pixel circuit; the pixel optical flow comparison module is respectively connected with the probe circuit in each column of pixel units and is used for generating multiple groups of optical flow difference data according to optical flow signals output by the plurality of pixel units in an exposure time period, each group of optical flow difference data is used for generating an optical flow difference subframe, and the optical flow difference subframe is used for indicating position information of a moving object at different time; and the signal processing module is respectively connected with the pixel circuits in each column of pixel units and is used for generating image data according to the electric signals output by the plurality of pixel units, and the image data is used for generating image frames.

Description

Technical Field

[0001] This application belongs to the field of electronic equipment technology, specifically relating to an image sensor, a shooting module, and an electronic device. Background Technology

[0002] Currently, mainstream CIS (Complementary Metal-Oxide Semiconductor Image Sensors) employ a rolling shutter readout method, which allows for high-speed and efficient image acquisition and output. However, because the reset and exposure times are fixed in rolling shutter readout, pixels cannot record the motion trajectory of moving objects in detail during their exposure period. Therefore, in the final image, fast-moving objects become blurred, resulting in loss of detail and image distortion, a problem that is particularly severe for high-pixel-count, high-resolution image sensors.

[0003] Therefore, it is necessary to detect the motion information of objects during the shooting process in order to repair the captured image based on the motion information. With the emergence of EVS (Event Vision Sensor), also known as DVS (Dynamic Vision Sensor), which can sense the rapid changes of moving objects, a technology is proposed to simultaneously realize CIS imaging and EVS or DVS imaging using a single chip.

[0004] However, the above technologies have the following drawbacks: DVS or EVS will cause optical and electrical noise to CIS when it is working, thereby reducing the quality of the CIS generated image; since DV or EVS pixels are completely incompatible with CIS pixels, the number of DVS or EVS pixels on a single chip is much less than the number of CIS pixels, resulting in the resolution of DVS or EVS images being much lower than that of CIS images, which reduces the motion detection effect; DVS or EVS signals cannot be temporally aligned with CIS image frames, making back-end signal processing difficult and reducing the efficiency of image restoration. Summary of the Invention

[0005] This application provides an image sensor, a shooting module, and an electronic device that can improve the accuracy of acquired motion and image information based on image capture and motion detection. Furthermore, the motion and image information is easy to process, thereby improving the efficiency of image restoration based on motion information.

[0006] In a first aspect, embodiments of this application provide an image sensor, comprising: a pixel array including multiple pixel units, each pixel unit including: a pixel circuit for converting light signals into electrical signals; and a probe circuit connected to the pixel circuit for monitoring changes in optical flow of the pixel circuit and outputting optical flow signals during the exposure period of the pixel circuit; a pixel optical flow comparison module connected to the probe circuit in each column of pixel units, the pixel optical flow comparison module being used to generate multiple sets of optical flow difference data based on the optical flow signals output by the multiple pixel units during the exposure period of the pixel circuit, each set of optical flow difference data being used to generate an optical flow difference subframe, the optical flow difference subframe being used to indicate the position information of a moving object at different times; and a signal processing module connected to the pixel circuit in each column of pixel units, the signal processing module being used to generate image data based on the electrical signals output by the multiple pixel units, the image data being used to generate image frames.

[0007] The aforementioned image sensor, in addition to the pixel circuit and signal processing module for generating image frames, adds a pixel optical flow comparison module and a probe circuit in each pixel unit. The probe circuit monitors the optical flow changes of the pixel circuit during the exposure period of the pixel circuit and outputs an optical flow signal. The pixel optical flow comparison module generates multiple sets of optical flow difference data for generating optical flow difference subframes based on the optical flow signals output by multiple pixel units during the exposure period of the pixel circuit. The optical flow difference subframes are used to indicate the position information of the moving object at different times. In this way, the image sensor can simultaneously capture images and detect motion. A probe circuit is added to each pixel unit to monitor changes in optical flow, ensuring that each pixel unit participates in motion detection. This increases the image resolution for motion detection, improving clarity, acquiring more detailed motion information, and enhancing the accuracy of motion detection. The probe circuit and pixel optical flow comparison module operate only during the pixel circuit's exposure period, without affecting the generation of image frames during the reading period, thus improving image frame quality and the accuracy of the acquired image information. Furthermore, motion information is output as optical flow difference subframes generated during the exposure period, enabling temporal alignment between the optical flow difference subframes and image frames. This facilitates backend processing of motion and image information, improving the efficiency of motion-based image restoration.

[0008] Optionally, the pixel optical flow comparison module includes: a plurality of pixel optical flow comparators, each pixel optical flow comparator being connected to a probe circuit in at least one column of pixel units, the pixel optical flow comparators being used to compare the optical flow signals output by the probe circuits in at least one pixel unit row by row, and outputting a first optical flow comparison signal and a second optical flow comparison signal; a first data scanner being connected to each pixel optical flow comparator, being used to scan the first optical flow comparison signal and output first optical flow difference data; and a second data scanner being connected to each pixel optical flow comparator, being used to scan the second optical flow comparison signal and output second optical flow difference data; wherein each set of optical flow difference data includes the first optical flow difference data and the second optical flow difference data.

[0009] This enables the classification and output of optical flow signal comparison results, thereby achieving fine division of optical flow difference data, improving the accuracy of optical flow detection, and thus improving the accuracy of detected motion information.

[0010] Optionally, each pixel circuit includes a photodiode; each probe circuit includes a probe sub-circuit, the input of which is connected to the photodiode, and the output of which is connected to two adjacent pixel optical flow comparators; each pixel optical flow comparator is connected to two adjacent columns of probe sub-circuits respectively, and the pixel optical flow comparator is used to compare the first optical flow signal and the second optical flow signal output by the two adjacent probe sub-circuits row by row, and output the first optical flow comparison signal and the second optical flow comparison signal.

[0011] In this way, optical flow detection is achieved for each individual pixel, allowing every single pixel to participate in motion information detection. This increases the image resolution for motion detection, thereby improving the clarity of motion detection, acquiring more motion detail information, and enhancing the accuracy of detected motion information. Furthermore, based on the classification output of the optical flow signal comparison results, fine-grained segmentation of optical flow difference data can be achieved, further improving the accuracy of optical flow detection and the accuracy of detected motion information.

[0012] Optionally, each pixel circuit includes N photodiodes, which are divided into two groups of photodiodes, and each group of photodiodes includes M photodiodes, where N and M are integers, N≥2, M≥1; each probe circuit includes two probe sub-circuits, the input of each probe sub-circuit is connected to a group of photodiodes, and the output of each probe sub-circuit is connected to a pixel optical flow comparator; each pixel optical flow comparator is connected to two probe sub-circuits in each column of probe circuits, and the pixel optical flow comparator is used to compare the first optical flow signal and the second optical flow signal output by the two probe sub-circuits in each probe circuit row by row, and output the first optical flow comparison signal and the second optical flow comparison signal.

[0013] In this way, optical flow detection is achieved within each synthesized pixel, which can further increase the image resolution of motion detection, thereby improving the clarity of motion detection, acquiring more motion detail information, and improving the accuracy of detected motion information. Furthermore, based on the classification output of the optical flow signal comparison results, fine segmentation of the optical flow difference data can be achieved, thereby improving the accuracy of optical flow detection and the accuracy of detected motion information.

[0014] Optionally, each pixel circuit includes four photodiodes, the four photodiodes are divided into two groups of photodiodes, and each group of photodiodes includes two photodiodes.

[0015] This enables optical flow detection within the four-in-one pixel, thereby achieving motion detection functionality based on the four-in-one pixel.

[0016] Optionally, each pixel circuit includes nine photodiodes, the nine photodiodes are divided into two groups of photodiodes, and each group of photodiodes includes three photodiodes.

[0017] This enables optical flow detection within the nine-in-one pixel, thereby achieving motion detection functionality based on the nine-in-one pixel.

[0018] Optionally, each pixel circuit includes 16 photodiodes, the 16 photodiodes are divided into two groups of photodiodes, and each group of photodiodes includes 8 photodiodes.

[0019] This enables optical flow detection within the 16-in-1 pixel, thereby achieving motion detection functionality based on the 16-in-1 pixel.

[0020] Optionally, the input terminal of each probe sub-circuit is connected to all the photodiodes in a set of photodiodes.

[0021] In this way, the signal amplitude of the input probe sub-circuit is larger, the signal change during optical flow detection is more obvious, and the accuracy of optical flow detection is improved.

[0022] Optionally, the input terminal of each probe sub-circuit is connected to one of the photodiodes in a set of photodiodes.

[0023] In this way, a probe sub-circuit is connected to a photodiode, which simplifies the structure and reduces the wiring difficulty in circuit design.

[0024] Optionally, in each probe circuit, the two photodiodes connected to the two probe sub-circuits are staggered in both the row and column directions.

[0025] In this way, by performing optical flow detection on the two photodiodes that are diagonally distributed in the composite pixel, optical flow detection can be achieved simultaneously in both row and column directions, thereby simultaneously achieving motion detection in both row and column directions and improving the accuracy of the detected motion information.

[0026] Optionally, both the first optical flow signal and the second optical flow signal are voltage signals. When the voltage amplitude of the first optical flow signal is greater than that of the second optical flow signal, the first optical flow comparison signal is a high-level signal and the second optical flow comparison signal is a low-level signal. When the voltage amplitude of the first optical flow signal is less than that of the second optical flow signal, the first optical flow comparison signal is a low-level signal and the second optical flow comparison signal is a high-level signal. When the voltage amplitude of the first optical flow signal is equal to that of the second optical flow signal, both the first optical flow comparison signal and the second optical flow comparison signal are low-level signals.

[0027] This allows optical flow difference data to be generated and output in single-bit binary form, which reduces the power consumption of generating optical flow difference subframes and makes it easier to increase the number of generated optical flow difference subframes in order to obtain more accurate motion information and achieve fine restoration of image information.

[0028] Optionally, each probe sub-circuit includes: a first transistor, the control terminal of which is connected to a photodiode, and the first terminal of which is connected to a power supply; a second transistor, the first terminal of which is connected to the second terminal of the first transistor; a third transistor, the first terminal of which is connected to the second terminal of the first transistor, and the second terminal of which is grounded; a fourth transistor, the first terminal of which is connected to a power supply, and the second terminal of which is connected to the second terminal of the second transistor; a fifth transistor, the control terminal of which is connected to the second terminal of the second transistor, and the first terminal of which is connected to a power supply; and a sixth transistor, the first terminal of which is connected to the second terminal of the fifth transistor, and the second terminal of which is connected to a pixel optical flow comparator; wherein a floating diffusion capacitor is formed between the second terminal of the second transistor, the second terminal of the fourth transistor, and the control terminal of the fifth transistor.

[0029] This enables the monitoring of optical flow changes in different pixel circuits and the output of optical flow signals line by line, allowing the image sensor to detect motion information of the shooting scene while acquiring image information of the shooting scene.

[0030] Secondly, embodiments of this application provide a shooting module, including: an image sensor as described in the first aspect.

[0031] Thirdly, embodiments of this application provide an electronic device, including: a camera module as described in the second aspect.

[0032] The second and third aspects mentioned above have various possible circuit structure designs similar to the first aspect and any possible implementation of the first aspect, and can produce corresponding technical effects, which will not be elaborated here. Attached Figure Description

[0033] Figure 1 A schematic diagram of an image sensor provided for some embodiments of this application;

[0034] Figure 2 One of the schematic diagrams for optical flow detection provided for some embodiments of this application;

[0035] Figure 3 A second schematic diagram of an image sensor provided for some embodiments of this application;

[0036] Figure 4 A schematic diagram of a rolling shutter reading method provided for some embodiments of this application;

[0037] Figure 5 A second schematic diagram of optical flow detection provided for some embodiments of this application;

[0038] Figure 6 Timing diagrams of the operation of each device in the probe sub-circuit provided for some embodiments of this application;

[0039] Figure 7 A third schematic diagram of an image sensor provided for some embodiments of this application;

[0040] Figure 8 A schematic diagram of individual pixel circuitry provided for some embodiments of this application;

[0041] Figure 9 One of the schematic diagrams of pixel units provided for some embodiments of this application;

[0042] Figure 10 One of the schematic diagrams of a pixel synthesis circuit provided for some embodiments of this application;

[0043] Figure 11 A second schematic diagram of a pixel synthesis circuit provided for some embodiments of this application;

[0044] Figure 12 A second schematic diagram of a pixel synthesis circuit provided for some embodiments of this application;

[0045] Figure 13 A second schematic diagram of a pixel unit provided for some embodiments of this application;

[0046] Figure 14 A third schematic diagram of a pixel unit provided for some embodiments of this application;

[0047] Figure 15 Fourth schematic diagram of a pixel unit provided for some embodiments of this application;

[0048] Figure 16 Schematic diagram of a pixel optical flow comparator provided for some embodiments of this application;

[0049] Figure 17 Fifth schematic diagram of a pixel unit provided for some embodiments of this application;

[0050] Figure 18 A sixth schematic diagram of pixel units provided for some embodiments of this application;

[0051] Figure 19 Seventh schematic diagram of pixel units provided for some embodiments of this application;

[0052] Figure 20 Eighth schematic diagram of pixel units provided for some embodiments of this application;

[0053] Figure 21 Structural block diagrams of the shooting module provided for some embodiments of this application;

[0054] Figure 22 Structural block diagrams of electronic devices provided for some embodiments of this application.

[0055] Figure label:

[0056] 100 - Image sensor, 102 - Pixel array, 104 - Pixel unit, 106 - Pixel circuit, 108 - Probe circuit, 110 - Pixel optical flow comparator module, 112 - Signal processing module, 114 - Pixel optical flow comparator, 116 - First data scanner, 118 - Second data scanner, 120 - Photodiode, 122 - Probe sub-circuit, 124 - Power supply, 126 - Column parallel analog-to-digital converter module, 130 - Column scanner, 132 - Row driver, 134 - First sample-and-hold circuit, 136 - Second sample-and-hold circuit, 138 - Differential amplifier, 140 - First comparator, 142 - Second comparator, 402 - First optical flow difference subframe, 404 - Second optical flow difference subframe, 406 - Third optical flow difference subframe, T 1 - First transistor, T2 - Second transistor, T3 - Third transistor, T4 - Fourth transistor, T5 - Fifth transistor, T6 - Sixth transistor, T7 - ​​Seventh transistor, T8 - Eighth transistor, T9 - Ninth transistor, T10 - Tenth transistor, T11 - Eleventh transistor, D1 - First diode, D2 - Second diode, D3 - Third diode, D4 ​​- Fourth diode, D5 - Fifth diode, D6 - Sixth diode, D7 - Seventh diode, D8 - Eighth diode, D9 - Ninth diode, D10 - Tenth diode, D11 - Eleventh diode, D12 - Twelfth diode, D13 - Thirteenth diode, D14 - Fourteenth diode, D15 - Fifteenth diode, D16 - Sixteenth diode, C FLUX -First floating diffusion capacitance, C FD - Second floating diffusion capacitor, 200- Shooting module, 300- Electronic device. Detailed Implementation

[0057] The embodiments of this application will now be described in detail. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0058] The terms "first" and "second" in the specification and claims of this application may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0059] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0060] The following is combined with Figures 1-22 The image sensor, imaging module, and electronic device according to embodiments of this application are described in detail.

[0061] like Figure 1 As shown, this application embodiment provides an image sensor. The image sensor 100 includes a pixel array 102, a pixel optical flow comparison module 110, and a signal processing module 112.

[0062] The pixel array 102 includes multiple pixel units 104, and each pixel unit 104 includes a pixel circuit 106 and a probe circuit 108.

[0063] In this configuration, the probe circuit 108 in each pixel unit 104 is connected to the pixel circuit 106. The pixel optical flow comparison module 110 is connected to the probe circuit 108 in each column of pixel units 104, and the signal processing module 112 is connected to the pixel circuit 106 in each column of pixel units 104.

[0064] During the operation of the image sensor 100, the pixel circuit 106 is used to acquire light signals and convert them into electrical signals. The probe circuit 108 is used to monitor the changes in optical flow of the pixel circuit 106 during the exposure period of the pixel circuit 106 and output optical flow signals.

[0065] Optionally, each pixel circuit 106 includes at least one PDD (Pinned-Photodiode), which is used to receive light signals and generate charges.

[0066] Understandably, each pixel circuit 106 needs to be exposed within one frame. During the exposure period of the pixel circuit 106, the PPD within the pixel circuit 106 receives light signals and generates charge. During the exposure period, all the charge generated by the PPD is stored in its own capacitance, and the accumulation of charge causes the voltage of the PPD to continuously decrease from the reset voltage value. The slope of the PPD voltage change is the optical flow of the pixel circuit 106.

[0067] The optical flow of the pixel circuit 106 is not constant. When there are moving objects in the shooting scene, their movement causes changes in the light and shadow within the scene over time, resulting in changes in the light intensity received by each pixel circuit 106 per unit time – this is the change in optical flow. Throughout the exposure period, due to the changing trajectory of the moving objects, the timing of the optical flow change in each pixel circuit 106 in the pixel array 102 is different. Based on this, the optical flow change of the pixel circuit 106 can reflect the motion trajectory and shape of objects in the shooting scene.

[0068] For example, such as Figure 2 As shown, taking a tennis ball flying from left to right as an example, the scene is captured by pixel array 102. Pixel A and pixel B are pixels at different positions in pixel array 102, with pixel B closer to the left edge of pixel array 102 than pixel A. During the exposure time of one frame, the tennis ball initially enters the imaging range of pixel array 102 from the leftmost edge and moves to the right. Before the tennis ball reaches the positions of pixels A and B, the optical flow of pixels A and B remains unchanged. After the right edge of the tennis ball reaches pixel B, pixel B receives light from the tennis ball and its optical flow changes. At this time, since pixel A is located to the right of pixel B, and the tennis ball has not yet reached the position of pixel A, the optical flow of pixel A remains unchanged, and there is an optical flow difference between pixels A and B. After the tennis ball reaches the position of pixel A, similarly, pixel A receives light from the tennis ball and its optical flow changes. When both pixels A and B are within the range of the tennis ball, the optical flow of pixels A and B remains unchanged, and there is no optical flow difference between them. When the tennis ball reaches pixel B at its left edge, pixel B stops receiving light from the ball and undergoes another change in optical flow. At this point, pixel A is still within the range of the tennis ball, and its optical flow remains unchanged. A difference in optical flow exists between pixels A and B. Similarly, when the tennis ball continues its movement and reaches pixel A at its left edge, pixel A also undergoes another change in optical flow because it no longer receives light from the ball. During the subsequent movement of the tennis ball, since both pixels A and B are outside the range of the tennis ball, their optical flows remain unchanged, and there is no difference in optical flow between them.

[0069] Based on this, the pixel optical flow comparison module 110 calculates optical flow difference data at multiple time points within the exposure period of the pixel circuit 106 according to a certain sampling frequency, based on the optical flow signals output by multiple pixel units 104. It then packages and outputs a set of optical flow difference data calculated at each time point, thereby outputting multiple sets of optical flow difference data within the exposure period of the pixel circuit 106. Each set of optical flow difference data is used to generate an optical flow difference subframe, which indicates the position information of a moving object at different times. All optical flow difference subframes within the exposure period of the pixel circuit 106 can reflect the movement of the moving object within the exposure period.

[0070] Optionally, the aforementioned optical flow difference data is in binary form, and the optical flow difference subframe is a binary image that includes only black and white pixels.

[0071] For example, such as Figure 5 As shown, following the example above, if pixel A and pixel B are two adjacent pixels, the pixel optical flow comparison module 110 will detect the optical flow difference between pixel A and pixel B in real time. At the first moment, the tennis ball moves to its right edge and reaches pixel B. Pixel B experiences a change in optical flow, while the optical flow of pixel A remains unchanged. An optical flow difference is generated between pixel A and pixel B, and the pixel optical flow comparison module 110 outputs a binary signal 1. This binary signal 1 serves as the pixel signal at the corresponding position in the optical flow difference subframe. Based on this, and based on the detection results of the optical flow difference between all pixels, a first optical flow difference subframe 402 at the first moment can be generated. The first optical flow difference subframe 402 is used to indicate the position information of the tennis ball at the first moment. At the second moment, when both pixel A and pixel B are within the range of the tennis ball, the optical flow of both pixels A and B remains unchanged, and there is no optical flow difference between them. The pixel optical flow comparison module 110 outputs a binary signal 0, which serves as the pixel signal at the corresponding position in the optical flow difference subframe. Based on this, and based on the optical flow difference detection results between all pixels, a second optical flow difference subframe 404 at the second time step can be generated. The second optical flow difference subframe 404 is used to indicate the position information of the tennis ball at the second time step. At the third time step, the tennis ball moves to its left edge and reaches pixel A. Pixel A experiences an optical flow change while the optical flow of pixel B remains unchanged. An optical flow difference is generated again between pixel A and pixel B. The pixel optical flow comparison module 110 outputs binary signal 1 again. Binary signal 1 serves as the pixel signal at the corresponding position in the optical flow difference subframe. Based on this, and based on the optical flow difference detection results between all pixels, a third optical flow difference subframe 406 at the third time step can be generated. The third optical flow difference subframe 406 is used to indicate the position information of the tennis ball at the third time step.

[0072] Optionally, the signal processing module 112 is used to generate image data based on the electrical signals output by the multiple pixel units 104 during the reading period of the pixel circuit 106. The image data is used to generate image frames, and the image frames are used to indicate scene information indicating the shooting scene.

[0073] Based on this, multiple optical flow difference subframes and one image frame can be generated within one frame, enabling simultaneous monitoring of moving objects and imaging of the shooting scene. Within one frame, the number of optical flow difference subframes is significantly greater than the number of image frames, allowing for motion monitoring of moving objects during the exposure period. Furthermore, the probe circuit 108 and pixel circuit 106 are physically isolated and output signals at different time intervals, meaning the generation times of the optical flow difference subframes and image frames are completely isolated, allowing for motion information detection without affecting image frame quality. Moreover, the optical flow difference data and image data are precisely aligned in time, ensuring that multiple optical flow difference subframes are temporally aligned with the image frame. This allows for effective supplementation of motion information within the exposure period based on the image frame, building upon the existing optical flow difference subframes. Additionally, the pixel optical flow comparison module 110 is placed outside the pixel array 102, preserving pixel miniaturization.

[0074] During the reading period of pixel circuit 106, probe circuit 108 and pixel optical flow comparison module 110 both stop working.

[0075] Optionally, optical flow difference subframes are used to process image frames to achieve motion blur removal, high-speed frame capture, slow-motion shooting, and other functions.

[0076] Optionally, the aforementioned optical flow difference data and image data are input into an ISP (Image Signal Processor) for processing. Through the ISP, the optical flow difference data is converted into optical flow difference subframes, and the image data is converted into image frames.

[0077] In practical applications, the ISP can be integrated inside the image sensor 100 or placed outside the image sensor 100; no specific restrictions are imposed here.

[0078] Optionally, the image sensor 100 may specifically be a CIS, i.e., a CMOS image sensor. The image sensor 100 employs a rolling shutter readout method.

[0079] Optionally, such as Figure 3 and Figure 7 As shown, the signal processing module 112 may specifically include a column-parallel analog-to-digital converter module 126 and a column scanner 130. The column-parallel analog-to-digital converter module 126 may include multiple analog-to-digital converters, each corresponding to a multiple column of pixel units 104.

[0080] Optionally, such as Figure 3 and Figure 7 As shown, the image sensor 100 also includes a row driver 132, which is connected to the pixel array 102. The row driver 132 is used to control the reset, exposure and readout of each pixel unit 104 in the pixel array 102.

[0081] Specifically, the row driver 132 controls the operation of each pixel unit 104 in the pixel array 102 in a row-parallel manner. The electrical signals generated by each pixel unit 104 are transmitted to the column-parallel analog-to-digital converter module 126 in a column-parallel manner for parallel signal processing to generate digital image data. The image data is then scanned column by column by the column scanner 130 into the ISP to generate image frames.

[0082] Based on this, the schematic diagram of the rolling shutter reading method is as follows: Figure 4 As shown. Specifically, within each frame, each pixel unit 104 undergoes three steps: reset, exposure, and readout. However, the operation steps of each row of pixel units 104 have a time difference from those of other rows. Driven by the row driver 132, each row of pixel units 104 performs reset, exposure, and readout row by row after receiving a control signal. The column-parallel analog-to-digital converter module 126 receives the electrical signal output from each row of pixel units 104 and performs analog-to-digital conversion. Based on this, the image sensor 100, using a rolling shutter operation mode, can perform high-speed and efficient image acquisition and output.

[0083] The image sensor 100 provided in this application embodiment, based on the pixel circuit 106 for generating image frames and the signal processing module 112, adds a pixel optical flow comparison module 110 and adds a probe circuit 108 in each pixel unit 104. Then, the probe circuit 108 monitors the optical flow changes of the pixel circuit 106 during the exposure period of the pixel circuit 106 and outputs an optical flow signal. And the pixel optical flow comparison module 110 generates multiple sets of optical flow difference data for generating optical flow difference subframes based on the optical flow signals output by multiple pixel units 104 during the exposure period of the pixel circuit 106. The optical flow difference subframes are used to indicate the position information of the moving object at different times. In this way, the image sensor 100 can simultaneously capture images and detect motion. Furthermore, a probe circuit 108 is added to each pixel unit 104 to monitor changes in optical flow, ensuring that each pixel unit 104 participates in motion information detection. This increases the image resolution for motion detection, thereby improving the clarity of motion detection, acquiring more detailed motion information, and enhancing the accuracy of the detected motion information. The probe circuit 108 and the pixel optical flow comparison module 110 only operate during the exposure period of the pixel circuit 106, without affecting the generation of image frames during the reading period, thus improving the quality of image frames and the accuracy of the acquired image information. Moreover, the motion information is output in the form of optical flow difference subframes generated during the exposure period, enabling temporal alignment between the optical flow difference subframes and the image frames. This facilitates back-end processing of motion and image information, improving the efficiency of image restoration based on motion information.

[0084] According to some embodiments of this application, optionally, such as Figure 3 and Figure 7 As shown, the pixel optical flow comparison module 110 includes multiple pixel optical flow comparators 114, a first data scanner 116, and a second data scanner 118.

[0085] Among them, such as Figure 3 , Figure 9 , Figure 13 , Figure 14 as well as Figure 15 As shown, each pixel optical flow comparator 114 is connected to a probe circuit 108 in at least one column of pixel units 104.

[0086] Optionally, such as Figure 3 and Figure 7 As shown, the first data scanner 116 is connected to all the pixel optical flow comparators 114, and the second data scanner 118 is also connected to all the pixel optical flow comparators 114.

[0087] During the operation of the pixel optical flow comparison module 110, the pixel optical flow comparator 114 is used to compare the optical flow signals output by the probe circuit 108 in at least one pixel unit 104 row by row, and output a first optical flow comparison signal and a second optical flow comparison signal; the first data scanner 116 is used to scan all the first optical flow comparison signals output by the pixel optical flow comparators 114, and output the first optical flow difference data; the second data scanner 118 is used to scan all the second optical flow comparison signals output by the pixel optical flow comparators 114, and output the second optical flow difference data.

[0088] Each set of optical flow difference data includes first optical flow difference sub-data and second optical flow difference sub-data.

[0089] According to an embodiment of the image sensor 100 of this application, the pixel optical flow comparison module 110 includes a plurality of pixel optical flow comparators 114, a first data scanner 116, and a second data scanner 118. Each pixel optical flow comparator 114 is connected to a probe circuit 108 in at least one column of pixel units 104. The first data scanner 116 is connected to each pixel optical flow comparator 114, and the second data scanner 118 is also connected to each pixel optical flow comparator 114. The pixel optical flow comparators 114 are used to compare the optical flow signals output by the probe circuits 108 in at least one pixel unit 104 row by row, and output a first optical flow comparison signal and a second optical flow comparison signal. The first data scanner 116 is used to scan the first optical flow comparison signals output by the plurality of pixel optical flow comparators 114, and output first optical flow difference data. The second data scanner 118 is used to scan the second optical flow comparison signals output by the plurality of pixel optical flow comparators 114, and output second optical flow difference data. Each set of optical flow difference data includes first optical flow difference data and second optical flow difference data. This enables the classification and output of optical flow signal comparison results, thereby achieving fine division of optical flow difference data, improving the accuracy of optical flow detection, and thus improving the accuracy of detected motion information.

[0090] According to some embodiments of this application, optionally, such as Figure 8 As shown, each pixel circuit 106 includes a photodiode 120. That is, the pixel circuit 106 is a separate pixel circuit.

[0091] Among them, photodiode 120 is the aforementioned PDD.

[0092] Specifically, such as Figure 8 As shown, the pixel circuit 106 includes a photodiode 120, a seventh transistor T7, an eighth transistor T8, a ninth transistor T9, and a tenth transistor T10.

[0093] In this configuration, the anode of photodiode 120 is grounded, the first terminal of the seventh transistor T7 is connected to the cathode of photodiode 120, the second terminal of the seventh transistor T7 is connected to the first terminal of the eighth transistor T8 and the control terminal of the ninth transistor T9, the second terminal of the eighth transistor T8 and the first terminal of the ninth transistor T9 are both connected to power supply 124, the second terminal of the ninth transistor T9 is connected to the first terminal of the tenth transistor T10, and the second terminal of the tenth transistor T10 is used to output electrical signals.

[0094] Optionally, a second floating diffusion capacitor C is formed between the second terminal of the seventh transistor T7, the first terminal of the eighth transistor T8, and the control terminal of the ninth transistor T9. FD Among them, the second floating diffusion capacitance C FDSpecifically, it can be a total capacitance node formed by the coupling of multiple parasitic capacitances. These parasitic capacitances mainly include: the depletion layer capacitance of the PN junctions of the substrates of the seventh transistor T7, the eighth transistor T8, and the ninth transistor T9; the overlap capacitance formed by the gate of the seventh transistor T7 overlapping with the total capacitance node; the channel capacitance introduced by the connection of the channel of the seventh transistor T7 to the total capacitance node; the junction capacitance of the first terminal of the eighth transistor T8; the overlap capacitance formed by the gate of the eighth transistor T8 overlapping with the total capacitance node; and the gate oxide capacitance introduced by the direct connection of the control terminal (gate) of the ninth transistor T9 to the total capacitance node. Among these, the second floating diffusion capacitance C... FD The capacitance value is related to the size of the ninth transistor T9. It can be increased by increasing the size of the ninth transistor T9, thereby increasing the second floating diffusion capacitance C. FD The capacitance value.

[0095] Optionally, in the pixel circuit 106, the photodiode 120 is used to convert the received light signal into photogenerated charge and store it, and the seventh transistor T7 is used to control whether to transfer the charge in the photodiode 120 to the second floating diffusion capacitor C. FD The second floating diffusion capacitor C FD The eighth transistor T8 is used to store the charge generated by the photodiode 120 during the exposure period, and the eighth transistor T8 is used to control the second floating diffusion capacitor C. FD Whether to perform a reset operation, that is, to control the eighth transistor T8 and the second floating diffusion capacitor C FD Whether the charge in the photodiode 120 is cleared, the ninth transistor T9 acts as a buffer amplifier to convert the charge generated by the photodiode 120 during the exposure period into an electrical signal, specifically a voltage signal. The tenth transistor T10 acts as a row selector to control whether to output the electrical signal generated by the ninth transistor T9.

[0096] Specifically, the charge generated by photodiode 120 during the exposure period is stored in the second floating diffusion capacitor C through the seventh transistor T7. FD During the read phase, the row driver 132 controls the tenth transistor T10 to turn on, storing data in the second floating diffusion capacitor C. FD The charge in the circuit is converted into an electrical signal by the ninth transistor T9 and then output via the tenth transistor T10. During the reset period, the line driver 132 controls the eighth transistor T8 to turn on, and the power signal from the power supply 124 enters the second floating diffusion capacitor C through the eighth transistor T8. FD To the second floating diffusion capacitor C FD The charge in the photodiode is cleared and reset, and the power signal of the power supply 124 enters the photodiode 120 through the seventh transistor T7 to clear and reset the charge in the photodiode 120.

[0097] Based on this, such as Figure 9 As shown, each probe circuit 108 includes a probe sub-circuit 122.

[0098] Each probe sub-circuit 122 has its input terminal connected to a photodiode 120 within the same pixel unit 104. Each probe sub-circuit 122 is used to monitor the optical flow changes of the pixel circuit 106 during the exposure period of the pixel circuit 106 by monitoring the voltage changes of the connected photodiode 120, and outputs an optical flow signal.

[0099] Optionally, the output of each probe sub-circuit 122 is connected to two adjacent pixel optical flow comparators 114. That is, each pixel optical flow comparator 114 is connected to the probe sub-circuit 122 in two adjacent columns of probe circuits 108.

[0100] Based on this, the optical flow signal output by the same probe sub-circuit 122 is transmitted to two adjacent pixel optical flow comparators 114 respectively. That is, a pixel optical flow comparator 114 can receive the optical flow signals output by two adjacent probe sub-circuits 122.

[0101] Based on this, during the operation of the pixel optical flow comparison module 110, the pixel optical flow comparator 114 is specifically used to compare the first optical flow signal and the second optical flow signal output by two adjacent probe sub-circuits 122 row by row, and output the first optical flow comparison signal and the second optical flow comparison signal according to the comparison result of the first optical flow signal and the second optical flow signal.

[0102] According to the image sensor 100 of this application embodiment, each pixel circuit 106 includes a photodiode 120; each probe circuit 108 includes a probe sub-circuit 122, the input terminal of which is connected to the photodiode 120, and the output terminal of which is connected to two adjacent pixel optical flow comparators 114; each pixel optical flow comparator 114 is connected to two adjacent columns of probe sub-circuits 122 respectively, and the pixel optical flow comparator 114 is used to compare the first optical flow signal and the second optical flow signal output by two adjacent probe sub-circuits 122 row by row, and output the first optical flow comparison signal and the second optical flow comparison signal. In this way, for each individual pixel, optical flow detection is achieved for every two adjacent individual pixels, so that each individual pixel participates in the motion information detection work, which facilitates increasing the image resolution of motion detection, thereby improving the clarity of motion detection, obtaining more motion detail information, and improving the accuracy of detected motion information. Furthermore, based on the classification output of the optical flow signal comparison results, fine division of optical flow difference data can be achieved, thereby improving the accuracy of optical flow detection and the accuracy of detected motion information.

[0103] According to some embodiments of this application, optionally, such as Figure 10 , Figure 11 or Figure 12 As shown, each pixel circuit 106 includes N photodiodes 120, the N photodiodes 120 include two groups of photodiodes 120, and each group of photodiodes 120 includes M photodiodes 120.

[0104] Where N and M are both integers, N≥2, M≥1. That is, pixel circuit 106 is a multi-in-one pixel circuit, such as a four-in-one pixel circuit, a nine-in-one pixel circuit, and a sixteen-in-one pixel circuit, etc.

[0105] Based on this, such as Figure 13 , Figure 14 or Figure 15 As shown, each probe circuit 108 includes two probe sub-circuits 122, and the two probe sub-circuits 122 correspond one-to-one with two sets of photodiodes 120.

[0106] Each probe sub-circuit 122 has its input terminal connected to a set of photodiodes 120. Each probe sub-circuit 122 is used to monitor the voltage change of the connected set of photodiodes 120 during the exposure period of the pixel circuit 106 and output an optical flow signal.

[0107] Optionally, the output of each probe sub-circuit 122 is connected to the pixel optical flow comparator 114.

[0108] In this configuration, two probe sub-circuits 122 in the same probe circuit 108 are connected to the same pixel optical flow comparator 114. That is, each pixel optical flow comparator 114 is connected to two probe sub-circuits 122 in each column of probe circuits 108.

[0109] Based on this, the optical flow signals output by the two probe sub-circuits 122 in the same probe circuit 108 are transmitted to the same pixel optical flow comparator 114. That is, a pixel optical flow comparator 114 can receive the first optical flow signal and the second optical flow signal output by the two probe sub-circuits 122 in the same probe circuit 108.

[0110] Based on this, during the operation of the pixel optical flow comparison module 110, the pixel optical flow comparator 114 is specifically used to compare the first optical flow signal and the second optical flow signal output by the two probe sub-circuits 122 in each probe circuit 108 row by row, and output the first optical flow comparison signal and the second optical flow comparison signal according to the comparison result of the first optical flow signal and the second optical flow signal.

[0111] Optionally, such as Figure 16As shown, each pixel optical flow comparator 114 may specifically include a first sample-and-hold circuit 134, a second sample-and-hold circuit 136, a differential amplifier 138, a first comparator 140, and a second comparator 142. The first sample-and-hold circuit 134 samples and holds the voltage value of the first optical flow signal, and the second sample-and-hold circuit 136 samples and holds the voltage value of the second optical flow signal. The differential amplifier 138 amplifies the voltage difference between the input first and second optical flow signals. The first comparator 140 compares the amplified first optical flow signal with a first voltage threshold V. TH_L The two optical flow signals are compared and a first optical flow comparison signal is output. The second comparator 142 is used to compare the amplified second optical flow signal with a second voltage threshold V. TH_R The comparison is performed, and a second optical flow comparison signal is output.

[0112] In practical applications, such as Figure 17 , Figure 18 , Figure 19 and Figure 20 As shown, each pixel unit 104 also includes at least one eleventh transistor T11.

[0113] In this circuit, the first terminal of the eleventh transistor T11 is connected to the probe sub-circuit 122, and the second terminal of the eleventh transistor T11 is connected to the pixel circuit 106. The eleventh transistor T11 is used to block the electrical connection between the probe sub-circuit 122 and the pixel circuit 106. In scenarios where optical flow detection is not required, the electrical connection between the probe sub-circuit 122 and the pixel circuit 106 is disconnected by turning off the eleventh transistor T11.

[0114] According to the image sensor 100 of the present application embodiment, each pixel circuit 106 includes N photodiodes 120, the N photodiodes 120 include two groups of photodiodes 120, and each group of photodiodes 120 includes M photodiodes 120, where N and M are both integers, N≥2, M≥1; each probe circuit 108 includes two probe sub-circuits 122, the input terminal of each probe sub-circuit 122 is connected to a group of photodiodes 120, and the output terminal of each probe sub-circuit 122 is connected to a pixel optical flow comparator 114; each pixel optical flow comparator 114 is connected to two probe sub-circuits 122 in each column of probe circuits 108, and the pixel optical flow comparator 114 is used to compare the first optical flow signal and the second optical flow signal output by the two probe sub-circuits 122 in each probe circuit 108 row by row, and output the first optical flow comparison signal and the second optical flow comparison signal. In this way, optical flow detection is achieved within each synthesized pixel, which can further increase the image resolution of motion detection, thereby improving the clarity of motion detection, acquiring more motion detail information, and improving the accuracy of detected motion information. Furthermore, based on the classification output of the optical flow signal comparison results, fine segmentation of the optical flow difference data can be achieved, thereby improving the accuracy of optical flow detection and the accuracy of detected motion information.

[0115] According to some embodiments of this application, optionally, such as Figure 10 As shown, each pixel circuit 106 includes four photodiodes 120, the four photodiodes 120 include two groups of photodiodes 120, and each group of photodiodes 120 includes two photodiodes 120.

[0116] That is, the pixel circuit 106 is a four-in-one pixel circuit, with one pixel circuit 106 corresponding to four sub-pixels.

[0117] Compared to individual pixel circuits, such as Figure 10 As shown, the four-in-one pixel circuit mainly replicates the photodiode 120 and the seventh transistor T7, then connects the four sets of photodiodes 120 and the seventh transistor T7 in parallel, and then connects them to the first terminal of the eighth transistor T8 and the control terminal of the ninth transistor T9.

[0118] The division method of the two groups of photodiodes 120 can be selected by those skilled in the art according to the actual situation, and no specific restrictions are made here.

[0119] For example, such as Figure 10As shown, the pixel circuit 106 includes four photodiodes 120, which are respectively labeled as first diode D1, second diode D2, third diode D3 and fourth diode D4. First diode D1 and third diode D3 are grouped together, and second diode D2 and fourth diode D4 are grouped together.

[0120] According to the image sensor 100 of this application embodiment, each pixel circuit 106 includes four photodiodes 120, the four photodiodes 120 include two groups of photodiodes 120, and each group of photodiodes 120 includes two photodiodes 120. In this way, optical flow detection within the four-in-one pixel can be realized, thereby realizing motion detection function based on the four-in-one pixel.

[0121] According to some embodiments of this application, optionally, such as Figure 11 As shown, each pixel circuit 106 includes 9 photodiodes 120, the 9 photodiodes 120 include two groups of photodiodes 120, and each group of photodiodes 120 includes 3 photodiodes 120.

[0122] That is, the pixel circuit 106 is a nine-in-one pixel circuit, with one pixel circuit 106 corresponding to nine sub-pixels.

[0123] Compared to individual pixel circuits, such as Figure 11 As shown, the nine-in-one pixel circuit mainly replicates the photodiode 120 and the seventh transistor T7, then connects the nine sets of photodiodes 120 and the seventh transistor T7 in parallel, and then connects them to the first terminal of the eighth transistor T8 and the control terminal of the ninth transistor T9.

[0124] The division method of the two groups of photodiodes 120 can be selected by those skilled in the art according to the actual situation, and no specific restrictions are made here.

[0125] For example, such as Figure 11 As shown, the pixel circuit 106 includes nine photodiodes 120, which are respectively labeled as first diode D1, second diode D2, third diode D3, fourth diode D4, fifth diode D5, sixth diode D6, seventh diode D7, eighth diode D8 and ninth diode D9. First diode D1, fourth diode D4 and seventh diode D7 are grouped together, and third diode D3, sixth diode D6 and ninth diode D9 are grouped together.

[0126] According to the image sensor 100 of this application embodiment, each pixel circuit 106 includes nine photodiodes 120, the nine photodiodes 120 include two groups of photodiodes 120, and each group of photodiodes 120 includes three photodiodes 120. In this way, optical flow detection within the nine-in-one pixel can be realized, thereby realizing motion detection function based on the nine-in-one pixel.

[0127] According to some embodiments of this application, optionally, such as Figure 12 As shown, each pixel circuit 106 includes 16 photodiodes 120, and the 16 photodiodes 120 include two groups of photodiodes 120, with each group of photodiodes 120 including 8 photodiodes 120.

[0128] That is, the pixel circuit 106 is a 16-in-1 pixel circuit, such as a 16-in-1 (Hex Binning, hexagonal arrangement) pixel circuit, where one pixel circuit 106 corresponds to 16 sub-pixels.

[0129] Compared to individual pixel circuits, such as Figure 12 As shown, the sixteen-in-one pixel circuit mainly replicates the photodiode 120 and the seventh transistor T7, then connects sixteen sets of photodiodes 120 and the seventh transistor T7 in parallel, and then connects them to the first terminal of the eighth transistor T8 and the control terminal of the ninth transistor T9.

[0130] The division method of the two groups of photodiodes 120 can be selected by those skilled in the art according to the actual situation, and no specific restrictions are made here.

[0131] For example, such as Figure 12 As shown, the pixel circuit 106 includes 16 photodiodes 120, which are respectively designated as diode D1, diode D2, diode D3, diode D4, diode D5, diode D6, diode D7, diode D8, diode D9, diode D10, diode D11, diode D12, diode D13, diode D14, diode D15, and diode D16. Diodes D1, D2, D5, D6, D9, D10, D13, and D14 are grouped together, and diodes D3, D4, D7, D8, D11, D12, D12, D15, and D16 are grouped together.

[0132] According to the image sensor 100 of this application embodiment, each pixel circuit 106 includes 16 photodiodes 120, the 16 photodiodes 120 include two groups of photodiodes 120, and each group of photodiodes 120 includes 8 photodiodes 120. In this way, optical flow detection can be realized within a 16-in-1 pixel, such as a 16-in-1 (Hex Binning) pixel, thereby realizing motion detection function based on 16-in-1 pixels.

[0133] According to some embodiments of this application, optionally, such as Figure 13 and Figure 14 As shown, the input terminal of each probe sub-circuit 122 is connected to all the photodiodes 120 in a set of photodiodes 120.

[0134] In practical applications, the grouping method of photodiodes 120 and the correspondence between the two probe sub-circuits 122 and the two groups of photodiodes 120 can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0135] According to the image sensor 100 of this application embodiment, the input terminal of each probe sub-circuit 122 is connected to all the photodiodes 120 in a group of photodiodes 120. In this way, the signal amplitude input to the probe sub-circuit 122 is larger, the signal change during the optical flow detection process is more obvious, and the accuracy of optical flow detection is improved.

[0136] According to some embodiments of this application, optionally, such as Figure 15 As shown, the input terminal of each probe sub-circuit 122 is connected to one of the photodiodes 120 in a set of photodiodes 120.

[0137] In practical applications, those skilled in the art can set the grouping method of photodiodes 120, the correspondence between the two probe sub-circuits 122 and the two groups of photodiodes 120, and the position of the photodiodes 120 actually connected to each probe sub-circuit 122 according to the actual situation, and no specific restrictions are made here.

[0138] According to the image sensor 100 of the embodiments of this application, the input terminal of each probe sub-circuit 122 is connected to one of the photodiodes 120 in a group of photodiodes 120. In this way, one probe sub-circuit 122 is connected to one photodiode 120, which simplifies the structure and reduces the wiring difficulty in circuit design.

[0139] According to some embodiments of this application, optionally, in each probe circuit 108, the two photodiodes 120 connected to the two probe sub-circuits 122 are staggered in both the row and column directions.

[0140] That is, the two photodiodes 120 connected to the two probe sub-circuits 122 in each probe circuit 108 are neither in the same row nor in the same column. In other words, the two photodiodes 120 connected to the two probe sub-circuits 122 in each probe circuit 108 are diagonally distributed in the two sets of photodiodes 120.

[0141] For example, such as Figure 10 As shown, for the four photodiodes 120 in the four-in-one pixel circuit 106, the first diode D1 and the third diode D3 are grouped together, and the second diode D2 and the fourth diode D4 are grouped together. In this case, in each probe circuit 108, the combination of the two photodiodes 120 connected to the two probe sub-circuits 122 may include: the first diode D1 and the fourth diode D4, or the second diode D2 and the third diode D3.

[0142] For example, such as Figure 11 As shown, for the nine photodiodes 120 in the nine-in-one pixel circuit 106, the first diode D1, the fourth diode D4, and the seventh diode D7 are grouped together, and the third diode D3, the sixth diode D6, and the ninth diode D9 are grouped together. In this case, in each probe circuit 108, the combination of two photodiodes 120 connected to the two probe sub-circuits 122 may include: first diode D1 and sixth diode D6, first diode D1 and ninth diode D9, third diode D3 and fourth diode D4, fourth diode D4 and ninth diode D9, seventh diode D7 and third diode D3, and seventh diode D7 and sixth diode D6.

[0143] For example, such as Figure 12As shown, for the 16 photodiodes 120 in the 16-in-1 pixel circuit 106, the first diode D1, the second diode D2, the fifth diode D5, the sixth diode D6, the ninth diode D9, the tenth diode D10, the thirteenth diode D13 and the fourteenth diode D14 are grouped together, and the third diode D3, the fourth diode D4, the seventh diode D7, the eighth diode D8, the eleventh diode D11, the twelfth diode D12, the fifteenth diode D15 and the sixteenth diode D16 are grouped together. At this time, in each probe circuit 108, the combination of the two photodiodes 120 connected to the two probe sub-circuits 122 may include: a combination of the first diode D1 or the second diode D2 with any one of the seventh diode D7, the eighth diode D8, the eleventh diode D11, the twelfth diode D12, the fifteenth diode D15, and the sixteenth diode D16; a combination of the fifth diode D5 or the sixth diode D6 with any one of the third diode D3, the fourth diode D4, the eleventh diode D11, the twelfth diode D12, the fifteenth diode D15, and the sixteenth diode D16; a combination of the ninth diode D9 or the tenth diode D10 with any one of the third diode D3, the fourth diode D4, the seventh diode D7, the eighth diode D8, the fifteenth diode D15, and the sixteenth diode D16; or a combination of the thirteenth diode D13 or the fourteenth diode D14 with any one of the third diode D3, the fourth diode D4, the seventh diode D7, the eighth diode D8, the eleventh diode D11, and the twelfth diode D12.

[0144] In practical applications, those skilled in the art can set the grouping method of photodiodes 120, the correspondence between the two probe sub-circuits 122 and the two groups of photodiodes 120, and the position of the photodiodes 120 actually connected to each probe sub-circuit 122 according to the actual situation, and no specific restrictions are made here.

[0145] According to the image sensor 100 of this application embodiment, in each probe circuit 108, the two photodiodes 120 connected to the two probe sub-circuits 122 are staggered in both the row and column directions. In this way, optical flow detection can be performed on the two diagonally distributed photodiodes 120 within each composite pixel, simultaneously achieving optical flow detection in both the row and column directions, thereby simultaneously achieving motion detection in both the row and column directions and improving the accuracy of the detected motion information.

[0146] According to some embodiments of this application, optionally, both the first optical flow comparison signal and the second optical flow comparison signal are binary values.

[0147] Specifically, both the first optical flow signal and the second optical flow signal are voltage signals. When the voltage amplitude of the first optical flow signal is greater than that of the second optical flow signal, the first optical flow comparison signal is a high-level signal (binary signal 1), and the second optical flow comparison signal is a low-level signal (binary signal 0). When the voltage amplitude of the first optical flow signal is less than that of the second optical flow signal, the first optical flow comparison signal is a low-level signal (binary signal 0), and the second optical flow comparison signal is a high-level signal (binary signal 1). When the voltage amplitude of the first optical flow signal is equal to that of the second optical flow signal, both the first and second optical flow comparison signals are low-level signals (binary signal 0).

[0148] According to the image sensor 100 of this application embodiment, both the first optical flow signal and the second optical flow signal are voltage signals. When the voltage amplitude of the first optical flow signal is greater than the voltage amplitude of the second optical flow signal, the first optical flow comparison signal is a high-level signal and the second optical flow comparison signal is a low-level signal. When the voltage amplitude of the first optical flow signal is less than the voltage amplitude of the second optical flow signal, the first optical flow comparison signal is a low-level signal and the second optical flow comparison signal is a high-level signal. When the voltage amplitude of the first optical flow signal is equal to the voltage amplitude of the second optical flow signal, both the first and second optical flow comparison signals are low-level signals. This allows the optical flow difference data to be generated and output in single-bit binary form, reducing the power consumption of generating optical flow difference subframes and facilitating an increase in the number of generated optical flow difference subframes to obtain more accurate motion information and achieve fine-grained restoration of image information.

[0149] According to some embodiments of this application, optionally, such as Figure 9 , Figure 13 , Figure 14 as well as Figure 15 As shown, the probe sub-circuit 122 includes a first transistor T1, a second transistor T2, a third transistor T3, a fourth transistor T4, a fifth transistor T5, and a sixth transistor T6.

[0150] In this configuration, the control terminal of the first transistor T1 is connected to the cathode of the photodiode 120, the first terminal of the first transistor T1 is connected to the power supply 124, the second terminal of the first transistor T1 is connected to the first terminal of the second transistor T2 and the first terminal of the third transistor T3, and the second terminal of the third transistor T3 and the anode of the photodiode 120 are both grounded.

[0151] Optionally, the first transistor T1 functions as an amplifier, monitoring and transmitting the voltage at the cathode of the photodiode 120. The voltage at the second terminal of the first transistor T1 changes with the voltage at the cathode of the photodiode 120. The first transistor T1 also serves to prevent charge overflow within the photodiode 120.

[0152] Optionally, the second terminal of the second transistor T2 is connected to the second terminal of the fourth transistor T4 and the control terminal of the fifth transistor T5, respectively, and the first terminal of the fourth transistor T4 and the first terminal of the fifth transistor T5 are both connected to the power supply 124.

[0153] A first floating diffusion capacitor C is formed between the second terminal of the second transistor T2, the second terminal of the fourth transistor T4, and the control terminal of the fifth transistor T5. FLUX .

[0154] Among them, the first floating diffusion capacitor C FLUX Specifically, it can be a total capacitance node formed by the coupling of multiple parasitic capacitances. In this embodiment, the first floating diffusion capacitance C can be adjusted by increasing the physical size of the second terminal of the second transistor T2, i.e., the drain diffusion region. FLUX .

[0155] Optionally, the first floating diffusion capacitor C FLUX The first transistor T1 is used to store the voltage at its second terminal. The second transistor T2 acts as a switch to control whether the voltage at the second terminal of the first transistor T1 affects the first floating diffusion capacitor C. FLUX Charging to control the first floating diffusion capacitor C FLUX Voltage changes.

[0156] Optionally, the third transistor T3 is used to connect its connection node to ground to reset the potential of its connection node and associated node, thereby eliminating residual signal interference between different frames and ensuring the orderly and accurate execution of optical flow detection between different frames.

[0157] Optionally, the fourth transistor T4 acts as a switch to control whether the power supply 124 is connected to the circuit. When the fourth transistor T4 is turned on, the power supply 124 can control the first floating diffusion capacitor C. FLUX The charge inside is cleared, thereby achieving a reset.

[0158] Optionally, the second terminal of the fifth transistor T5 is connected to the first terminal of the sixth transistor T6, and the second terminal of the sixth transistor T6 is connected to the pixel optical flow comparator 114.

[0159] The fifth transistor, T5, also functions as an amplifier. It monitors and transmits the first floating diffusion capacitance C.FLUX The voltage is used to generate the aforementioned optical flow signal.

[0160] Optionally, the sixth transistor T6 acts as a switch to control whether the optical flow signal is transmitted to the corresponding pixel optical flow comparator 114, so as to realize the line-by-line comparison of the optical flow signal.

[0161] The timing diagrams for the second transistor T2, the fourth transistor T4, and the sixth transistor T6 are as follows: Figure 6 As shown. Based on this, for a complete optical flow detection process, the working process of probe sub-circuit 122 may specifically include the following steps:

[0162] Step 1: The fourth transistor T4 is closed, allowing power supply 124 to charge the first floating diffusion capacitor C. FLUX The internal charge is cleared.

[0163] Step 2: The fourth transistor T4 is turned off, and the second transistor T2 is turned on, so that the first transistor T1 can control the first floating diffusion capacitor C. FLUX Charging is performed to reflect the voltage on photodiode 120. Since the first transistor T1 directly reads the instantaneous voltage on photodiode 120, the magnitude of the optical current of photodiode 120 is directly reflected by the first transistor T1's action on the first floating diffusion capacitor C. FLUX After charging, the first floating diffusion capacitor C FLUX The magnitude of the voltage on the photodiode 120. When the light flow changes due to the movement of an object, the first floating diffusion capacitor C is charged. FLUX The voltage on the circuit will also change accordingly. If the optical flow remains constant, the first floating diffusion capacitor C will change after charging. FLUX The voltage on it remains unchanged.

[0164] Step 3: The second transistor T2 is turned off, and the first floating diffusion capacitor C... FLUX Charging complete. The sixth transistor T6 closes, causing the fifth transistor T5 to charge the first floating diffusion capacitor C. FLUX The voltage reading on the capacitor is converted into an optical flow signal and output, completing this optical flow detection. Therefore, the optical flow signal is the first floating diffusion capacitor C after charging. FLUX The magnitude of the voltage on the photodiode 120 and the change in the optical flow signal directly reflect the change in the optical flow on the photodiode 120.

[0165] Based on this, taking pixel circuit 106 as an example of a single pixel circuit, the workflow of image sensor 100 within one frame may specifically include the following steps:

[0166] Step 1: During the reset period, the row driver 132 generates a reset signal to sequentially close the eighth transistor T8 in each pixel circuit 106, so that the power supply 124 clears the second floating diffusion capacitor C in the pixel circuit 106. FD The charge inside, and the charge in photodiode 120 is cleared by the seventh transistor T7.

[0167] Step 2: During the exposure period, the seventh transistor T7 in each pixel circuit 106 is turned off. The probe circuit 108 sequentially performs optical flow detection on each row of pixel circuits 106, and the detected optical flow signal is input into the pixel optical flow comparison module 110. After each pair of all pixel circuits 106 completes one optical flow detection, the pixel optical flow comparison module 110 outputs a set of optical flow difference data. During the exposure period, the pixel optical flow comparison module 110 sequentially outputs multiple sets of optical flow difference data to generate multiple optical flow difference subframes.

[0168] Step 3: During the reading period, the probe circuit 108 and the pixel optical flow comparison module 110 stop working. The row driver 132 scans the pixel circuit 106 row by row, causing the pixel circuit 106 to output electrical signals. After analog-to-digital conversion by the column parallel analog-to-digital converter module 126, the electrical signals are digitized into image data and scanned and output by the column scanner 130 to generate image frames.

[0169] According to an embodiment of the image sensor 100 of this application, the probe sub-circuit 122 includes a first transistor T1, a second transistor T2, a third transistor T3, a fourth transistor T4, a fifth transistor T5, and a sixth transistor T6. The control terminal of the first transistor T1 is connected to the photodiode 120, and the first terminal of the first transistor T1 is connected to the power supply 124. The first terminal of the second transistor T2 is connected to the second terminal of the first transistor T1. The first terminal of the third transistor T3 is connected to the second terminal of the first transistor T1, and the second terminal of the third transistor T3 is grounded. The first terminal of the fourth transistor T4 is connected to the power supply 124, and the second terminal of the fourth transistor T4 is connected to the second terminal of the second transistor T2. The control terminal of the fifth transistor T5 is connected to the second terminal of the second transistor T2, and the first terminal of the fifth transistor T5 is connected to the power supply 124. The first terminal of the sixth transistor T6 is connected to the second terminal of the fifth transistor T5, and the second terminal of the sixth transistor T6 is connected to the pixel optical flow comparator 114. A first floating diffusion capacitor C is formed between the second terminal of the second transistor T2, the second terminal of the fourth transistor T4, and the control terminal of the fifth transistor T5. FLUX This allows for the monitoring of optical flow changes in different pixel circuits 106 and the output of optical flow signals line by line, enabling the image sensor 100 to detect motion information of the shooting scene while acquiring image information of the shooting scene.

[0170] According to some embodiments of this application, optionally, such as Figure 21 As shown, this application embodiment also provides a shooting module 200. The shooting module 200 includes the image sensor 100 from any of the above embodiments. The shooting module 200 provided in this application embodiment includes the image sensor 100 from any of the above embodiments and achieves the same technical effect; therefore, to avoid repetition, it will not be described again here.

[0171] According to some embodiments of this application, optionally, such as Figure 22 As shown in the figure, this application embodiment also provides an electronic device 300. The electronic device 300 includes the shooting module 200 in the above embodiments. The electronic device 300 provided in this application embodiment includes the shooting module 200 in the above embodiments and can achieve the same technical effect; therefore, to avoid repetition, it will not be described again here.

[0172] It should be noted that the electronic device 300 in the embodiments of this application includes mobile electronic devices and non-mobile electronic devices.

[0173] In practical applications, the electronic device 300 can be a terminal or other devices besides a terminal. For example, the electronic device 300 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a personal computer (PC), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit it.

[0174] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0175] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An image sensor, characterized in that, include: A pixel array comprising multiple pixel units, each pixel unit comprising: Pixel circuits are used to convert optical signals into electrical signals; and A probe circuit, connected to the pixel circuit, is used to monitor the optical flow changes of the pixel circuit and output an optical flow signal during the exposure period of the pixel circuit. The pixel optical flow comparison module is connected to the probe circuit in each column of pixel units. The pixel optical flow comparison module is used to generate multiple sets of optical flow difference data based on the optical flow signals output by the multiple pixel units during the exposure period of the pixel circuit. Each set of optical flow difference data is used to generate an optical flow difference subframe. The optical flow difference subframe is used to indicate the position information of the moving object at different times. The signal processing module is connected to the pixel circuit in each column of pixel units. The signal processing module is used to generate image data based on the electrical signals output by the multiple pixel units. The image data is used to generate image frames.

2. The image sensor according to claim 1, characterized in that, The pixel optical flow comparison module includes: Multiple pixel optical flow comparators, each pixel optical flow comparator is connected to a probe circuit in at least one column of pixel units, the pixel optical flow comparators are used to compare the optical flow signals output by the probe circuit in at least one pixel unit row by row, and output a first optical flow comparison signal and a second optical flow comparison signal; A first data scanner, connected to an optical flow comparator for each pixel, is used to scan the first optical flow comparison signal and output the first optical flow difference data. The second data scanner, connected to the optical flow comparator of each pixel, is used to scan the second optical flow comparison signal and output the second optical flow difference data; Each set of optical flow difference data includes the first optical flow difference sub-data and the second optical flow difference sub-data.

3. The image sensor according to claim 2, characterized in that, Each pixel circuit includes a photodiode; Each probe circuit includes a probe sub-circuit, the input of which is connected to the photodiode, and the output of which is connected to two adjacent pixel optical flow comparators. Each pixel optical flow comparator is connected to two adjacent columns of probe sub-circuits. The pixel optical flow comparator is used to compare the first optical flow signal and the second optical flow signal output by the two adjacent probe sub-circuits row by row, and output the first optical flow comparison signal and the second optical flow comparison signal.

4. The image sensor according to claim 3, characterized in that, Each pixel circuit includes N photodiodes, the N photodiodes are divided into two groups of photodiodes, and each group of photodiodes includes M photodiodes, where N and M are integers, N≥2, M≥1; Each probe circuit includes two probe sub-circuits. The input terminal of each probe sub-circuit is connected to a set of photodiodes, and the output terminal of each probe sub-circuit is connected to the pixel optical flow comparator. Each pixel optical flow comparator is connected to two probe sub-circuits in each column of probe circuits. The pixel optical flow comparator is used to compare the first optical flow signal and the second optical flow signal output by the two probe sub-circuits in each probe circuit row by row, and output the first optical flow comparison signal and the second optical flow comparison signal.

5. The image sensor according to claim 4, characterized in that, Each pixel circuit includes four photodiodes, which are divided into two groups of two photodiodes.

6. The image sensor according to claim 4, characterized in that, Each pixel circuit includes nine photodiodes, which are divided into two groups of three photodiodes.

7. The image sensor according to claim 4, characterized in that, Each pixel circuit includes 16 photodiodes, which are divided into two groups of 8 photodiodes.

8. The image sensor according to claim 4, characterized in that, The input terminal of each probe sub-circuit is connected to all the photodiodes in a set of photodiodes.

9. The image sensor according to claim 4, characterized in that, The input terminal of each probe sub-circuit is connected to one of the photodiodes in a set of photodiodes.

10. The image sensor according to claim 9, characterized in that, In each probe circuit, the two photodiodes connected to the two probe sub-circuits are staggered in both the row and column directions.

11. The image sensor according to any one of claims 3 to 10, characterized in that, Both the first optical flow signal and the second optical flow signal are voltage signals. When the voltage amplitude of the first optical flow signal is greater than the voltage amplitude of the second optical flow signal, the first optical flow comparison signal is a high-level signal and the second optical flow comparison signal is a low-level signal. When the voltage amplitude of the first optical flow signal is less than the voltage amplitude of the second optical flow signal, the first optical flow comparison signal is a low-level signal and the second optical flow comparison signal is a high-level signal. When the voltage amplitude of the first optical flow signal is equal to the voltage amplitude of the second optical flow signal, both the first optical flow comparison signal and the second optical flow comparison signal are low-level signals.

12. The image sensor according to any one of claims 3 to 10, characterized in that, Each probe subcircuit includes: A first transistor, the control terminal of the first transistor is connected to the photodiode, and the first terminal of the first transistor is connected to a power supply; The second transistor has a first terminal connected to the second terminal of the first transistor; A third transistor, wherein the first terminal of the third transistor is connected to the second terminal of the first transistor, and the second terminal of the third transistor is grounded; A fourth transistor, wherein a first terminal of the fourth transistor is connected to the power supply, and a second terminal of the fourth transistor is connected to the second terminal of the second transistor; The fifth transistor has its control terminal connected to the second terminal of the second transistor, and its first terminal connected to the power supply. The sixth transistor, the first terminal of which is connected to the second terminal of the fifth transistor, and the second terminal of the sixth transistor is connected to the pixel optical flow comparator; A first floating diffusion capacitor is formed between the second terminal of the second transistor, the second terminal of the fourth transistor, and the control terminal of the fifth transistor.

13. A shooting module, characterized in that, The image sensor included in any one of claims 1 to 12.

14. An electronic device, characterized in that, Includes the shooting module as described in claim 13.

Images