Pixel data processing method and image sensor

Abstract

The present disclosure provides a pixel data processing method and an image sensor, the pixel data processing method comprising: in the exposure phase of any frame image of the image sensor, storing the pixel data of each pixel of the frame image in the corresponding signal storage unit; in the data reading phase of the frame image, performing pixel merging binning processing on the pixel data of each designated pixel group of the frame image, wherein the designated pixel group includes at least one group of multiple pixels corresponding to the same color. According to the technical solution of the present disclosure, it is possible to reduce the amount of data of the output pixel data, so as to save the storage resources of the digital signal corresponding to the pixel data, and reduce the amount of processing operations of the digital signal corresponding to the pixel data.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a pixel data processing method and an image sensor. Background Art

[0002] With the development of image sensor technology, in order to achieve higher definition effects, the number of pixels in a single image is increasing. When the number of image pixels increases to a certain extent, it limits the improvement of frame rate to a certain extent, and also limits the improvement of processing efficiency of image pixel data. Summary of the invention

[0003] The present disclosure provides a pixel data processing method and an image sensor.

[0004] In a first aspect, the present disclosure provides a pixel data processing method, which is applied to an image sensor. The pixel data processing method includes:

[0005] During the exposure phase of any frame image of the image sensor, the pixel data of each pixel of the frame image is stored in the corresponding signal storage unit;

[0006] During the data readout phase of the frame image, pixel data of each designated pixel group of the frame image is subjected to a pixel merging binning process, wherein the designated pixel group includes at least one group of multiple pixels corresponding to the same color.

[0007] In a second aspect, the present disclosure provides an image sensor, the image sensor comprising a pixel array, the pixel array comprising a plurality of pixels, each column of pixels in each odd-numbered row corresponding to the same color, each column of pixels in each even-numbered row corresponding to the same color, the plurality of pixels being divided into a plurality of designated pixel groups, the designated pixel groups comprising at least one group of a plurality of pixels corresponding to the same color;

[0008] The pixels include:

[0009] A photoelectric conversion unit, used for outputting pixel data of the pixels of any frame image of the image sensor to a signal storage unit during an exposure phase of the frame image;

[0010] A signal storage unit, used for storing pixel data of the pixels of the frame image;

[0011] An output unit, used for outputting the pixel data of the pixel of the frame image stored in the corresponding signal storage unit during the data reading phase of the frame image;

[0012] The image sensor further includes a signal merging unit, which is used to perform a pixel merging binning process on the pixel data of the designated pixel group during the data readout phase of the frame image.

[0013] According to the pixel data processing method and the technical solution of the image sensor provided by the embodiments of the present disclosure, when performing pixel exposure and readout inside the image sensor, the pixel data of the image pixels are subjected to pixel merging and binning processing before being output. On the one hand, the frame rate of the image sensor can be effectively improved while ensuring image clarity, and the processing efficiency of the pixel data can be improved, which is conducive to realizing high-speed camera of the image sensor. On the other hand, the amount of output pixel data can be reduced to save storage resources for digital signals corresponding to the pixel data, reduce the area of ​​the memory unit used to store the digital signals corresponding to the pixel data, and reduce the amount of processing operations for the digital signals corresponding to the pixel data, so as to cope with the growing demand for chip miniaturization.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 A flowchart of a pixel data processing method provided by an embodiment of the present disclosure;

[0017] Figure 2 A schematic diagram of a division method of a specified pixel group in a pixel array of an image sensor;

[0018] Figure 3 This is a flowchart of a specific implementation of step S12 in the embodiment of the present disclosure;

[0019] Figure 4 This is a flowchart of a specific implementation of step S32 in the embodiment of the present disclosure;

[0020] Figure 5 It is a structural schematic diagram of a column-level single-slope ADC;

[0021] Figure 6 A schematic diagram of the working waveform of a column-level single-slope ADC;

[0022] Figure 7 It is a schematic diagram of the working waveform of another column-level single-slope ADC;

[0023] Figure 8 A schematic diagram of a division method of a specified pixel group in a pixel array of another image sensor;

[0024] Fig. 9 A schematic diagram of a method for reading pixel data of a frame of image after pixel merging and binning processing;

[0025] Fig.10 A schematic diagram of a division method of a specified pixel group in a pixel array of another image sensor;

[0026] Fig.11 A schematic diagram of another method for reading pixel data of a frame of image after pixel merging and binning processing;

[0027] Fig.12 A schematic diagram of a division method of a specified pixel group in a pixel array of another image sensor;

[0028] Fig.13 A schematic diagram of another method for reading pixel data of a frame of image after pixel merging and binning processing;

[0029] Fig.14 A schematic diagram of a circuit structure of a pixel in an image sensor provided by an embodiment of the present disclosure;

[0030] Fig.15 A schematic diagram of a circuit structure of a pixel in another image sensor provided by an embodiment of the present disclosure;

[0031] Fig.16 A schematic diagram of a circuit structure of a pixel in another image sensor provided by an embodiment of the present disclosure;

[0032] Fig.17 A schematic diagram of a circuit structure of a pixel in another image sensor provided by an embodiment of the present disclosure;

[0033] Fig.18 A schematic diagram of a circuit timing of a pixel in an image sensor provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0035] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0036] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0037] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0038] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0039] In the related art, in order to alleviate the contradiction between clarity and frame rate, a pixel data processing method of pixel merging binning mode has been developed. The commonly used binning method is implemented by digital processing after the image sensor collects pixel data and outputs digital signals, that is, through an external memory, such as static random access memory (SRAM), to store the digital code values ​​of multiple rows of pixel signals of the image, and then perform pixel data merging processing on the digital code values ​​of the pixel signals. However, since storing the digital code values ​​of pixel signals requires a large amount of SRAM storage unit area, in the context of the development of increasingly larger pixel arrays, the area and cost issues will become more and more prominent, and for portable wearable devices such as mobile phones, the internal space is small, and the area will restrict the development of chip design.

[0040] In order to effectively solve the problems existing in the above-mentioned related technologies, the embodiments of the present disclosure provide a pixel data processing method and an image sensor.

[0041] Figure 1 A flowchart of a pixel data processing method provided in an embodiment of the present disclosure.

[0042] The present disclosure provides a pixel data processing method, which can be applied to an image sensor, wherein the image sensor includes a pixel array composed of a plurality of pixels arranged in an array, for example, the pixel array includes N rows and M columns of pixels, where N and M are both positive integers, that is, any frame of image captured by the image sensor is composed of a plurality of pixels arranged in an array, wherein each column of pixels in each odd-numbered row has the same color, and each column of pixels in each even-numbered row has the same color, and the plurality of pixels are divided into a plurality of designated pixel groups, wherein each pixel includes a signal storage unit, and the signal storage unit is used to store pixel data of the corresponding pixel, such as Figure 1 As shown, the pixel data processing method includes:

[0043] Step S11 : During the exposure phase of any frame image of the image sensor, the pixel data of each pixel of the frame image is stored in a corresponding signal storage unit.

[0044] Step S12: During the data reading phase of the frame image, pixel data of each designated pixel group of the frame image is subjected to a pixel merging binning process, wherein each designated pixel group includes at least one group of multiple pixels corresponding to the same color.

[0045] According to the technical solution of the pixel data processing method of the embodiment of the present disclosure, when pixel exposure and readout are performed inside the image sensor, the pixel data of the image pixels are subjected to pixel merging and binning processing before output. On the one hand, the frame rate of the image sensor can be effectively improved while ensuring image clarity, and the processing efficiency of the pixel data can be improved, which is conducive to realizing high-speed camera of the image sensor. On the other hand, the amount of output pixel data can be reduced to save storage resources for digital signals corresponding to the pixel data, reduce the area of ​​the memory unit used to store the digital signals corresponding to the pixel data, and reduce the amount of processing operations for the digital signals corresponding to the pixel data, so as to cope with the growing demand for chip miniaturization.

[0046] In some embodiments, the image sensor collects image information based on a global shutter exposure method, and the image information includes pixel data of image pixels.

[0047] In some embodiments, in the exposure phase of any frame of image pixels, before storing the pixel data of each pixel of the frame of image into the corresponding signal storage unit, it also includes: performing exposure processing on the frame of image pixels based on a global shutter exposure method. In the exposure method based on a global shutter, all image pixels of the frame of image are exposed at the same time.

[0048] In some embodiments, the data readout stage of the frame image is located in the exposure stage of the frame image after the frame image. In this embodiment, in the exposure stage of the first frame image pixel, the first frame image pixel is exposed based on the exposure method of the global shutter, and the pixel data of each pixel of the first frame image is stored in the corresponding signal storage unit; then, the next frame operation is entered; in the exposure stage of the nth (n≥2) frame image pixel, the nth frame image pixel is exposed based on the exposure method of the global shutter, and at the same time, the data readout stage of the n-1th frame image is entered, and the pixel data of each specified pixel group of the n-1th frame image is binned and read out.

[0049] In this embodiment, different from the traditional global shutter exposure method, after exposing the image pixels of the n-1 frame, the pixel data of each designated pixel group of the n-1 frame image is subjected to pixel merging binning processing and readout, while the image pixels of the n-1 frame are exposed, that is, the exposure process of the n-1 frame and the readout process of the pixel data of the n-1 frame are carried out in parallel, so the working time of the n-1 frame can be determined according to the maximum value of the exposure time of the n-1 frame and the readout time of the pixel data of the n-1 frame, that is, the working time of the n-1 frame Row_time=max{exposure time of the n-1 frame, readout time of the pixel data of the n-1 frame}, while in the traditional global shutter exposure method, the working time of the n-1 frame Row_time=exposure time of the n-1 frame+readout time of the pixel data of the n-1 frame. It can be seen that compared with the traditional global shutter exposure method, this embodiment can effectively reduce the working time of each frame in the image sensor, which is conducive to improving the frame rate of the image sensor and realizing high-speed photography of the image sensor.

[0050] In some embodiments, the pixel data includes a pixel reset voltage (pixel V RST ) and the pixel exposure voltage (pixel V SIG ); In the image sensor, each pixel includes a corresponding signal storage unit, and the signal storage unit has a first storage capacitor and a second storage capacitor. The first storage capacitor is used to store a pixel reset voltage of the pixel, and the second storage capacitor is used to store a pixel exposure voltage generated after the pixel is exposed.

[0051] In some embodiments, in step S11, pixel data of each pixel of the frame image is stored in a corresponding signal storage unit, including: storing a pixel reset voltage of each pixel of the frame image in a first storage capacitor in the corresponding signal storage unit; and storing a pixel exposure voltage of each pixel of the frame image in a second storage capacitor in the corresponding signal storage unit.

[0052] In some embodiments, in order to realize that the exposure process of the nth frame and the readout process of the pixel data of the n-1th frame are performed in parallel to reduce the working time of each frame, in the exposure stage of each frame image pixel, when the frame image is the 1st frame image, after the exposure process of the frame image pixels is performed, the pixel data of the frame image pixels are first stored in the corresponding storage capacitor, so that in the exposure stage of the next frame image pixels, the pixel data of the frame image pixels are binned and read out. When the frame image is the nth frame image, while the exposure process of the frame image pixels is performed, the pixel data of the n-1th frame image pixels are binned and read out, and then the pixel data of the frame image pixels are stored in the corresponding storage capacitor.

[0053] Specifically, in the exposure phase of the first frame image pixels, the first frame image pixels are exposed, and the pixel reset voltage of each pixel of the first frame image is stored in the corresponding first storage capacitor, and the pixel exposure voltage generated after each pixel of the first frame image is exposed is stored in the corresponding second storage capacitor; in the exposure phase of the nth frame image pixels, while the nth frame image pixels are exposed, the pixel data of the n-1th frame image pixels are read from the first storage capacitor and the second storage capacitor corresponding to each pixel in each row of pixels, and then the pixel reset voltage of each pixel of the nth frame image is stored in the corresponding first storage capacitor, and the pixel exposure voltage generated after each pixel of the nth frame image is exposed is stored in the corresponding second storage capacitor. That is, the reading of the pixel data of the n-1th frame image pixels is performed before the storage of the pixel data of the nth frame image pixels, thereby effectively preventing the pixel data of the n-1th frame image pixels from affecting the storage of the pixel data of the nth frame image pixels.

[0054] In some embodiments, every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and every two adjacent even-numbered rows of pixels constitute a designated pixel group. Figure 2 FIG. 1 is a schematic diagram of a division method of a specified pixel group in a pixel array of an image sensor, for example, Figure 2As shown, the first row of pixels R1 and the third row of pixels R3 adjacent to each other in the column direction constitute a designated pixel group, the second row of pixels R2 and the fourth row of pixels R4 adjacent to each other in the column direction constitute a designated pixel group, and so on, the pixel array can be divided into multiple designated pixel groups. Among them, in the two rows of pixels in each designated pixel group, the pixel colors corresponding to the pixels located in the same column are the same, the pixel colors corresponding to the pixels located in the odd columns are the same, and the pixel colors corresponding to the pixels located in the even columns are the same. For example, in the designated pixel group composed of the first row of pixels R1 and the third row of pixels R3, the pixel colors corresponding to the pixels located in the first column C1 are all blue B, and the pixel colors corresponding to the pixels located in the second column C2 are all green G; in the designated pixel group composed of the second row of pixels R2 and the fourth row of pixels R4, the pixel colors corresponding to the pixels located in the first column C1 are all green G, and the pixel colors corresponding to the pixels located in the second column C2 are all red R.

[0055] In some embodiments, when every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and every two adjacent even-numbered rows of pixels constitute a designated pixel group, in the process of reading out the pixel data of any frame, the pixel data of each designated pixel group of the frame image is read out in sequence, and the pixel data of each designated pixel group of the frame image is subjected to pixel merging binning processing. Figure 2 As shown, in the process of reading out the pixel data of any frame, from top to bottom in the column direction, first, the pixel data of each row of pixels in the designated pixel group composed of the first row of pixels R1 and the third row of pixels R3 are read out from the signal storage units of the first row of pixels R1 and the third row of pixels R3, and the pixel data of the first row of pixels R1 and the third row of pixels R3 are subjected to pixel merging binning processing; then, the pixel data of each row of pixels in the designated pixel group composed of the second row of pixels R2 and the fourth row of pixels R4 are read out from the signal storage units of the second row of pixels R2 and the fourth row of pixels R4, and the pixel data of the second row of pixels R2 and the fourth row of pixels R4 are subjected to pixel merging binning processing; and so on, pixel data reading and pixel merging binning processing are performed on each designated pixel group in turn.

[0056] Figure 3 is a flowchart of a specific implementation of step S12 in the embodiment of the present disclosure. In some embodiments, such as Figure 3 As shown, in the case where every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and every two adjacent even-numbered rows of pixels constitute a designated pixel group, in step S12, pixel data of each designated pixel group of the frame image is subjected to pixel merging binning processing, which may further include:

[0057] Step S31 : for each designated pixel group, the pixel reset voltage and the pixel exposure voltage of each pixel in each row of pixels in the designated pixel group are read out and differentially processed to obtain pixel differential data of each pixel in each row of pixels in the designated pixel group.

[0058] by Figure 2 Taking the designated pixel group composed of the first row of pixels R1 and the third row of pixels R3 as an example, the pixel reset voltage and the pixel exposure voltage of each pixel in the first row of pixels R1 in the designated pixel group are read out and differentially processed to obtain the pixel differential data of each pixel in the first row of pixels R1 in the designated pixel group, and the pixel reset voltage and the pixel exposure voltage of each pixel in the third row of pixels R3 in the designated pixel group are read out and differentially processed to obtain the pixel differential data of each pixel in the third row of pixels R3 in the designated pixel group. The pixel differential data of a pixel is the difference between the pixel reset voltage and the pixel exposure voltage of the pixel.

[0059] Step S32, quantizing the pixel differential data of each row of pixels according to the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels to obtain quantized data corresponding to each row of pixels.

[0060] In some embodiments, through a column-level single-slope ADC, the pixel differential data of each row of pixels in a specified pixel group is quantized according to the difference between the pixel differential data of each row of pixels and a preset ramp reference signal corresponding to each row of pixels, so as to obtain the quantized data corresponding to each row of pixels, thereby realizing analog-to-digital conversion of the pixel differential data of each row of pixels.

[0061] The slope of the preset ramp reference signal corresponding to a row of pixels in each specified pixel group is 1 / k1, and the slope of the preset ramp signal corresponding to another row of pixels is 1 / k2, k1 is the preset weight value corresponding to a row of pixels in each specified pixel group, k2 is the preset weight value corresponding to another row of pixels in each specified pixel group, and 0 <k1<1,k2=1-k1。

[0062] Step S33: performing pixel merging on the quantized data corresponding to two rows of pixels in the specified pixel group.

[0063] Figure 4 is a flowchart of a specific implementation of step S32 in the embodiment of the present disclosure. In some embodiments, such as Figure 4 As shown, in step S32, according to the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels, the pixel differential data of each row of pixels is quantized to obtain the quantized data corresponding to each row of pixels, which may further include:

[0064] Step S41 : Detecting the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels by a comparator of the column-level single-slope ADC.

[0065] Figure 5 Schematic diagram of the structure of a column-level single-slope ADC. Figure 5 As shown, the column-level single-slope ADC (analog-to-digital converter) includes a comparator and a counter, wherein the comparator has two input terminals, wherein the first input terminal is used to input a preset slope reference signal corresponding to each row of pixels, and the second input terminal is used to input pixel differential data of each row of pixels, and the output terminal of the comparator is connected to the counter.

[0066] In step S41, the pixel differential data of each row of pixels in the specified pixel group are sequentially input into the second input end of the comparator, and the corresponding preset ramp reference signal corresponding to each row of pixels in the specified pixel group is sequentially input into the first input end of the comparator. The comparator compares the preset ramp reference signal corresponding to each row of pixels with the corresponding pixel differential data, and outputs a corresponding signal to the counter according to the difference. The output signal of the comparator can represent the difference between the ramp reference signal and the corresponding pixel differential data. For example, when the ramp reference signal is greater than the pixel differential data, the comparator outputs a high-level signal, and when the ramp reference signal is equal to or less than the pixel differential data, the comparator outputs a low-level signal.

[0067] The preset ramp reference signal may be generated by a preset digital-to-analog converter (DAC), and the digital-to-analog converter (DAC) may sequentially output ramp reference signals with different slopes corresponding to different rows of pixels in a specified pixel group.

[0068] Step S42: At the same time, the counter of the column-level single-slope ADC counts according to the difference, so as to quantize the pixel difference data of each row of pixels into the count data of the counter.

[0069] Step S43: When the comparator flips, the counter stops counting to obtain the counting data corresponding to each row of pixels, and the quantized data includes the counting data.

[0070] like Figure 5 As shown, the counter can count according to the output signal of the comparator. For example, when the output signal of the comparator changes from a falling edge (low level) to a rising edge (high level), the counter starts counting. When the output signal of the comparator changes from a rising edge (high level) to a falling edge (low level), the comparator flips and the counter stops counting. The output signal of the counter can be represented as a pulse signal, and the number of pulses can be used to represent the count value, so that the pixel differential data of each row of pixels is quantized into the count data of the counter, and the count data corresponding to each row of pixels is obtained. The quantized data includes the count data.

[0071] In step S33, pixel merging of the quantized data corresponding to two rows of pixels in the designated pixel group may further include: accumulating the count data corresponding to the two rows of pixels in the designated pixel group by a counter.

[0072] After the pixel differential data of two rows of pixels in the specified pixel group are continuously input into the comparator of the column-level single slope ADC, the comparator continuously outputs the corresponding signal, and the counter counts according to the output signal of the comparator until the comparator stops outputting the signal. The final output counting result is the accumulated value of the counting data corresponding to the two rows of pixels in the specified pixel group, and the final output counting result can be stored in the latch. Then, the pixel data of the next specified pixel group is read out, differentially processed and quantized.

[0073] Figure 6 Schematic diagram of the working waveform of a column-level single-slope ADC. Figure 6 As shown, Figure 2 Taking the first to fourth rows of pixels R1-R4 as an example, when the designated pixel group is not divided for pixel merging binning processing, and the slope of the ramp reference signal corresponding to each row of pixels is 1, the counter of the column-level single-slope ADC quantizes the pixel differential data of the first row of pixels R1 to obtain a count value of NUM1, the counter of the column-level single-slope ADC quantizes the pixel differential data of the second row of pixels R2 to obtain a count value of NUM2, the counter of the third row of pixels R3 to obtain a count value of NUM3, and the counter of the fourth row of pixels R4 to obtain a count value of NUM4.

[0074] Figure 7 FIG. 1 is a schematic diagram of the working waveform of another column-level single-slope ADC. Figure 7As shown, when dividing a specified pixel group for pixel merging binning processing, and the slope of the ramp reference signal corresponding to one row of pixels in the specified pixel group is 1 / k1, and the slope of the ramp reference signal corresponding to another row of pixels is 1 / k2, the weight value assigned to the first row of pixels R1 is k1 (0 < k1 < 1), and the weight value assigned to the third row of pixels R3 is k2 = 1 - k1. When quantifying the pixel differential data of the first row of pixels R1 in the specified pixel group through a column-level single-slope ADC, the slope of the ramp reference signal corresponding to the first row of pixels R1 is 1 / k1, and the count value obtained by the counter is k1 * NUM1; when quantifying the pixel differential data of the third row of pixels R3 in the specified pixel group through a column-level single-slope ADC, the slope of the ramp reference signal corresponding to the third row of pixels R3 is 1 / k2, and the counter will add the count value k2 * NUM3 corresponding to the third row of pixels R3 to the count value k1 * NUM1 corresponding to the first row of pixels R1 that was previously held. Finally, the count value corresponding to the first row of pixels R1 and the third row of pixels R3 in the specified pixel group is k1 * NUM1 + k2 * NUM3, thereby achieving the purpose of quantifying and weighted averaging the pixel data of the first row of pixels R1 and the third row of pixels R3 in the specified pixel group, and achieving the pixel merging binning effect.

[0075] Similarly, the weight value assigned to the second row of pixels R2 is k1 (0 < k1 < 1), and the weight value assigned to the fourth row of pixels R4 is k2 = 1 - k1. When quantifying the pixel differential data of the second row of pixels R2 in the specified pixel group through a column-level single-slope ADC, the slope of the ramp reference signal corresponding to the second row of pixels R2 is 1 / k1, and the count value obtained by the counter is k1 * NUM2; when quantifying the pixel differential data of the fourth row of pixels R4 in the specified pixel group through a column-level single-slope ADC, the slope of the ramp reference signal corresponding to the fourth row of pixels R4 is 1 / k2, and the counter will add the count value k2 * NUM4 corresponding to the fourth row of pixels R4 to the count value k1 * NUM2 corresponding to the second row of pixels R2 that was previously held. Finally, the count value corresponding to the second row of pixels R2 and the fourth row of pixels R4 in the specified pixel group is k1 * NUM2 + k2 * NUM4, thereby achieving the purpose of quantifying and weighted averaging the pixel data of the second row of pixels R2 and the fourth row of pixels R4 in the specified pixel group, and achieving the pixel merging binning effect.

[0076] In a column-level single-slope ADC, the slope of the ramp reference signal * counting time = the size of the signal to be quantized. The signal to be quantized is the input pixel differential data, and the counting time is the counting value. Therefore, when the same signal to be quantized is quantized by different ramp reference signals, the counting value is inversely proportional to the slope of the ramp reference signal. In order to facilitate the explanation of the principle of binning the two rows of pixels in the specified pixel group through the column-level single-slope ADC, the count value obtained by quantizing the pixel differential data of each row of pixels through a ramp reference signal with a slope of 1 is used as the reference count value. When different weight values ​​k1 and k2 are assigned to the two rows of pixels in the specified pixel group, and the slopes of the ramp reference signals used to quantize the pixel differential data of the two rows of pixels are configured to be 1 / k1 and 1 / k2 respectively, the count value obtained by quantizing the pixel differential data of any row of pixels in the specified pixel group through the corresponding ramp reference signal is the product of the reference count value obtained by quantizing the pixel differential data of the row of pixels through the ramp reference signal with a slope of 1 and the weight value assigned to the row of pixels. The count values ​​corresponding to the two rows of pixels in the specified pixel group are accumulated, which is equivalent to quantizing and weighted averaging the pixel data of the two rows of pixels in the specified pixel group, thereby achieving the pixel binning effect.

[0077] In this embodiment, based on the method of ramp reference signals with different slopes, it is achieved that when pixels are exposed and read out inside the image sensor, the pixel data of the image pixels are subjected to pixel merging and binning processing before output. On the one hand, the frame rate of the image sensor can be effectively improved while ensuring image clarity, and the processing efficiency of the pixel data can be improved, which is conducive to realizing high-speed camera of the image sensor. On the other hand, the amount of output pixel data can be reduced to save storage resources of digital signals corresponding to the pixel data, reduce the area of ​​memory units used to store digital signals corresponding to the pixel data, and reduce the amount of processing operations of digital signals corresponding to the pixel data, so as to cope with the growing demand for chip miniaturization.

[0078] In some embodiments, during the readout process of pixel data of any frame, before reading out the pixel data of the frame pixels from the first storage capacitor and the second storage capacitor corresponding to each pixel in each row of pixels, the pixel data of each specified pixel group of the frame image is first subjected to pixel merging and binning processing, and then read out. The pixel data of the frame pixels read out is the pixel data after pixel merging and binning processing.

[0079] In this embodiment, the pixels located in each column of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and the pixels located in each column of every two adjacent even-numbered rows of pixels constitute a designated pixel group; or, the pixels located in each row of every two adjacent odd-numbered columns of pixels constitute a designated pixel group, and the pixels located in each row of every two adjacent even-numbered columns of pixels constitute a designated pixel group; or, the pixels in every two adjacent odd-numbered columns of pixels in every two adjacent odd-numbered rows of pixels constitute a designated pixel group, the pixels in every two adjacent even-numbered columns of pixels in every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and the pixels in every two adjacent odd-numbered columns of pixels in every two adjacent even-numbered rows of pixels constitute a designated pixel group, and the pixels in every two adjacent even-numbered columns of pixels in every two adjacent even-numbered rows of pixels constitute a designated pixel group.

[0080] In this embodiment, in step S12, the step of performing pixel merging and binning processing on the pixel data of each specified pixel group of the frame image may further include: binning processing on the pixel reset voltage of each specified pixel group of the frame image; and binning processing on the pixel exposure voltage of each specified pixel group of the frame image.

[0081] Figure 8 FIG. 1 is a schematic diagram of a division method of a specified pixel group in a pixel array of another image sensor, for example, Figure 8As shown, each column of pixels in the first row of pixels R1 and the third row of pixels R3 adjacent to each other along the column direction constitutes a designated pixel group, each column of pixels in the second row of pixels R2 and the fourth row of pixels R4 adjacent to each other along the column direction constitutes a designated pixel group, each column of pixels in the fifth row of pixels R5 and the seventh row of pixels R7 adjacent to each other along the column direction constitutes a designated pixel group, and each column of pixels in the sixth row of pixels R6 and the eighth row of pixels R8 adjacent to each other along the column direction constitutes a designated pixel group. For example, in the first row of pixels R1 and the third row of pixels R3, the corresponding two pixels located in the first column C1 constitute a designated pixel group, the corresponding two pixels located in the second column C2 constitute a designated pixel group, and the corresponding two pixels located in the third column C3 constitute a designated pixel group, and so on, the first row of pixels R1 and the third row of pixels R3 are divided into a plurality of designated pixel groups, and similarly, the second row of pixels R2 and the fourth row of pixels R4 are divided into a plurality of designated pixel groups, the fifth row of pixels R5 and the seventh row of pixels R7 are divided into a plurality of designated pixel groups, and the sixth row of pixels R6 and the eighth row of pixels R8 are divided into a plurality of designated pixel groups. Among them, in each designated pixel group, the colors corresponding to the two pixels are the same. For example, in the designated pixel group consisting of the pixels of the first row R1 and the first column C1 and the pixels of the third row R3 and the first column C1, the pixel colors of the two pixels are both blue B; in the designated pixel group consisting of the pixels of the first row R1 and the second column C2 and the pixels of the third row R3 and the second column C2, the pixel colors of the two pixels are both green G; in the designated pixel group consisting of the pixels of the second row R2 and the first column C1 and the pixels of the fourth row R4 and the first column C1, the pixel colors of the two pixels are both green G; in the designated pixel group consisting of the pixels of the second row R2 and the second column C2 and the pixels of the fourth row R4 and the second column C2, the pixel colors of the two pixels are both red R.

[0082] In the case where the pixels located in each column of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and the pixels located in each column of every two adjacent even-numbered rows of pixels constitute a designated pixel group, in step S12, in the process of reading out the pixel data of any frame, before reading out the pixel data of the frame pixels from the first storage capacitor and the second storage capacitor corresponding to each pixel in each row of pixels, for each designated pixel group of the frame image, the pixel reset voltages of the two pixels constituting the designated pixel group are subjected to a pixel merging binning process, and the pixel exposure voltages of the two pixels constituting the designated pixel group are subjected to a pixel merging binning process, and then the pixel data of the frame pixels are read out, and the pixel data of the frame pixels read out is the pixel data after the pixel merging binning process. For example, Figure 8 As shown, Figure 8The middle arrow indicates a pixel merging binning process. In the process of reading pixel data of any frame, for the first row of pixels R1 and the third row of pixels R3, the pixel reset voltage of the designated pixel group composed of each column of pixels in the first row of pixels R1 and the third row of pixels R3 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group composed of each column of pixels in the first row of pixels R1 and the third row of pixels R3 is subjected to a pixel merging binning process; at the same time, for the second row of pixels R2 and the fourth row of pixels R4, the pixel reset voltage of the designated pixel group composed of each column of pixels in the second row of pixels R2 and the fourth row of pixels R4 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group composed of each column of pixels in the second row of pixels R2 and the fourth row of pixels R4 is subjected to a pixel merging binning process; and so on, pixel merging binning is performed on each designated pixel group at the same time.

[0083] In some embodiments, the pixel data of the pixels in each column of every two adjacent odd-numbered rows of pixels are binned, and the pixel data of the pixels in each column of every two adjacent even-numbered rows of pixels are binned, that is, the pixel data of the designated pixel group composed of pixels in the same column of every other row are averaged to merge the pixel data of the designated pixel group into the pixel data of one pixel for reading. For each designated pixel group, assuming that the pixel reset voltage of one pixel in the designated pixel group is V RST1 , the pixel reset voltage of the other pixel is V RST2 , after the pixel merging binning process, the pixel reset voltage of each pixel in the specified pixel group becomes V RST_bin =(V RST1 +V RST2 ) / 2, similarly, assuming that in the specified pixel group, the pixel exposure voltage of a pixel is V SIG1 , the pixel exposure voltage of the other pixel is V SIG2 , after the pixel merging binning process, the pixel exposure voltage of each pixel in the specified pixel group becomes V SIG_bin =(V SIG1 +V SIG2 ) / 2.

[0084] Fig. 9 Schematic diagram of a method for reading pixel data of a frame of image after pixel merging binning processing. In some embodiments, Fig. 9As shown, after the pixel data of each designated pixel group in any frame image is binned by the above-mentioned alternate row and same column pixel binning method, the pixel data is read out row by row. When reading out, for every two adjacent odd-numbered rows of pixels, only the pixel data of one row of pixels can be read out, and for every two adjacent even-numbered rows of pixels, only the pixel data of one row of pixels can be read out. In this way, the number of rows in the pixel data of the frame image read out will be reduced to half of the original number, while the number of columns remains unchanged. For example, Fig. 9 As shown, after the pixel data of each specified pixel group in any frame image is binned and processed by the above-mentioned alternate row and same column pixel merging method, when the pixel data is read out, the pixel data of the first row of pixels R1, the pixel data of the second row of pixels R2, the pixel data of the fifth row of pixels R5 and the pixel data of the sixth row of pixels R6 are read out pixel by pixel row.

[0085] Fig.10 FIG. 1 is a schematic diagram of a division method of a specified pixel group in a pixel array of another image sensor, for example, Fig.10As shown, each row of pixels in the first column of pixels C1 and the third column of pixels C3 adjacent in the row direction constitutes a designated pixel group, each row of pixels in the second column of pixels C2 and the fourth column of pixels C4 adjacent in the row direction constitutes a designated pixel group, each row of pixels in the fifth column of pixels C5 and the seventh column of pixels C7 adjacent in the row direction constitutes a designated pixel group, and each row of pixels in the sixth column of pixels C6 and the eighth column of pixels C8 adjacent in the row direction constitutes a designated pixel group. For example, in the first column of pixels C1 and the third column of pixels C3, two pixels corresponding to the first row R1 constitute a designated pixel group, two pixels corresponding to the second row R2 constitute a designated pixel group, and two pixels corresponding to the third row R3 constitute a designated pixel group, and so on, the first column of pixels C1 and the third column of pixels C3 are divided into a plurality of designated pixel groups. Similarly, the second column of pixels C2 and the fourth column of pixels C4 can be divided into a plurality of designated pixel groups, the fifth column of pixels C5 and the seventh column of pixels C7 can be divided into a plurality of designated pixel groups, and the sixth column of pixels C6 and the eighth column of pixels C8 can be divided into a plurality of designated pixel groups. Among them, in each designated pixel group, the colors corresponding to the two pixels are the same. For example, in the designated pixel group consisting of the pixels of the first row R1 of the first column C1 and the pixels of the first row R1 of the third column C3, the pixel colors of the two pixels are both blue B; in the designated pixel group consisting of the pixels of the first row R1 of the second column C2 and the pixels of the first row R1 of the fourth column C4, the pixel colors of the two pixels are both green G; in the designated pixel group consisting of the pixels of the second row R2 of the first column C1 and the pixels of the second row R2 of the third column C3, the pixel colors of the two pixels are both green G; in the designated pixel group consisting of the pixels of the second row R2 of the second column C2 and the pixels of the second row R2 of the fourth column C4, the pixel colors of the two pixels are both red R.

[0086] In the case where the pixels in each row of every two adjacent odd-numbered columns of pixels constitute a designated pixel group, and the pixels in each row of every two adjacent even-numbered columns of pixels constitute a designated pixel group, in step S12, in the process of reading out the pixel data of any frame, before reading out the pixel data of the frame pixels from the first storage capacitor and the second storage capacitor corresponding to each pixel in each row of pixels, for each designated pixel group of the frame image, the pixel reset voltages of the two pixels constituting the designated pixel group are subjected to a pixel merging binning process, and the pixel exposure voltages of the two pixels constituting the designated pixel group are subjected to a pixel merging binning process, and then the pixel data of the frame pixels are read out, and the pixel data of the frame pixels read out is the pixel data after the pixel merging binning process. For example, Fig.10 As shown, Fig.10The middle arrow indicates a pixel merging binning process. In the process of reading pixel data of any frame, for the first column of pixels C1 and the third column of pixels C3, the pixel reset voltage of the designated pixel group composed of each row of pixels in the first column of pixels C1 and the third column of pixels C3 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group composed of each row of pixels in the first column of pixels C1 and the third column of pixels C3 is subjected to a pixel merging binning process; at the same time, for the second column of pixels C2 and the fourth column of pixels C4, the pixel reset voltage of the designated pixel group composed of each row of pixels in the second column of pixels C2 and the fourth column of pixels C4 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group composed of each row of pixels in the second column of pixels C2 and the fourth column of pixels C4 is subjected to a pixel merging binning process; and so on, pixel merging binning process is performed on each designated pixel group at the same time.

[0087] In some embodiments, the pixel data of the pixels located in each row of every two adjacent odd-numbered columns of pixels are binned, and the pixel data of the pixels located in each row of every two adjacent even-numbered columns of pixels are binned, that is, the pixel data of the designated pixel group composed of pixels in the same row of every other column are averaged to merge the pixel data of the designated pixel group into the pixel data of one pixel for reading. For each designated pixel group, assuming that the pixel reset voltage of one pixel in the designated pixel group is V RST1 , the pixel reset voltage of the other pixel is V RST2 , after the pixel merging binning process, the pixel reset voltage of each pixel in the specified pixel group becomes V RST_bin =(V RST1 +V RST2 ) / 2, similarly, assuming that in the specified pixel group, the pixel exposure voltage of a pixel is V SIG1 , the pixel exposure voltage of the other pixel is V SIG2 , after the pixel merging binning process, the pixel exposure voltage of each pixel in the specified pixel group becomes V SIG_bin =(V SIG1 +V SIG2 ) / 2.

[0088] Fig.11 FIG. 1 is a schematic diagram of another method for reading pixel data of a frame of image after pixel merging binning processing. In some embodiments, for example, Fig.11As shown, after the pixel data of each designated pixel group in any frame image is binned by the above-mentioned method of binning pixels in the same row of alternate columns, the pixel data is read out row by row. When reading out row by row, for every two adjacent odd-numbered columns of pixels, only the pixel data of one column of pixels can be read out, and for every two adjacent even-numbered columns of pixels, only the pixel data of one column of pixels can be read out. In this way, the number of columns in the pixel data of the frame image read out will be reduced to half of the original number, while the number of rows remains unchanged. For example, Fig.11 As shown, after the pixel data of each specified pixel group in any frame image is binned and processed by the above-mentioned alternate column and same row pixel merging method, when the pixel data is read out row by row, the pixel data of the first column pixel C1, the pixel data of the second column pixel C2, the pixel data of the column row pixel C5 and the pixel data of the sixth column pixel C6 are read out.

[0089] Fig.12 FIG. 1 is a schematic diagram of a division method of a specified pixel group in a pixel array of another image sensor, for example, Fig.12As shown, every two adjacent odd-numbered columns of pixels in the first row of pixels R1 and the third row of pixels R3 adjacent in the column direction constitute a designated pixel group, every two adjacent even-numbered columns of pixels in the first row of pixels R1 and the third row of pixels R3 adjacent in the column direction constitute a designated pixel group, every adjacent odd-numbered columns of pixels in the second row of pixels R2 and the fourth row of pixels R4 adjacent in the column direction constitute a designated pixel group, every adjacent even-numbered columns of pixels in the second row of pixels R2 and the fourth row of pixels R4 adjacent in the column direction constitute a designated pixel group, every adjacent odd-numbered columns of pixels in the fifth row of pixels R5 and the seventh row of pixels R7 adjacent in the column direction constitute a designated pixel group, every adjacent even-numbered columns of pixels in the fifth row of pixels R5 and the seventh row of pixels R7 adjacent in the column direction constitute a designated pixel group, every adjacent odd-numbered columns of pixels in the sixth row of pixels R6 and the eighth row of pixels R8 adjacent in the column direction constitute a designated pixel group, and every adjacent even-numbered columns of pixels in the sixth row of pixels R6 and the eighth row of pixels R8 adjacent in the column direction constitute a designated pixel group. For example, in the first row of pixels R1 and the third row of pixels R3, the corresponding four pixels located in the first column C1 and the third column C3 constitute a designated pixel group, and the corresponding four pixels located in the second column C2 and the fourth column C4 constitute a designated pixel group, and so on, the first row of pixels R1 and the third row of pixels R3 are divided into multiple designated pixel groups. Similarly, the second row of pixels R2 and the fourth row of pixels R4 can be divided into multiple designated pixel groups, the fifth row of pixels R5 and the seventh row of pixels R7 can be divided into multiple designated pixel groups, and the sixth row of pixels R6 and the eighth row of pixels R8 can be divided into multiple designated pixel groups. Among them, the colors corresponding to the four pixels in each designated pixel group are the same. For example, in the designated pixel group composed of the pixels of the first column C1 and the third column C3 in the first row of pixels R1 and the third row of pixels R3, the pixel colors of the four pixels are all blue B; in the designated pixel group composed of the pixels of the second column C2 and the fourth column C4 in the first row of pixels R1 and the third row of pixels R3, the pixel colors of the four pixels are all green G; in the designated pixel group composed of the pixels of the first column C1 and the third column C3 in the second row of pixels R2 and the fourth row of pixels R4, the pixel colors of the four pixels are all green G; in the designated pixel group composed of the pixels of the second column C2 and the fourth column C4 in the second row of pixels R2 and the fourth row of pixels R4, the pixel colors of the four pixels are all red R.

[0090] In the case where pixels of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, pixels of every two adjacent even-numbered columns of pixels constitute a designated pixel group, and pixels of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and pixels of every two adjacent even-numbered columns of pixels constitute a designated pixel group, in step S12, in the process of reading pixel data of any frame, before reading pixel data of the frame pixels from the first storage capacitor and the second storage capacitor corresponding to each pixel in each row of pixels, for each designated pixel group of the frame image, pixel reset voltages of the two pixels constituting the designated pixel group are subjected to pixel merging binning processing, and pixel exposure voltages of the two pixels constituting the designated pixel group are subjected to pixel merging binning processing, and then pixel data of the frame pixels are read out, and the pixel data of the frame pixels read out is pixel data after pixel merging binning processing. For example, Fig.12 As shown, Fig.12 The middle arrow indicates a pixel merging binning process. In the process of reading pixel data of any frame, for the first row of pixels R1 and the third row of pixels R3, the pixel reset voltage of the designated pixel group consisting of every two adjacent odd-numbered columns of pixels in the first row of pixels R1 and the third row of pixels R3 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group consisting of every two adjacent odd-numbered columns of pixels in the first row of pixels R1 and the third row of pixels R3 is subjected to a pixel merging binning process; at the same time, the pixel reset voltage of the designated pixel group consisting of every two adjacent even-numbered columns of pixels in the first row of pixels R1 and the third row of pixels R3 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group consisting of every two adjacent even-numbered columns of pixels in the first row of pixels R1 and the third row of pixels R3 is subjected to a pixel merging binning process. ; At the same time, for the second row of pixels R2 and the fourth row of pixels R4, the pixel reset voltage of the designated pixel group composed of every two adjacent odd-numbered columns of pixels in the second row of pixels R2 and the fourth row of pixels R4 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group composed of every two adjacent odd-numbered columns of pixels in the second row of pixels R2 and the fourth row of pixels R4 is subjected to a pixel merging binning process; at the same time, the pixel reset voltage of the designated pixel group composed of every two adjacent even-numbered columns of pixels in the second row of pixels R2 and the fourth row of pixels R4 is subjected to a pixel merging binning process, and the pixel exposure voltage of the designated pixel group composed of every two adjacent even-numbered columns of pixels in the second row of pixels R2 and the fourth row of pixels R4 is subjected to a pixel merging binning process; and so on, pixel merging binning is performed on each designated pixel group at the same time.

[0091] In some embodiments, the pixel data of the pixels located in each two adjacent odd columns of pixels in each two adjacent odd rows are binned, the pixel data of the pixels located in each two adjacent even columns of pixels in each two adjacent odd rows are binned, the pixel data of the pixels located in each two adjacent odd columns of pixels in each two adjacent even rows are binned, and the pixel data of the pixels located in each two adjacent even columns of pixels in each two adjacent even rows are binned, that is, the pixel data of the designated pixel group composed of pixels of the same color in alternate rows and columns are averaged to merge the pixel data of the designated pixel group into the pixel data of one pixel for reading. For each designated pixel group, assuming that the pixel reset voltages of the four pixels in the designated pixel group are V RST1 、V RST2 、V RST3 、V RST4 , after the pixel merging binning process, the pixel reset voltage of each pixel in the specified pixel group becomes V RST_bin =(V RST1 +V RST2 +V RST3 +V RST4 ) / 4, similarly, assuming that in the specified pixel group, the pixel exposure voltages of the four pixels are V SIG1 、V SIG2 、V SIG3 、V SIG4 , after the pixel merging binning process, the pixel exposure voltage of each pixel in the specified pixel group becomes V SIG_bin =(V SIG1 +V SIG2 +V SIG3 +V SIG4 ) / 4.

[0092] Fig.13 FIG. 1 is a schematic diagram of another method for reading pixel data of a frame of image after pixel merging binning processing. In some embodiments, for example, Fig.13As shown, after the pixel data of each designated pixel group in any frame image is binned by the above-mentioned method of merging pixels of the same color in alternate rows and columns, the pixel data is read out row by row. When reading out, for each two adjacent odd-numbered columns of pixels in each two adjacent odd-numbered rows, only the pixel data of one odd-numbered column pixel in one of the odd-numbered rows can be read out, for each two adjacent even-numbered columns of pixels in each two adjacent odd-numbered rows, only the pixel data of one even-numbered column pixel in one of the odd-numbered rows can be read out, for each two adjacent odd-numbered columns of pixels in each two adjacent even-numbered rows, only the pixel data of one odd-numbered column pixel in one of the even-numbered rows can be read out, and for each two adjacent even-numbered columns of pixels in each two adjacent even-numbered rows, only the pixel data of one even-numbered column pixel in one of the even-numbered rows can be read out. In this way, the number of rows and columns in the pixel data of the frame image read out will be reduced to half of the original number, and the number of pixel data of the frame image read out will be reduced to 1 / 4 of the original number. For example, Fig.13 As shown, after the pixel data of each designated pixel group in any frame image is binned and processed by the above-mentioned alternate row and alternate column same color pixel merging method, when the pixel data is read out, the pixel data of the first column C1 pixel, the second column C2 pixel, the fifth column C5 pixel, and the sixth column C6 pixel in the first row of pixels R1, the pixel data of the first column C1 pixel, the second column C2 pixel, the fifth column C5 pixel, and the sixth column C6 pixel in the second row of pixels R2 are read out, the pixel data of the first column C1 pixel, the second column C2 pixel, the fifth column C5 pixel, and the sixth column C6 pixel in the fifth row of pixels R5, and the pixel data of the first column C1 pixel, the second column C2 pixel, the fifth column C5 pixel, and the sixth column C6 pixel in the sixth row of pixels R6 are read out.

[0093] Fig.14 A schematic diagram of the circuit structure of a pixel in an image sensor provided by an embodiment of the present disclosure.

[0094] In the embodiment of the present disclosure, the image sensor includes a pixel array composed of a plurality of pixels, such as a pixel array composed of N rows and M columns of pixels, where N and M are both positive integers. Fig.14 The circuit structure of the N-1th row of pixels and the Nth row of pixels in a column of pixels is shown as follows: Fig.14 As shown, each pixel in the image sensor includes: a photoelectric conversion unit 101, which is used to generate pixel data during exposure, the pixel data including a pixel reset voltage and a pixel exposure voltage, and output the pixel data to a corresponding signal storage unit 102; the signal storage unit 102, which is used to store the pixel data output by the photoelectric conversion unit 101; an output unit 103, which is used to output the pixel data stored in the signal storage unit 102 to a transmission unit 104; and the transmission unit 104, which is used to transmit the pixel data.

[0095] Furthermore, if Fig.14 As shown, in each pixel, the photoelectric conversion unit 101 includes: a photodiode D0, a transfer transistor T01, a reset transistor T02, a source follower transistor T03 and a bias control transistor T04.

[0096] Among them, one end of the photodiode D0 is grounded GND, and the other end is connected to the second electrode of the transfer transistor T01; the control electrode of the transfer transistor T01 is connected to the transmission control signal TG0, and the first electrode of the transfer transistor T01 is connected to the floating node FD0; the first electrode of the reset transistor T02 is connected to the high level voltage VDD, the second electrode is connected to the floating node FD0, and the control electrode is connected to the reset control signal RST0; the first electrode of the source follower transistor T03 is connected to the high level voltage VDD, the second electrode is connected to the output node P0, and the control electrode is connected to the floating node FD0; the first electrode of the bias control transistor T04 is connected to the output node P0, the second electrode is grounded GND, and the control electrode is connected to the bias control signal BIAS0.

[0097] The signal storage unit 102 includes a reset voltage storage unit and an exposure voltage storage unit. The reset voltage storage unit includes a reset voltage storage control transistor T05 and a first storage capacitor C01 , and the exposure voltage storage unit includes an exposure voltage storage control transistor T08 and a second storage capacitor C02 .

[0098] Among them, the second electrode (output end) of the reset voltage storage control transistor T05 is connected to the output node P0, the first electrode (input end) is connected to the first end of the first storage capacitor C01, and the control electrode is connected to the reset voltage storage control signal GSSR0; the second end of the first storage capacitor C01 is grounded GND.

[0099] The second electrode (output end) of the exposure voltage storage control transistor T08 is connected to the output node P0, the first electrode (input end) is connected to the first end of the second storage capacitor C02, and the control electrode is connected to the exposure voltage storage control signal GSSS0; the second end of the second storage capacitor C02 is grounded GND.

[0100] The transmission unit 104 includes a reset voltage transmission line R and an exposure voltage transmission line S. The reset voltage transmission line R is used to transmit the pixel reset voltage V RST , the exposure voltage transmission line S is used to transmit the pixel exposure voltage V SIG .

[0101] The output unit 103 includes a reset voltage output unit and an exposure voltage output unit. The reset voltage output unit includes a first output control transistor T06 and a second output control transistor T07 . The exposure voltage output unit includes a third output control transistor T09 and a fourth output control transistor T10 .

[0102] The first electrode of the first output control transistor T06 is connected to the high level voltage VDD, the second electrode is connected to the first electrode of the second output control transistor T07, and the control electrode is connected to the first end of the first storage capacitor C01; the second electrode of the second output control transistor T07 is connected to the reset voltage transmission line R, and the control electrode is connected to the row selection readout signal GSEL <n-1>.

[0103] The first electrode of the third output control transistor T09 is connected to the high level voltage VDD, the second electrode is connected to the first electrode of the fourth output control transistor T10, and the control electrode is connected to the first end of the second storage capacitor C02; the second electrode of the fourth output control transistor T10 is connected to the exposure voltage transmission line S, and the control electrode is connected to the row selection readout signal GSEL <n>, N is the number of rows of the pixel array.

[0104] In some embodiments, each column of pixels is correspondingly provided with a column of transmission units 104, that is, each column of pixels is correspondingly provided with a reset voltage transmission line R and an exposure voltage transmission line S, each column of reset voltage transmission line R is connected to each pixel in the column, and each column of exposure voltage transmission line S is connected to each pixel in the column.

[0105] In the embodiment of the present disclosure, the image sensor further includes a signal merging unit, which is used to perform a pixel merging binning process on the pixel data of a specified pixel group.

[0106] In some embodiments, the signal merging unit is used to perform pixel merging binning processing on the pixel data of the designated pixel group after the output unit 103 outputs the pixel data. In this case, every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and every two adjacent even-numbered rows of pixels constitute a designated pixel group. The signal merging unit may include a differential unit and a column-level single-slope ADC, and the column-level single-slope ADC includes a comparator and a counter. Among them, the differential unit is used to perform differential processing on the pixel reset voltage and the pixel exposure voltage of each pixel in each row of pixels in the designated pixel group to obtain the pixel differential data of each pixel in each row of pixels in the designated pixel group; the comparator is used to detect the difference between the pixel differential data of each row of pixels in the designated pixel group and the preset ramp reference signal corresponding to each row of pixels; the counter is used to: count according to the difference detected by the comparator to quantize the pixel differential data of each row of pixels in the designated pixel group into the count data of the counter; when the comparator flips, the counter stops counting to obtain the count data corresponding to each row of pixels; the count data corresponding to two rows of pixels in the designated pixel group are accumulated. A differential unit may be correspondingly arranged for each column of pixels to sample the difference between the pixel reset voltage and the pixel exposure voltage of each row of pixels in the column of pixels, and output the sample to the column-level single-slope ADC.

[0107] For a detailed description of the column-level single-slope ADC, reference may be made to the above description of the column-level single-slope ADC, which will not be repeated here.

[0108] In some embodiments, the signal merging unit is used to perform a pixel merging binning process on the pixel data of a designated pixel group before the output unit 103 outputs the pixel data.

[0109] Fig.15 A schematic diagram of the circuit structure of a pixel in another image sensor provided by an embodiment of the present disclosure.

[0110] In some embodiments, the pixels in each column of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and the pixels in each column of every two adjacent even-numbered rows of pixels constitute a designated pixel group; each designated pixel group is correspondingly configured with a signal merging unit, and the signal merging unit is disposed between two pixels in each designated pixel group. Fig.15 As shown, taking a designated pixel group consisting of pixels in any column of the i-th row and the j-th row as an example, where i, j∈N, and j=i+2, the signal merging unit BIN1 includes a first signal merging transistor Tr1 and a second signal merging transistor Ts1.

[0111] Among them, the input end (first electrode) of the first signal merging transistor Tr1 is connected to the second end of the first storage capacitor C01 of a pixel in the specified pixel group, the output end (second electrode) of the first signal merging transistor Tr1 is connected to the second end of the first storage capacitor C01 of another pixel in the specified pixel group, and the control electrode of the first signal merging transistor Tr1 is connected to the first merging control signal R1; the input end (first electrode) of the second signal merging transistor Ts1 is connected to the second end of the second storage capacitor C02 of a pixel in the specified pixel group, the output end (second electrode) of the second signal merging transistor Ts1 is connected to the second end of the second storage capacitor C02 of another pixel in the specified pixel group, and the control electrode of the second signal merging transistor Ts1 is connected to the second merging control signal S1.

[0112] Fig.16 A schematic diagram of the circuit structure of a pixel in another image sensor provided by an embodiment of the present disclosure.

[0113] In some embodiments, the pixels in each row of every two adjacent odd-numbered columns of pixels constitute a designated pixel group, and the pixels in each row of every two adjacent even-numbered columns of pixels constitute the designated pixel group; each designated pixel group is correspondingly configured with a signal merging unit, and the signal merging unit is disposed between two pixels in each designated pixel group. Fig.16 As shown, taking a designated pixel group consisting of any row of pixels in the Kth column and the Lth column as an example, where K, L∈M, and L=K+2, the signal merging unit BIN2 includes a first signal merging transistor Tr2 and a second signal merging transistor Ts2.

[0114] Among them, the input end (first electrode) of the first signal merging transistor Tr2 is connected to the second end of the first storage capacitor C01 of a pixel in the specified pixel group, the output end (second electrode) of the first signal merging transistor Tr2 is connected to the second end of the first storage capacitor C01 of another pixel in the specified pixel group, and the control electrode of the first signal merging transistor Tr1 is connected to the third merging control signal R2; the input end (first electrode) of the second signal merging transistor Ts2 is connected to the second end of the second storage capacitor C02 of a pixel in the specified pixel group, the output end (second electrode) of the second signal merging transistor Ts2 is connected to the second end of the second storage capacitor C02 of another pixel in the specified pixel group, and the control electrode of the second signal merging transistor Ts2 is connected to the fourth merging control signal S2.

[0115] Fig.17 A schematic diagram of the circuit structure of a pixel in another image sensor provided by an embodiment of the present disclosure.

[0116] In some embodiments, pixels in every two adjacent odd columns of pixels in every two adjacent odd rows of pixels constitute a designated pixel group, and pixels in every two adjacent even columns of pixels in every two adjacent odd rows of pixels constitute a designated pixel group; and pixels in every two adjacent odd columns of pixels in every two adjacent even rows of pixels constitute a designated pixel group, and pixels in every two adjacent even columns of pixels in every two adjacent even rows of pixels constitute a designated pixel group; each designated pixel group is correspondingly configured with a signal merging unit.

[0117] like Fig.17 As shown, taking the designated pixel group consisting of the Kth column pixels and the Lth column pixels in the i-th row pixels and the j-th row pixels as an example, wherein i, j∈N, and j=i+2, K, L∈M, and L=K+2, the signal merging unit includes the signal merging unit including the first signal merging unit and the second signal merging unit, the first signal merging unit including the third signal merging transistor Tr3, the fifth signal merging transistor Tr4 and the seventh signal merging transistor Tr5, the second signal merging unit including the fourth signal merging transistor Ts3, the sixth signal merging transistor Ts4 and the eighth signal merging transistor Ts5;

[0118] Among them, the second end of the first storage capacitor C01 of the K-th column pixel in the specified pixel group is connected through the third signal merging transistor Tr3, and the second end of the first storage capacitor C01 of the L-th column pixel in the specified pixel group is connected through the seventh signal merging transistor Tr5. For example, the first electrode of the third signal merging transistor Tr3 is connected to the second end of the first storage capacitor C01 of a pixel in the K-th column pixel in the specified pixel group, the second electrode is connected to the second end of the first storage capacitor C01 of another pixel in the K-th column pixel in the specified pixel group, and the control electrode is connected to the fifth merging control signal R3; the first electrode of the seventh signal merging transistor Tr5 is connected to the second end of the first storage capacitor C01 of a pixel in the L-th column pixel in the specified pixel group, the second electrode is connected to the second end of the first storage capacitor C01 of another pixel in the L-th column pixel in the specified pixel group, and the control electrode is connected to the fifth merging control signal R3.

[0119] The second end of the second storage capacitor C02 of the Kth column pixel in the designated pixel group is connected through the fourth signal merging transistor Ts3, and the second end of the second storage capacitor C02 of the Lth column pixel in the designated pixel group is connected through the eighth signal merging transistor Ts5. For example, the first electrode of the fourth signal merging transistor Ts3 is connected to the second end of the second storage capacitor C02 of a pixel in the Kth column pixel in the designated pixel group, the second electrode is connected to the second end of the second storage capacitor C02 of another pixel in the Kth column pixel in the designated pixel group, and the control electrode is connected to the sixth merging control signal S3; the first electrode of the eighth signal merging transistor Ts5 is connected to the second end of the second storage capacitor C02 of a pixel in the Lth column pixel in the designated pixel group, the second electrode is connected to the second end of the second storage capacitor C02 of another pixel in the Lth column pixel in the designated pixel group, and the control electrode is connected to the sixth merging control signal S3.

[0120] The second end of the first storage capacitor C01 of the pixel in the i-th row or the j-th row in the designated pixel group is connected through the fifth signal merging transistor Tr4, and the second end of the second storage capacitor C02 of the pixel in the i-th row or the j-th row in the designated pixel group is connected through the sixth signal merging transistor Ts4. For example, the first electrode of the fifth signal merging transistor Tr4 is connected to the second end of the first storage capacitor C01 of a pixel in the i-th row of pixels in the designated pixel group, the second electrode is connected to the second end of the first storage capacitor C01 of another pixel in the i-th row of pixels in the designated pixel group, and the control electrode is connected to the fifth merging control signal R3; the first electrode of the sixth signal merging transistor Ts4 is connected to the second end of the second storage capacitor C02 of a pixel in the i-th row of pixels in the designated pixel group, the second electrode is connected to the second end of the second storage capacitor C02 of another pixel in the i-th row of pixels in the designated pixel group, and the control electrode is connected to the sixth merging control signal S3.

[0121] In the above-mentioned embodiment, based on configuring a signal merging unit in the circuit structure of the pixel, the pixel data of the image pixels are subjected to pixel merging and binning processing before output when the pixel exposure and readout are performed inside the image sensor in a circuit merging manner. On the one hand, the frame rate of the image sensor can be effectively improved while ensuring the image clarity, and the processing efficiency of the pixel data can be improved, which is conducive to realizing high-speed photography of the image sensor. On the other hand, the amount of output pixel data can be reduced to save storage resources of the digital signal corresponding to the pixel data, reduce the area of ​​the memory unit used to store the digital signal corresponding to the pixel data, and reduce the processing operation amount of the digital signal corresponding to the pixel data, so as to cope with the growing demand for chip miniaturization.

[0122] Fig.18 A schematic diagram of a circuit timing sequence of a pixel in an image sensor provided in an embodiment of the present disclosure. In the embodiment of the present disclosure, the working time of a frame can be divided into a reset time period t1 and an exposure time period t2. Figures 14 to 18 , further supplementary explanation is given to the pixel data processing method of the embodiment of the present disclosure.

[0123] It should be noted that the transistors used in the image sensor can be thin film transistors or field effect transistors or other devices with the same or similar characteristics. Since the source and drain of the transistors used are symmetrical, there is no difference between the source and drain. In the embodiments of the present disclosure, in order to distinguish the source and drain of the transistor, one of the electrodes is called the first electrode, the other electrode is called the second electrode, and the gate is called the control electrode. In addition, transistors can be divided into N-type and P-type according to the characteristics of the transistor. The following embodiments are explained with P-type transistors. When a P-type transistor is used, the first electrode is the drain of the P-type transistor, the second electrode is the source of the P-type transistor, and the N-type is the opposite.

[0124] In the following description, each transistor is described as a P-type transistor. In this case, it can be understood that each transistor is turned on when the control pole is in a high level state, and is turned off when the control pole is in a low level state.

[0125] like Fig.14 and Fig.18 As shown, in the n-1th frame, during the reset time period t1 of the pixel of the n-1th frame image, each pixel of the n-1th frame is reset. Specifically, in each pixel, the reset control signal RST0 is enabled, and the reset control signal RST0 is at a high level, so that the reset transistor T02 is turned on under the action of the high level, and at the same time, the transmission control signal TG0 is enabled twice, so that the transmission transistor T01 is turned on under the action of the high level, thereby resetting the photodiode D0 and the floating node FD0 to clear the accumulated charges on the photodiode D0 and the floating node FD0; in addition, the reset voltage storage control transistor T05 and the exposure voltage storage control transistor T08 are both at low levels, and the reset voltage storage control transistor T05 and the exposure voltage storage control transistor T08 are both turned off.

[0126] In the exposure time period t2 of the n-1th frame image pixel, each pixel of the n-1th frame is exposed. Specifically, when n-1=1, in the exposure time period t2 of the n-1th frame image pixel, in each pixel, the reset control signal RST0 is enabled, and the reset control signal RST0 is at a high level, so that the reset transistor T02 is turned on under the action of the high level, and the pixel reset voltage V is generated. RST , pixel reset voltage V RST is written into the floating node FD0; at the same time, the reset voltage storage control signal GSSR0 is enabled, so that the reset voltage storage control transistor T05 is turned on, and the pixel reset voltage V RST The first storage capacitor C01 in the storage unit 102 is written through the turned-on source follower transistor T03 and the turned-on reset voltage storage control transistor T05; then, the transmission control signal TG0 is enabled, so that the transmission transistor T01 is turned on under the action of a high level, and the falling edge of the second pulse to the falling edge of the third pulse in the TG0 signal is the exposure time. The photodiode D0 generates a photoelectric effect during exposure, and the charge accumulated by the photoelectric effect is transferred to the floating node FD0 through the turned-on transmission transistor T01 to generate a pixel exposure voltage V SIG At the same time, the exposure voltage storage control signal GSSS0 is enabled, and the pulse of the GSSS0 signal is generated after the third pulse of the TG0 signal, so that the exposure voltage storage control transistor T08 is turned on, and the pixel exposure voltage V SIG The second storage capacitor C02 in the storage unit 102 is written through the turned-on source follower transistor T03 and the turned-on exposure voltage storage control transistor T08.

[0127] When n-1 is greater than 1, in the case where every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and every two adjacent even-numbered rows of pixels constitute a designated pixel group, in the exposure time period t2 of the n-1th frame image pixel, the row selection readout signal GSEL0 is read row by row. <1> GSEL1 <1> , row selection readout signal GSEL0 <3> GSEL1 <3> , row selection readout signal GSEL0 <2> GSEL1 <2> , row selection readout signal GSEL0 <4> GSEL1 <4> ,.....,GSEL0 <n-2>、GSEL1 <n-2>、GSEL0 <n>、GSEL1 <n>Enable to sequentially turn on the second output control transistor T07 and the fourth output control transistor T10 of each row of pixels in each designated pixel group, so that the pixel reset voltage V of the n-2 frame pixel stored in the first storage capacitor C01 is RST After being amplified by the first output control transistor T06, it is output to the reset voltage transmission line R through the turned-on second output control transistor T07, so that the pixel exposure voltage V of the n-2 frame pixel stored in the second storage capacitor C02 is SIG After being amplified by the third output control transistor T09, it is output to the exposure voltage transmission line S through the turned-on fourth output control transistor T10, and the pixel data of each row of pixels in each designated pixel group in the n-2th frame image are read out in sequence through the reset voltage transmission line R and the exposure voltage transmission line S. The pixel differential data of each pixel in each row of pixels in the designated pixel group is sampled through the differential unit and input into the column-level single-slope ADC for quantization and pixel merging.

[0128] When n-1 is greater than 1, in the case where the pixels located in each column of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and the pixels located in each column of every two adjacent even-numbered rows of pixels constitute a designated pixel group, in the exposure time period t2 of the n-1th frame image pixel, the first merging control signal R1 and the second merging control signal S1 are enabled to turn on the first signal merging transistor Tr1 and the second signal merging transistor Ts1, and pixel merging binning processing is performed on the pixel data of each designated pixel group, that is, the pixel reset voltage and the pixel exposure voltage of each designated pixel group are respectively averaged, and then the first merging control signal R1 and the second merging control signal S1 are low level, so that the first signal merging transistor Tr1 and the second signal merging transistor Ts1 are turned off; then, the second output control transistor T07 and the fourth output control transistor T10 of the pixel row to be read are controlled to be turned on through the row selection readout signal, so that the pixel reset voltage V of the n-2th frame pixel stored in the first storage capacitor C01 is RST After being amplified by the first output control transistor T06, it is output to the reset voltage transmission line R through the turned-on second output control transistor T07, so that the pixel exposure voltage V of the n-2 frame pixel stored in the second storage capacitor C02 is SIG After being amplified by the third output control transistor T09, it is output to the exposure voltage transmission line S through the turned-on fourth output control transistor T10, and the pixel data of the n-2th frame image is read out in sequence through the reset voltage transmission line R and the exposure voltage transmission line S.

[0129] When n-1 is greater than 1, in the case where the pixels located in each row of every two adjacent odd-numbered columns of pixels constitute a designated pixel group, and the pixels located in each row of every two adjacent even-numbered columns of pixels constitute the designated pixel group, in the exposure time period t2 of the n-1th frame image pixel, the third merging control signal R2 and the fourth merging control signal S2 are enabled to turn on the first signal merging transistor Tr2 and the second signal merging transistor Ts2, and pixel merging binning processing is performed on the pixel data of each designated pixel group, that is, the pixel reset voltage and the pixel exposure voltage of each designated pixel group are respectively averaged, and then the third merging control signal R2 and the fourth merging control signal S2 are low level, so that the first signal merging transistor Tr2 and the second signal merging transistor Ts2 are turned off; then, the second output control transistor T07 and the fourth output control transistor T10 of the pixel row to be read are controlled to be turned on through the row selection readout signal, so that the pixel reset voltage V of the n-2th frame pixel stored in the first storage capacitor C01 is RST After being amplified by the first output control transistor T06, it is output to the reset voltage transmission line R through the turned-on second output control transistor T07, so that the pixel exposure voltage V of the n-2 frame pixel stored in the second storage capacitor C02 is SIG After being amplified by the third output control transistor T09, it is output to the exposure voltage transmission line S through the turned-on fourth output control transistor T10, and the pixel data of the n-2th frame image is read out in sequence through the reset voltage transmission line R and the exposure voltage transmission line S.

[0130] When n-1 is greater than 1, in the case where pixels of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, pixels of every two adjacent even-numbered columns of pixels constitute a designated pixel group, and pixels of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and pixels of every two adjacent even-numbered columns of pixels constitute a designated pixel group, and pixels of every two adjacent even-numbered rows of pixels constitute a designated pixel group, in the exposure time period t2 of the pixel of the n-1th frame image, the fifth merging control signal R3 and the sixth merging control signal S3 are enabled to enable the third signal merging transistor Tr3, the fifth signal merging transistor Tr4, the seventh signal merging transistor Tr5, the fourth signal merging transistor Ts3, the sixth signal merging transistor Ts4, and the eighth signal merging transistor Ts5. The binning transistor Ts6 is turned on, and the pixel data of each designated pixel group is binned, that is, the pixel reset voltage and the pixel exposure voltage of each designated pixel group are averaged, and then the fifth binning control signal R3 and the sixth binning control signal S3 are low level, so that the third signal binning transistor Tr3, the fifth signal binning transistor Tr4, the seventh signal binning transistor Tr5, the fourth signal binning transistor Ts3, the sixth signal binning transistor Ts4, and the eighth signal binning transistor Ts6 are turned off; then, the second output control transistor T07 and the fourth output control transistor T10 of the pixel row to be read are turned on through the row selection readout signal, so that the pixel reset voltage V of the n-2 frame pixel stored in the first storage capacitor C01 is RST After being amplified by the first output control transistor T06, it is output to the reset voltage transmission line R through the turned-on second output control transistor T07, so that the pixel exposure voltage V of the n-2 frame pixel stored in the second storage capacitor C02 is SIG After being amplified by the third output control transistor T09, it is output to the exposure voltage transmission line S through the turned-on fourth output control transistor T10, and the pixel data of the n-2th frame image is read out in sequence through the reset voltage transmission line R and the exposure voltage transmission line S.

[0131] Then, in each pixel, the reset control signal RST0 is enabled, and the reset control signal RST0 is at a high level, so that the reset transistor T02 is turned on under the action of the high level, generating a pixel reset voltage V RST , the pixel reset voltage V of the pixel in the n-1th frame RST is written into the floating node FD0; at the same time, the reset voltage storage control signal GSSR0 is enabled, so that the reset voltage storage control transistor T05 is turned on, and the pixel reset voltage V RST The first storage capacitor C01 in the storage unit 102 is written through the turned-on source follower transistor T03 and the turned-on reset voltage storage control transistor T05. Then, the transmission control signal TG0 is enabled, so that the transmission transistor T01 is turned on under the action of a high level. The falling edge of the second pulse to the falling edge of the third pulse in the TG0 signal is the exposure time. The photodiode D0 generates a photoelectric effect during exposure, and the charge accumulated by the photoelectric effect is transferred to the floating node FD0 through the turned-on transmission transistor T01 to generate the pixel exposure voltage V of the pixel of the n-1th frame. SIG Then, the exposure voltage storage control signal GSSS0 is enabled, and the pulse of the GSSS0 signal is generated after the third pulse of the TG0 signal, so that the exposure voltage storage control transistor T08 is turned on, and the pixel exposure voltage V SIG The second storage capacitor C02 in the storage unit 102 is written through the turned-on source follower transistor T03 and the turned-on exposure voltage storage control transistor T08.

[0132] After the exposure of the n-1th frame is completed, the nth frame is entered. In the nth frame, during the reset time period t1 of the image pixel of the nth frame, each pixel of the nth frame is reset. Specifically, in each pixel, the reset control signal RST0 is enabled, and the reset control signal RST0 is at a high level, so that the reset transistor T02 is turned on under the action of the high level, and at the same time, the transmission control signal TG0 is enabled twice, so that the transmission transistor T01 is turned on under the action of the high level, thereby resetting the photodiode D0 and the floating node FD0 to clear the accumulated charges on the photodiode D0 and the floating node FD0; in addition, the reset voltage storage control transistor T05 and the exposure voltage storage control transistor T08 are both at a low level, and the reset voltage storage control transistor T05 and the exposure voltage storage control transistor T08 are both turned off.

[0133] Regarding the specific working principle of the exposure time period t2 of the n-th frame image pixel, please refer to the above description of the specific working process of the exposure time period t2 of the n-1th frame image pixel when n-1 is greater than 1, which will not be repeated here.

[0134] In some embodiments, when pixels of the same column in alternate rows are binned, pixels of the same row in alternate columns are binned, or pixels of the same color in alternate rows and alternate columns are binned, after binning is performed on any frame of pixels and pixel data of any frame of image is read out, the pixel data processing method further includes: obtaining the difference between the pixel reset voltage and the pixel exposure voltage of each pixel read out; and converting the difference into a digital signal and outputting it. Wherein, each column of pixels is correspondingly provided with a column of readout circuits, and the readout circuits include a sampling amplifier circuit and an analog-to-digital conversion ADC circuit. The sampling amplifier circuit outputs a pixel reset voltage V RST and the pixel exposure voltage V output by the exposure voltage transmission line S SIG The difference V RST -V SIG The image is sampled and amplified, quantized and converted into a digital signal by an analog-to-digital conversion (ADC) circuit, and then output to a digital circuit (eg, image processor, ISP) for deep image processing.

[0135] In some embodiments, the end time of the exposure time period of the nth frame image pixel is equal to the end time of reading out the pixel data of the n-1th frame image pixel. That is, the exposure time of the nth frame image pixel and the readout time of the pixel data of the n-1th frame image pixel end synchronously. In this case, after the exposure of the previous frame of two adjacent frames ends, when the next frame is working, the pixel data of the previous frame image can be directly started to be read out without waiting, which is conducive to improving the attenuation phenomenon of the accumulated photons or charges after the pixel exposure. For example, after the exposure of the first frame ends, when entering the second frame work, the pixel data of the first frame image can be directly started to be read out, and the exposure end time of the second frame is the same as the end time of reading out the pixel data of the first frame image.

[0136] In some embodiments, the start time of the exposure time period of the nth frame image pixels is equal to the start time of reading out the pixel data of the n-1th frame image pixels. That is, the exposure time of the nth frame image pixels and the readout time of the pixel data of the n-1th frame image pixels start synchronously. In this case, when the exposure of the current frame ends, but the readout of the pixel data of the previous frame of the current frame has not yet ended, the next frame of the current frame can be processed after the readout of the pixel data of the previous frame ends.

[0137] In some embodiments, the start time of the exposure time period of the n-th frame image pixels is less than the start time of reading out the pixel data of the n-1-th frame image pixels, and the end time of the exposure time period of the n-th frame image pixels is greater than the end time of reading out the pixel data of the n-1-th frame image pixels. That is, the start time of the exposure time period of the n-th frame image pixels is earlier than the start time of reading out the pixel data of the n-1-th frame image pixels, and the end time of the exposure time period of the n-th frame image pixels is later than the end time of reading out the pixel data of the n-1-th frame image pixels.

[0138] In some embodiments, the start time of the exposure time period of the n-th frame image pixel is greater than the start time of reading out the pixel data of the n-1-th frame image pixel, and the end time of the exposure time period of the n-th frame image pixel is less than the end time of reading out the pixel data of the n-1-th frame image pixel. That is, the start time of the exposure time period of the n-th frame image pixel is later than the start time of reading out the pixel data of the n-1-th frame image pixel, and the end time of the exposure time period of the n-th frame image pixel is earlier than the end time of reading out the pixel data of the n-1-th frame image pixel. In this case, after the exposure of the previous frame of two adjacent frames is completed, when the next frame is operated, the pixel data of the previous frame image can be directly started to be read without waiting, which is conducive to improving the attenuation phenomenon of the photons or charges accumulated after the pixel exposure. After the exposure of the next frame is completed, the pixel data of the previous frame is waited to be read before the next frame of the next frame is operated. For example, after the exposure of the first frame is completed, when entering the second frame operation, the pixel data of the first frame image can be directly started to be read. After the exposure of the second frame is completed, the pixel data of the first frame image can be waited to be read before the third frame operation is performed.

[0139] In some embodiments, before any frame of image pixels is exposed, the pixel data processing method further includes: resetting the frame of image pixels during the reset time period of the frame of image pixels of the image sensor. Before each frame of image pixels is exposed, each frame of image pixels is reset to clear the charge generated by the pixel during the last exposure, which is conducive to eliminating the influence of the pixel's last operation history state.

[0140] The present disclosure also provides an image sensor, which includes a pixel array, the pixel array includes a plurality of pixels, the plurality of pixels may include red pixels (R), green pixels (G) and blue pixels (B), each column of pixels in each odd row has the same color, each column of pixels in each even row has the same color, the plurality of pixels are divided into a plurality of designated pixel groups, the designated pixel groups include at least one group of a plurality of pixels corresponding to the same color. Each pixel includes:

[0141] A photoelectric conversion unit, used for outputting pixel data of the pixels of any frame image of the image sensor to a signal storage unit during an exposure phase of the frame image;

[0142] A signal storage unit, used for storing pixel data of the pixels of the frame image;

[0143] An output unit, used for outputting the pixel data of the pixel of the frame image stored in the corresponding signal storage unit during the data reading phase of the frame image;

[0144] The image sensor also includes a signal merging unit, which is used to perform pixel merging binning processing on the pixel data of the designated pixel group during the data reading phase of the frame image.

[0145] In some embodiments, the pixel data includes a pixel reset voltage of the pixel and a pixel exposure voltage generated after the pixel is exposed; the signal storage unit includes: a reset voltage storage control transistor and a first storage capacitor, an exposure voltage storage control transistor and a second storage capacitor, the input ends of the reset voltage storage control transistor and the exposure voltage storage control transistor are both connected to the output end of the photoelectric conversion unit, the output end of the reset voltage storage control transistor is connected to the first end of the first storage capacitor, and the output end of the exposure voltage storage control transistor is connected to the first end of the second storage capacitor; the photoelectric conversion unit is used to output the pixel reset voltage of the pixel of any frame image of the image sensor to the input end of the reset voltage storage control transistor, and output the pixel exposure voltage of the pixel of the frame image to the input end of the exposure voltage storage control transistor; the reset voltage storage control transistor is used to store the pixel reset voltage of the corresponding pixel to the corresponding first storage capacitor; the exposure voltage storage control transistor is used to store the pixel exposure voltage of the corresponding pixel to the corresponding second storage capacitor.

[0146] In some embodiments, pixels in each column of every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and pixels in each column of every two adjacent even-numbered rows of pixels constitute a designated pixel group; or, pixels in each row of every two adjacent odd-numbered columns of pixels constitute a designated pixel group, and pixels in each row of every two adjacent even-numbered columns of pixels constitute a designated pixel group; each designated pixel group is correspondingly configured with a signal merging unit; the signal merging unit includes a first signal merging transistor and a second signal merging transistor, the input end of the first signal merging transistor is connected to the second end of the first storage capacitor of a pixel in the designated pixel group, and the output end of the first signal merging transistor is connected to the output end of the first signal merging transistor. The first signal merging transistor is connected to the second end of the first storage capacitor of a pixel in the specified pixel group; the input end of the second signal merging transistor is connected to the second end of the second storage capacitor of a pixel in the specified pixel group, and the output end of the second signal merging transistor is connected to the second end of the second storage capacitor of another pixel in the specified pixel group; the first signal merging transistor is used to perform pixel merging binning processing on the pixel reset voltages of two pixels in the specified pixel group before the output unit in the specified pixel group outputs the pixel data of the corresponding pixel; the second signal merging transistor is used to perform pixel merging binning processing on the pixel exposure voltages of two pixels in the specified pixel group before the output unit in the specified pixel group outputs the pixel data of the corresponding pixel.

[0147] In some embodiments, pixels in every two adjacent odd columns of pixels in every two adjacent odd rows of pixels constitute a designated pixel group, and pixels in every two adjacent even columns of pixels in every two adjacent odd rows of pixels constitute a designated pixel group; and pixels in every two adjacent odd columns of pixels in every two adjacent even rows of pixels constitute a designated pixel group, and pixels in every two adjacent even columns of pixels in every two adjacent even rows of pixels constitute a designated pixel group; a signal merging unit is configured corresponding to each designated pixel group; the signal merging unit includes a first signal merging unit and a second signal merging unit, the first signal merging unit includes a third signal merging transistor, a fifth signal merging transistor and a seventh signal merging transistor, and the second signal merging unit includes a fourth signal merging transistor, a sixth signal merging transistor and an eighth signal merging transistor.

[0148] Among them, the second end of the first storage capacitor of the first column of pixels in the specified pixel group is connected through the third signal merging transistor, and the second end of the first storage capacitor of the second column of pixels in the specified pixel group is connected through the seventh signal merging transistor; the second end of the second storage capacitor of the first column of pixels in the specified pixel group is connected through the fourth signal merging transistor, and the second end of the second storage capacitor of the second column of pixels in the specified pixel group is connected through the eighth signal merging transistor; the second end of the first storage capacitor of the first row or second row of pixels in the specified pixel group is connected through the fifth signal merging transistor, and the second end of the second storage capacitor of the first row or second row of pixels in the specified pixel group is connected through the sixth signal merging transistor; the first signal merging unit is used to perform pixel merging binning processing on the pixel reset voltages of multiple pixels in the specified pixel group before the output unit in the specified pixel group outputs the pixel data of the corresponding pixel; the second signal merging unit is used to perform pixel merging binning processing on the pixel exposure voltages of multiple pixels in the specified pixel group before the output unit in the specified pixel group outputs the pixel data of the corresponding pixel.

[0149] In some embodiments, every two adjacent odd-numbered rows of pixels constitute a designated pixel group, and every two adjacent even-numbered rows of pixels constitute a designated pixel group; the pixel data includes a pixel reset voltage of the pixel and a pixel exposure voltage generated after the pixel is exposed; the signal merging unit includes a differential unit and a column-level single-slope ADC, and the column-level single-slope ADC includes a comparator and a counter.

[0150] Wherein, the differential unit is used to perform differential processing on the pixel reset voltage and the pixel exposure voltage of each pixel in each row of pixels in the specified pixel group to obtain the pixel differential data of each pixel in each row of pixels in the specified pixel group. The comparator is used to detect the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels. The counter is used to: count according to the difference detected by the comparator to quantize the pixel differential data of each row of pixels in the specified pixel group into the count data of the counter; when the comparator flips, the counter stops counting to obtain the count data corresponding to each row of pixels; the count data corresponding to two rows of pixels in the specified pixel group are accumulated; wherein the slope of the preset ramp reference signal corresponding to one row of pixels in each specified pixel group is 1 / k1, and the slope of the preset ramp signal corresponding to another row of pixels is 1 / k2, k1 is the preset weight value corresponding to one row of pixels in each specified pixel group, k2 is the preset weight value corresponding to another row of pixels in each specified pixel group, 0 <k1<1,k2=1-k1。

[0151] The image sensor provided in the embodiment of the present disclosure is used to implement the above-mentioned pixel data processing method. For specific related descriptions, please refer to the related descriptions of the pixel data processing method in the above-mentioned embodiment, which will not be repeated here.

[0152] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.< / n> < / n> < / n>

Claims

1. A pixel data processing method, characterized in that: Applied to an image sensor, the method comprises: In an exposure phase of any frame image of the image sensor, pixel data of each pixel of the frame image is stored in a corresponding signal storage unit, wherein the pixel data includes a pixel reset voltage of the pixel and a pixel exposure voltage generated after the pixel is exposed; In the data reading stage of the frame image, pixel data of each designated pixel group of the frame image is subjected to a pixel merging binning process, wherein the designated pixel group includes at least one group of multiple pixels corresponding to the same color; The frame image is composed of a plurality of pixels arranged in an array, each column of pixels in each odd-numbered row has the same color, each column of pixels in each even-numbered row has the same color, and the plurality of pixels are divided into a plurality of the designated pixel groups; In the case where every two adjacent odd-numbered rows of pixels constitute one designated pixel group, and every two adjacent even-numbered rows of pixels constitute one designated pixel group, the step of performing pixel merging binning processing on the pixel data of each designated pixel group in the designated pixel groups comprises: The pixel reset voltage and the pixel exposure voltage of each pixel in each row of pixels of the frame image are read out and differentially processed to obtain pixel differential data of each pixel in each row of pixels; According to the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels, the pixel differential data of each row of pixels is quantized to obtain the quantized data corresponding to each row of pixels, wherein the slope of the preset ramp reference signal corresponding to one row of pixels in each specified pixel group is 1 / k1, and the slope of the preset ramp signal corresponding to another row of pixels is 1 / k2, k1 is a preset weight value corresponding to one row of pixels in each specified pixel group, k2 is a preset weight value corresponding to another row of pixels in each specified pixel group, and 0 <k1<1,k2=1-k1; The quantized data corresponding to two rows of pixels in the specified pixel group are pixel-merged.

2. The pixel data processing method according to claim 1, characterized in that: The step of quantizing the pixel differential data of each row of pixels according to the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels to obtain the quantized data corresponding to each row of pixels includes: Detecting the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels by a comparator of a column-level single-slope ADC; At the same time, counting is performed according to the difference by the counter of the column-level single-slope ADC, so as to quantize the pixel differential data of each row of pixels into the count data of the counter; When the comparator flips, the counter stops counting to obtain counting data corresponding to each row of pixels, and the quantized data includes the counting data; The step of performing pixel merging on the quantized data corresponding to two rows of pixels in the specified pixel group includes: The counter accumulates the count data corresponding to two rows of pixels in the designated pixel group.

3. The pixel data processing method according to claim 1, characterized in that: The plurality of pixels are divided into a plurality of the designated pixel groups including: The pixels in each column of every two adjacent odd-numbered rows of pixels constitute one designated pixel group, and the pixels in each column of every two adjacent even-numbered rows of pixels constitute one designated pixel group; or, The pixels in each row of every two adjacent odd-numbered columns of pixels constitute one designated pixel group, and the pixels in each row of every two adjacent even-numbered columns of pixels constitute one designated pixel group; or, Each two adjacent odd-numbered columns of pixels in each two adjacent odd-numbered rows of pixels constitute one of the designated pixel groups, and each two adjacent even-numbered columns of pixels in each two adjacent odd-numbered rows of pixels constitute one of the designated pixel groups; and, Every two adjacent odd-numbered columns of pixels in every two adjacent even-numbered rows of pixels constitute one designated pixel group, and every two adjacent even-numbered columns of pixels in every two adjacent even-numbered rows of pixels constitute one designated pixel group.

4. The pixel data processing method according to claim 3, characterized in that: The step of performing pixel merging binning processing on the pixel data of each designated pixel group of the frame image comprises: Binning the pixel reset voltage of each designated pixel group of the frame image; and The pixel exposure voltages of each designated pixel group of the frame image are binned.

5. The pixel data processing method according to claim 1, characterized in that: The storing pixel data of each pixel of the frame image into the corresponding signal storage unit includes: storing a pixel reset voltage of each pixel of the frame image in a first storage capacitor in a corresponding signal storage unit; and The pixel exposure voltage of each pixel of the frame image is stored in the second storage capacitor in the corresponding signal storage unit.

6. The pixel data processing method according to claim 1, characterized in that: The data reading phase of the frame image is located in the exposure phase of the next frame image after the frame image.

7. An image sensor, characterized in that: A pixel array includes a plurality of pixels, each column of pixels in each odd-numbered row corresponds to the same color, each column of pixels in each even-numbered row corresponds to the same color, the plurality of pixels are divided into a plurality of designated pixel groups, the designated pixel groups include at least one group of a plurality of pixels corresponding to the same color; The pixels include: A photoelectric conversion unit, used for outputting pixel data of the pixels of any frame image of the image sensor to a signal storage unit during an exposure phase of the frame image, wherein the pixel data includes a pixel reset voltage of the pixel and a pixel exposure voltage generated after the pixel is exposed; A signal storage unit, used for storing pixel data of the pixels of the frame image; An output unit, used for outputting the pixel data of the pixel of the frame image stored in the corresponding signal storage unit during the data reading phase of the frame image; The image sensor further comprises a signal merging unit, which is used to perform pixel merging binning processing on the pixel data of the designated pixel group during the data reading phase of the frame image; In the case where every two adjacent odd-numbered rows of pixels constitute one designated pixel group, and every two adjacent even-numbered rows of pixels constitute one designated pixel group, pixel data of the designated pixel group is subjected to a pixel merging binning process, including: The pixel reset voltage and the pixel exposure voltage of each pixel in each row of pixels of the frame image are read out and differentially processed to obtain pixel differential data of each pixel in each row of pixels; According to the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels, the pixel differential data of each row of pixels is quantized to obtain the quantized data corresponding to each row of pixels, wherein the slope of the preset ramp reference signal corresponding to one row of pixels in each specified pixel group is 1 / k1, and the slope of the preset ramp signal corresponding to another row of pixels is 1 / k2, k1 is a preset weight value corresponding to one row of pixels in each specified pixel group, k2 is a preset weight value corresponding to another row of pixels in each specified pixel group, and 0 <k1<1,k2=1-k1; The quantized data corresponding to two rows of pixels in the specified pixel group are pixel-merged.

8. The image sensor according to claim 7, characterized in that: The signal storage unit comprises: a reset voltage storage control transistor and a first storage capacitor, an exposure voltage storage control transistor and a second storage capacitor, the input ends of the reset voltage storage control transistor and the exposure voltage storage control transistor are both connected to the output end of the photoelectric conversion unit, the output end of the reset voltage storage control transistor is connected to the first end of the first storage capacitor, and the output end of the exposure voltage storage control transistor is connected to the first end of the second storage capacitor; The photoelectric conversion unit is used to output the pixel reset voltage of the pixel of any frame image of the image sensor to the input end of the reset voltage storage control transistor, and output the pixel exposure voltage of the pixel of the frame image to the input end of the exposure voltage storage control transistor during the exposure stage of the frame image of the image sensor; The reset voltage storage control transistor is used for storing the pixel reset voltage of the corresponding pixel into the corresponding first storage capacitor; The exposure voltage storage control transistor is used for storing the pixel exposure voltage of the corresponding pixel in the corresponding second storage capacitor.

9. The image sensor according to claim 8, characterized in that: The pixels in each column of every two adjacent odd-numbered rows of pixels constitute one designated pixel group, and the pixels in each column of every two adjacent even-numbered rows of pixels constitute one designated pixel group; or, the pixels in each row of every two adjacent odd-numbered columns of pixels constitute one designated pixel group, and the pixels in each row of every two adjacent even-numbered columns of pixels constitute one designated pixel group; each designated pixel group is correspondingly configured with one signal merging unit; The signal merging unit includes a first signal merging transistor and a second signal merging transistor, wherein the input end of the first signal merging transistor is connected to the second end of the first storage capacitor of a pixel in the specified pixel group, and the output end of the first signal merging transistor is connected to the second end of the first storage capacitor of another pixel in the specified pixel group; the input end of the second signal merging transistor is connected to the second end of the second storage capacitor of a pixel in the specified pixel group, and the output end of the second signal merging transistor is connected to the second end of the second storage capacitor of another pixel in the specified pixel group; The first signal combining transistor is used for performing a pixel combining binning process on the pixel reset voltages of two pixels in the specified pixel group before the output unit in the specified pixel group outputs the pixel data of the corresponding pixel; The second signal combining transistor is used for performing a pixel combining binning process on the pixel exposure voltages of two pixels in the designated pixel group before the output unit in the designated pixel group outputs the pixel data of the corresponding pixel.

10. The image sensor according to claim 8, characterized in that The pixels of every two adjacent odd-numbered columns in every two adjacent odd-numbered rows of pixels constitute one designated pixel group, and the pixels of every two adjacent even-numbered columns in every two adjacent odd-numbered rows of pixels constitute one designated pixel group; and the pixels of every two adjacent odd-numbered columns in every two adjacent even-numbered rows of pixels constitute one designated pixel group, and the pixels of every two adjacent even-numbered columns in every two adjacent even-numbered rows of pixels constitute one designated pixel group; each designated pixel group is correspondingly configured with one signal merging unit; The signal merging unit includes a first signal merging unit and a second signal merging unit, the first signal merging unit includes a third signal merging transistor, a fifth signal merging transistor and a seventh signal merging transistor, and the second signal merging unit includes a fourth signal merging transistor, a sixth signal merging transistor and an eighth signal merging transistor; The second ends of the first storage capacitors of the first column of pixels in the specified pixel group are connected via the third signal merging transistor, and the second ends of the first storage capacitors of the second column of pixels in the specified pixel group are connected via the seventh signal merging transistor; The second ends of the second storage capacitors of the first column of pixels in the specified pixel group are connected via the fourth signal merging transistor, and the second ends of the second storage capacitors of the second column of pixels in the specified pixel group are connected via the eighth signal merging transistor; The second ends of the first storage capacitors of the pixels in the first row or the second row in the specified pixel group are connected via the fifth signal merging transistor, and the second ends of the second storage capacitors of the pixels in the first row or the second row in the specified pixel group are connected via the sixth signal merging transistor; The first signal merging unit is used for performing a pixel merging binning process on the pixel reset voltages of a plurality of pixels in the designated pixel group before the output unit in the designated pixel group outputs the pixel data of the corresponding pixel; The second signal merging unit is used to perform a pixel merging binning process on the pixel exposure voltages of a plurality of pixels in the designated pixel group before the output unit in the designated pixel group outputs the pixel data of the corresponding pixel.

11. The image sensor according to claim 7, characterized in that: The signal merging unit includes a differential unit and a column-level single-slope ADC, and the column-level single-slope ADC includes a comparator and a counter; The differential unit is used to perform differential processing on the pixel reset voltage and the pixel exposure voltage of each pixel in each row of pixels in the specified pixel group to obtain pixel differential data of each pixel in each row of pixels in the specified pixel group; The comparator is used to detect the difference between the pixel differential data of each row of pixels in the specified pixel group and the preset ramp reference signal corresponding to each row of pixels; The counter is used to: count according to the difference detected by the comparator to quantify the pixel differential data of each row of pixels in the specified pixel group into the counting data of the counter; when the comparator flips, the counter stops counting to obtain the counting data corresponding to each row of pixels; and accumulate the counting data corresponding to two rows of pixels in the specified pixel group.

Images