Pixel merging method and related image readout circuit

Abstract

The present invention discloses a pixel merging method for processing pixel data from an image sensor including a pixel array, wherein the pixel merging method comprises executing a first scanning process, wherein the first scanning process is to scan a sensing area of ​​the image sensor to obtain a first amount of pixel data; executing a second scanning process, wherein the second scanning process is to scan the sensing area after the first scanning process is completed to obtain a second amount of pixel data; and performing the pixel merging on the second amount of pixel data according to an offset value, a merging size and an arithmetic value, wherein the offset value and the arithmetic value are determined according to the first amount of pixel data.

Description

Technical Field

[0001] The present invention relates to a pixel merging method and an image reading circuit, and in particular to a pixel merging method and an image reading circuit capable of reducing quantization errors of pixel merging. Background Art

[0002] Existing image processing technology for pixel merging is usually applied to the front-end analog circuit or the back-end digital processing unit of the image sensor, which can combine multiple pixels into a single pixel. One pixel merging method is to accumulate the charges or sensing values ​​of multiple pixels, and then output an accumulated result as the merged pixel data of the merged pixel to improve the signal-to-noise ratio (SNR). However, an upper limit of the pixel data range is easily exceeded, and a maximum pixel data with errors is output. Another pixel merging method is to accumulate the charges or sensing values ​​of multiple pixels, and calculate an average result as the merged pixel data of the merged pixel to maintain the original value of the pixel. However, quantization errors will be generated when the merged pixel is rounded or unconditionally carried / discarded. Therefore, there is a need for improvement in the prior art. Summary of the invention

[0003] Therefore, the main purpose of the present invention is to provide a pixel merging method and an image readout circuit to reduce the quantization error of pixel merging.

[0004] An embodiment of the present invention discloses a pixel merging method for processing pixel data from an image sensor including a pixel array, wherein the pixel merging method includes executing a first scanning process, wherein the first scanning process is to scan a sensing area of ​​the image sensor to obtain a first amount of pixel data; executing a second scanning process, wherein the second scanning process is to scan the sensing area after the first scanning process is completed to obtain a second amount of pixel data; and performing the pixel merging on the second amount of pixel data according to an offset value, a merging size and an arithmetic value, wherein the offset value and the arithmetic value are determined based on the first amount of pixel data.

[0005] Another embodiment of the present invention discloses an image readout circuit for performing pixel merging on pixel data obtained from an image sensor comprising a pixel array, the image readout circuit comprising a scanning circuit for performing a first scanning process, wherein the first scanning process is to scan a sensing area of ​​the image sensor to obtain a first number of pixel data, and to perform a second scanning process, wherein the second scanning process is to scan the sensing area after the first scanning process is completed to obtain a second number of pixel data; and a pixel merging processing circuit for performing the pixel merging on the second number of pixel data according to an offset value, a merging size and an arithmetic value, wherein the offset value and the arithmetic value are determined based on the first number of pixel data. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 FIG. 4 is a schematic diagram of an image readout circuit according to an embodiment of the present invention.

[0007] Figure 2 FIG. 4 is a schematic diagram of raw pixel data obtained through a pre-scanning process according to an embodiment of the present invention.

[0008] Figure 3 FIG. 4 is a schematic diagram of raw pixel data obtained through a normal scanning process according to an embodiment of the present invention.

[0009] The reference numerals are described as follows:

[0010] 10 Image readout circuit

[0011] 102 Scanning Circuit

[0012] 104 pixel merging processing circuit

[0013] 106 Preliminary pixel merging circuit

[0014] 106_2 Accumulator

[0015] 106_4 Divider

[0016] 108 Normal pixel merging circuit

[0017] 108_2 Subtraction circuit

[0018] 108_4 Accumulator

[0019] 108_6 Divider

[0020] 110 Judgment Circuit DETAILED DESCRIPTION

[0021] Please refer to Figure 1 , Figure 11 is a schematic diagram of an image readout circuit 10 according to an embodiment of the present invention. The image readout circuit 10 may be a readout circuit of an image sensor including a pixel array, and is used to scan the pixel array and perform a pixel merging method on pixel data obtained from the image sensor, wherein the pixel data may be raw pixel data of an image. In one embodiment, the image sensor is suitable for sensing a biometric feature image, such as a fingerprint image. The image readout circuit 10 includes a scanning circuit 102 and a pixel merging processing circuit 104. The scanning circuit 102 is used to perform a pre-scanning process, which is to scan a sensing area of ​​the image sensor to obtain a first amount of pixel data, and after the pre-scanning process is completed, a normal scanning process is performed, which is to scan the sensing area (after the pre-scanning process is completed) to obtain a second amount of pixel data. The pre-scanning process is used to obtain information about the possible data range of a current sensed image. That is, the image readout circuit 10 can use the first amount of pixel data obtained from the pre-scanning process to generate the required information, which is then used in a normal pixel merging process.

[0022] The pixel merging processing circuit 104 is used to perform a normal pixel merging process on the second amount of pixel data according to a merging size, an offset value and an arithmetic value, wherein the offset value and the arithmetic value are the necessary information. The offset value is determined according to the first amount of pixel data corresponding to the image. The first amount of pixel data and the second amount of pixel data are pixel data of a fingerprint image obtained by the image sensor at different times.

[0023] In detail, the pixel merging processing circuit 104 of the image readout circuit 10 of the present invention may further include a preliminary pixel merging circuit 106, a normal pixel merging circuit 108, and a judgment circuit 110, wherein the preliminary pixel merging circuit 106 includes an accumulator 106_2 and a divider 106_4, which are used to process a first original pixel data, that is, the first number of pixel data obtained from the pre-scan process. The normal pixel merging circuit 108 includes a subtraction circuit 108_2, an accumulator 108_4, and a divider 108_6, which are used to process a second original pixel data, that is, the second number of pixel data obtained from the normal scanning process. The preliminary pixel merging circuit 106 can selectively perform or not perform pixel merging on the first number of pixel data. The judgment circuit 110 is coupled to the divider 106_4, the subtraction circuit 108_2, and the divider 108_6.

[0024] In one embodiment, when the preliminary pixel merging circuit 106 does not perform pixel merging on the first number of pixel data, the first number of pixel data may bypass the accumulator 106_2 and the divider 106_4, and may not be processed by the accumulator 106_2 and the divider 106_4, but may be processed by the determination circuit 110. In this case, the determination circuit 110 may determine a maximum pixel data and a minimum pixel data and a difference d1 between the maximum pixel data and the minimum pixel data from the first number of pixel data. The minimum pixel data of the first number of pixel data may be used as an offset value X1 applied to the normal pixel merging process. The determination circuit 110 may further determine the difference d1 between the maximum pixel data and the minimum pixel data, and further generate an arithmetic value applied to the normal pixel merging process. The arithmetic value is equal to h / d1, where h is a maximum ideal pixel data according to a data resolution (i.e., the highest grayscale value that the data resolution can provide). The arithmetic value represents an amplification ratio of the highest grayscale value that the data resolution can provide within the sensed data range where the first number of pixel data is located. The arithmetic value is used to reduce the error caused by pixel overflow that occurs in the normal pixel merging process. For example, h can be 255 based on 8-bit pixel data, where 255 represents the highest grayscale value, and d1 can be the difference between the maximum pixel data (e.g., 163) and the minimum pixel data (e.g., 100) from the first number of pixel data. In this example, the arithmetic value h / d1 is equal to the rounded value of 255 / (163-100). The output data of the normal pixel merging circuit 108 is Db_out and the normal pixel merging circuit 108 can calculate the output data in different ways, which can be distinguished by Db_out_1, Db_out_2, etc. In this example, the formula (1) of the output data Db_out_1 can be summarized as follows:

[0025]

[0026] In formula (1), Zi represents each pixel data of the second number of pixel data obtained from performing the normal scanning process, and N is the binning size, which is used to perform the normal pixel binning process on the second number of pixel data. In another embodiment, the offset value can be 0, and the output data Db_out_2 output by the divider 108_6 can be summarized as follows:

[0027]

[0028] In equation (1) or equation (2), N / (h / d1) is a second arithmetic value. In the normal pixel binning of the embodiment of the present invention, the accumulated data is divided by the second arithmetic value smaller than the binning size, thereby reducing the quantization error.

[0029] In another embodiment, when the preliminary pixel combining circuit 106 is used to perform pixel combining on the first number of pixel data, the accumulator 106_2, the divider 106_4 and the determination circuit 110 are all enabled. In this case, the accumulator 106_2 can accumulate every k pixel data of the first number of pixel data corresponding to an m2*n2 array in the pixel array to generate a plurality of accumulated data. k is a combining size used to perform preliminary pixel combining on the first number of pixel data, and k is equal to m2*n2. For example, every four pixel data of the 16 pixel data (i.e., k=2*2=4) are accumulated to generate four accumulated data. Then, each accumulated data is divided by 4 by the divider 106_4 to generate a combined pixel data to generate four combined pixel data (i.e., the third number of combined pixel data). Then, the determination circuit 110 determines the smallest one of the four combined pixel data as an offset value X2, and determines a difference d2 between a minimum combined pixel data and a maximum combined pixel data in the third number of combined pixel data. According to this embodiment, the divider 108_6 of the normal pixel combining circuit 108 can replace X1 and d1 in equation (1) or equation (2) with X2 and d2 to calculate the output data.

[0030] The following are examples of preliminary pixel binning and normal pixel binning. Figure 2 As shown, it is assumed that raw pixel data A0-A15 are obtained according to the pre-scan process, and raw pixel data AA0-AA15 are obtained according to the normal scanning process, wherein the raw pixel data A0-A15 are obtained by the image sensor scanning a sensing area A in the pre-scan process, and the raw pixel data AA0-AA15 are obtained by the image sensor scanning a sensing area AA in the normal scanning process. The sensing area A can be the same as or different from the sensing area AA (different size or different position).

[0031] The preliminary pixel merging circuit 106 is used to perform pixel merging on the original pixel data A0-A15. In detail, the accumulator 106_2 accumulates every four pixel data of a 2*2 array in the pixel array of the image sensor corresponding to the pixel array of the image sensor to generate four accumulated data. Here, the merging size is a 2*2 array. The divider 106_4 divides each accumulated data by 4 (merging size) to generate four merged pixel data B0-B3. For example, B0=(A0+A1+A2+A3) / 4. The judgment circuit 110 determines a minimum value among the four merged pixel data B0-B3 as the offset value X2, and determines the difference d2 between a minimum merged pixel data and a maximum merged pixel data among the four merged pixel data B0-B3.

[0032] about Figure 3, the normal pixel merging circuit 108 is used to perform pixel merging on the original pixel data AA0-AA15. In detail, the subtraction circuit 108_2 subtracts the offset value X2 from each original pixel data AA0-AA15 to generate offset pixel data AA0' to AA15' to prevent overflow of the pixel data range. The accumulator 108_4 accumulates every q offset pixel data to generate an accumulated offset pixel data. In addition, the q offset pixel data correspond to an m1*n1 array, which is a merging size used to perform pixel merging on the second number of pixel data (where q is N in equation (1) and equation (2)). Figure 3 In the example of , the binning size is a 2*2 array, q is equal to 4, and the accumulator 108_4 accumulates every four offset pixel data to generate accumulated offset pixel data. Then, the divider 108_6 divides each accumulated offset pixel data by the second arithmetic value to generate a binning pixel data, such as BB0, BB1, BB2 or BB3, which is the output data Db_out calculated according to equation (1) or equation (2). The aforementioned second arithmetic value is smaller than the binning size, so the quantization error is reduced by performing the division step of the divider 108_6.

[0033] The following is another example. Assuming that the data resolution is 8-bit, the resolution of the image sensor is at least 640*480 pixels, the first number of pixel data obtained by the pre-scan process is 640*480 pixel data, and the binning size used for the preliminary pixel binning process is an 8*8 array. Therefore, the third number of binning pixel data is 80*60 pixel data. The minimum pixel data of the third number of binning pixel data is 100, the maximum pixel data of the third number of binning pixel data is 163, and the highest grayscale value is 255. In other words, the binning pixel data are all within the grayscale value range of 100 to 163, and do not cover the grayscale value range of 0-99 and 164-255. Therefore, the offset value X2 is determined to be 100, and the arithmetic value (h / d2) is 255 / (163-100)=4 (rounded value). Assuming that the second amount of pixel data obtained by the normal scanning process is 640*480 pixel data, and the merging size of the normal pixel merging process is 8*8 array, the normal pixel merging process will generate 80*60 merged pixel data. Here, according to formula (1), the output data Db_out can be Here, Zi represents each pixel data of the second amount of pixel data.

[0034] In another embodiment, the second arithmetic value is equal to the binning size minus a predefined shift value, and the predefined shift value can be the same as or different from the arithmetic value (h / d). In this example, the output data Db_out_3 can be summarized as follows:

[0035]

[0036] In another embodiment, the offset value X may be 0, and the output data Db_out_4 may be summarized in equation (4) as follows:

[0037]

[0038] In equations (3) and (4), the offset value X and the difference value d may be the aforementioned X1 and d1 for not performing preliminary pixel merging, or the offset value X and the difference value d may be the aforementioned X2 and d2 for performing preliminary pixel merging, where N is the merging size. It is worth noting that the merging size used in the pre-scan process may be different from the merging size in the normal scanning process.

[0039] Therefore, the pixel merging method and the image output circuit of the present invention perform a pre-scanning process to obtain information for normal pixel merging, thereby reducing quantization errors. Furthermore, those skilled in the art can appropriately implement the pixel merging method according to different requirements, without being limited to the above embodiments, and all are applicable to the present invention.

[0040] In summary, the present invention provides a pixel merging method and a related image readout circuit to reduce the quantization error of pixel merging and improve the signal-to-noise ratio (SNR) of the image.

[0041] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A pixel merging method for processing pixel data from an image sensor comprising a pixel array, wherein the pixel merging method comprises: Executing a first scanning process, wherein the first scanning process is to scan a sensing area of ​​the image sensor to obtain a first amount of pixel data; Executing a second scanning process, wherein the second scanning process is to scan the sensing area after the first scanning process is completed to obtain a second amount of pixel data, wherein both the first amount of pixel data and the second amount of pixel data are data that have not been subjected to pixel merging calculation; and The pixel merging is performed on the second number of pixel data according to an offset value, a merging size and an arithmetic value, wherein the corresponding number of pixels decreases after the pixel merging is performed on the second number of pixel data, and the offset value and the arithmetic value are determined based on the first number of pixel data.

2. The pixel merging method as described in claim 1 further includes determining the offset value and the arithmetic value according to the first number of pixel data, wherein the offset value is a minimum pixel data of the first number of pixel data, and the arithmetic value is equal to h divided by d1, wherein h is a maximum ideal pixel data based on a data resolution, and d1 is a difference between a maximum pixel data of the first number of pixel data and the minimum pixel data of the first number of pixel data.

3. The pixel merging method according to claim 1, wherein: The step of performing the pixel merging on the second amount of pixel data comprises: subtracting the offset value from each pixel data of the second number of pixel data to generate a corresponding offset pixel data of the second number; summing up every q offset pixel data of the second number of offset pixel data to generate a summed data, wherein each q offset pixel data corresponds to an m1*n1 array in the pixel array, and q is the binning size for performing the pixel binning on the second number of pixel data; as well as The summed data is divided by a second arithmetic value to generate a third amount of binning pixel data, wherein the second arithmetic value is equal to the binning size divided by the arithmetic value.

4. The pixel merging method according to claim 1, wherein: The step of performing the pixel merging on the second amount of pixel data comprises: subtracting the offset value from each of the second number of pixel data to generate a corresponding second number of offset pixel data; summing up every q offset pixel data of the second number of offset pixel data to generate a summed data, wherein each q offset pixel data corresponds to an m1*n1 array in the pixel array, and q is the binning size for performing the pixel binning on the second number of pixel data; as well as The summed data is divided by a second arithmetic value to generate a third amount of binned pixel data, wherein the second arithmetic value is equal to the binning size minus a predefined shift value.

5. The pixel merging method as claimed in claim 1, further comprising: performing the pixel merging on the first amount of pixel data to generate a third amount of merged pixel data; and The offset value and the arithmetic value are determined according to the third number of merged pixel data, wherein the offset value is a minimum merged pixel data of the third number of merged pixel data, and the arithmetic value is equal to h / d2, wherein h is a maximum ideal pixel data based on a data resolution, and d2 is a difference between a maximum merged pixel data of the third number of merged pixel data and the minimum merged pixel data of the third number of merged pixel data.

6. The pixel merging method according to claim 5, characterized in that: The step of performing the pixel merging on the first amount of pixel data comprises: summing up every k pixel data of the first number of pixel data to generate summed data, wherein every k pixel data corresponds to an m2*n2 array in the pixel array, and k is a binning size for performing the pixel binning on the first number of pixel data; as well as The summed data is divided by k to generate a binned pixel data of the third amount of binned pixel data.

7. The pixel merging method according to claim 1, wherein: The first amount of pixel data and the second amount of pixel data are pixel data of a fingerprint image obtained from the image sensor.

8. An image readout circuit for performing pixel merging on pixel data obtained from an image sensor comprising a pixel array, the image readout circuit comprising: a scanning circuit for executing a first scanning process, wherein the first scanning process is to scan a sensing area of ​​the image sensor to obtain a first amount of pixel data, and executing a second scanning process, wherein the second scanning process is to scan the sensing area after the first scanning process is completed to obtain a second amount of pixel data, wherein both the first amount of pixel data and the second amount of pixel data are data that have not yet been subjected to pixel merging calculation; and A pixel merging processing circuit is used to perform the pixel merging on the second number of pixel data according to an offset value, a merging size and an arithmetic value, wherein the corresponding number of pixels becomes smaller after the pixel merging is performed on the second number of pixel data, and the offset value and the arithmetic value are determined based on the first number of pixel data.

9. The image readout circuit according to claim 8, wherein: The pixel merging processing circuit includes a judgment circuit for determining the offset value and the arithmetic value based on the first number of pixel data, wherein the offset value is a minimum pixel data of the first number of pixel data, and the arithmetic value is equal to h divided by d1, wherein h is a maximum ideal pixel data based on a data resolution, and d1 is a difference between a maximum pixel data of the first number of pixel data and the minimum pixel data of the first number of pixel data.

10. The image readout circuit according to claim 8, wherein: The pixel merging processing circuit comprises: a subtraction circuit for subtracting the offset value from each of the pixel data from the second number of pixel data to generate a corresponding number of offset pixel data of the second number; a summer, used for summing up every q offset pixel data of the second number of offset pixel data to generate a summed data, wherein each q offset pixel data corresponds to an m1*n1 array in the pixel array, and q is the binning size for performing the pixel binning on the second number of pixel data; as well as A divider is used for dividing the summed data by a second arithmetic value to generate a third amount of binning pixel data, wherein the second arithmetic value is equal to the binning size divided by the arithmetic value.

11. The image readout circuit according to claim 8, wherein: The pixel merging process comprises: a subtraction circuit for subtracting the offset value from each of the pixel data from the second number of pixel data to generate a corresponding number of offset pixel data of the second number; a summer, used for summing up every q offset pixel data of the second number of offset pixel data to generate a summed data, wherein each q offset pixel data corresponds to an m1*n1 array in the pixel array, and q is the binning size for performing the pixel binning on the second number of pixel data; as well as A divider is used for dividing the summed data by a second arithmetic value to generate a third amount of binning pixel data, wherein the second arithmetic value is equal to the binning size minus a predefined shift value.

12. The image readout circuit according to claim 8, wherein: The pixel merging processing circuit is used to perform the pixel merging on the first number of pixel data to generate a third number of merged pixel data, and to determine the offset value and the arithmetic value based on the third number of merged pixel data, wherein the offset value is a minimum merged pixel data of the third number of merged pixel data, and the arithmetic value is equal to h / d2, wherein h is a maximum ideal pixel data based on a data resolution, and d2 is a difference between a maximum merged pixel data of the third number of merged pixel data and the minimum merged pixel data of the third number of merged pixel data.

13. The image readout circuit according to claim 12, wherein: The pixel merging processing circuit further comprises: a summer, used for summing up every k pixel data of the first number of pixel data to generate summed data, wherein every k pixel data corresponds to an m2*n2 array in the pixel array, and k is a binning size for performing the pixel binning on the first number of pixel data; and A divider is coupled to a judgment circuit and is used for dividing the summed data by k to generate a combined pixel data of the third amount of combined pixel data.

14. The image readout circuit according to claim 8, wherein: The first amount of pixel data and the second amount of pixel data are pixel data of a fingerprint image obtained from the image sensor.

Images