Image sensor and method to correct output of pixels of a pixel array thereof

Abstract

An image sensor and a method for correcting an output of a pixel of a pixel array thereof, the image sensor including: a pixel array including at least one light-shielded area where no light enters and an imaging area where light enters, wherein each pixel includes a photoelectric conversion element; a black level processing unit correcting an output of each pixel in the imaging area; and a memory storing a predetermined black level reference of each pixel in the imaging area. The processing unit calculates a slope determined by an average output value of the pixels in the at least one light-shielded area at the time of imaging acquired during imaging and a reference average output value of the pixels in the at least one light-shielded area under certain conditions acquired before imaging, and corrects the output of each pixel in the imaging area using the predetermined black level reference and the slope.

Description

Technical Field

[0001] This disclosure relates to an image sensor, and more particularly to an image sensor having black level correction and a method for correcting the output of pixels in its pixel array. Background Technology

[0002] The black level of the image sensor is set using the output of the shaded area during or before imaging. Here, the image sensor may contain not only a photodiode array but also signal processing circuitry. Recently, an increasing number of semiconductor chips have multilayer structures, where the signal processing circuitry is often placed on a layer below the image sensor.

[0003] Signal processing circuitry generates heat during its operation, and this heat varies depending on the load on the circuitry. The black level changes with this heat, therefore the black level of the image sensor also varies depending on the load and location of the signal processing circuitry, which is difficult to compensate for and / or correct. Therefore, an image sensor with black level correction that responds to thermal changes is needed. Summary of the Invention

[0004] The following overview is merely representative and not limiting. By using embodiments, the above problems can be overcome, and other advantages can be achieved.

[0005] The image sensor according to this disclosure includes: a pixel array comprising at least one light-shielding region where light does not enter and an imaging region where light enters; a black-level processing unit for correcting the output of each pixel in the imaging region of the pixel array; and a memory for storing a predetermined black-level reference for each pixel in the imaging region of the pixel array, wherein the processing unit is configured to calculate a slope and correct the output of each pixel in the imaging region using the predetermined black-level reference and the slope, the slope being determined by an average output value of pixels in at least one light-shielding region acquired during imaging and a reference average output value of pixels in at least one light-shielding region acquired before imaging under certain conditions.

[0006] In one embodiment, the slope is determined by the average output value of pixels in two shaded regions acquired during imaging and the baseline average output value of pixels in two shaded regions acquired before imaging under certain conditions.

[0007] According to this disclosure, appropriate black level correction can be performed in response to temperature distribution or changes and temperature variations on the image sensor. Attached Figure Description

[0008] Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following accompanying drawings, wherein, unless otherwise specified, the same reference numerals throughout the various views refer to the same parts.

[0009] Figure 1 This is a block diagram of an image sensor.

[0010] Figure 2 This shows the configuration of a single pixel.

[0011] Figure 3 This demonstrates the process of correcting the output.

[0012] Figure 4 The photograph shows an example of temperature distribution or variation on the pixel array of an image sensor.

[0013] Corresponding reference numerals throughout the various views of the accompanying drawings indicate corresponding components. Those skilled in the art will understand that the elements in the figures are shown for simplicity and clarity only and are not necessarily drawn to scale. For example, the dimensions of some elements in the figures may be enlarged relative to other elements to aid in understanding the various embodiments of the invention.

[0014] Explanation of icon numbers

[0015] 10: Pixel array;

[0016] 10a: Imaging area;

[0017] 10b, R1, R2: Shading areas;

[0018] 12: Row selection circuit;

[0019] 14: Parallel ADC;

[0020] 16: Image signal processor;

[0021] 18: Black level processing unit;

[0022] 18a: Memory;

[0023] 20: MIPI;

[0024] 100: Image sensor;

[0025] 114: Photodiode;

[0026] 116: Transfer transistor;

[0027] 118: Floating diffusion;

[0028] 120: Reset transistor;

[0029] 124: Source follower transistor;

[0030] 126: Row selection transistor;

[0031] 128: Bit line;

[0032] RS: Row Selection Line;

[0033] RST: Reset line;

[0034] TX: Transfer control line. Detailed Implementation

[0035] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not required to practice the invention. In other instances, well-known materials or methods have not been described in detail in order to avoid obscuring the invention.

[0036] Throughout this specification, the reference to "an embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the appearance of the phrase "in an embodiment" or "in an embodiment" in different places throughout this specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any suitable combination and / or sub-combination in one or more embodiments.

[0037] Figure 1 This is a block diagram of an image sensor 100 according to an embodiment. In this embodiment, the image sensor 100 is mounted on a single semiconductor chip. The image sensor 100 has a pixel array 10. The pixel array 10 contains pixels arranged in a matrix. Each pixel contains a photodiode (PD) as a photoelectric conversion element. Here, the pixel array 10 has a light-entering imaging region 10a at its center and a light-shielding region 10b where no light enters, these regions being positioned adjacent to the top and bottom edges. The pixel array 10 includes at least one light-shielding region 10b where no light enters and a light-entering imaging region 10a, wherein each pixel of the pixel array 10 contains a photoelectric conversion element. At least one light-shielding region may be comprised of two light-shielding regions located at two positions adjacent to the imaging region. Figure 1 As shown, the shading area at one location and the shading area at another location can be positioned across the imaging area.

[0038] Row selection circuit 12 and column parallel ADC 14 are connected to pixel array 10. Row selection circuit 12 selects a row of pixels in pixel array 10 and outputs the analog signal from each pixel in the row to the corresponding column bit line. Column parallel ADC 14 converts the analog signal supplied from each column bit line into an output digital signal (image signal).

[0039] Therefore, the image signal (output digital signal) from the pixels in the selected row is output in parallel from the column-parallel ADC 14. In the row selection circuit 12, a row is selected sequentially, and when all rows have been selected, the image signal of one frame is output from the column-parallel ADC 14.

[0040] Image signals from the column-parallel ADC 14 are provided to an image signal processor (ISP) 16, where they are processed in various ways. In this embodiment, the ISP 16 has a black-level processing unit 18, in which the image signal is corrected, for example, from IN(x,y) to OUT(x,y), as will be discussed later. A memory 18a is connected to the black-level processing unit 18. In the memory 18a, a predetermined black-level reference for each pixel, determined in advance, is stored in a lookup table, such as LUT(x,y), as will be discussed later. The black-level processing unit 18 is used to correct the output of each pixel in the imaging region of the pixel array. The memory is used to store the predetermined black-level reference for each pixel in the imaging region of the pixel array.

[0041] The image signal (e.g., OUT(x,y)) obtained from the image signal processor 16 is provided to the Mobile Industry Processor Interface (MIPI) 20 and output in a predetermined manner. For example, the output digital signal (e.g., OUT(x,y)) is transmitted to an external device or displayed on a monitor. The terms "image signal," "output digital signal," and the abbreviation "output" are used interchangeably throughout this disclosure.

[0042] Figure 2 The configuration of a single pixel is shown. In this example, the pixel has a photodiode 114. The photodiode 114 is connected to a floating diffuser 118 with a predetermined capacitance via a transfer transistor 116. The floating diffuser 118 stores the charge transferred from the photodiode 114. The floating diffuser 118 is connected to the gate of a source follower transistor 124. The drain of the source follower transistor 124 is connected to a power supply, and its source is connected to the drain of a row select transistor 126.

[0043] The gate of the row selection transistor 126 is connected to the row selection line RS, and the source of the row selection transistor 126 is connected to the bit line 128. When the row selection line RS is high, the row selection transistor 126 is turned on and sets the voltage of the bit line (image signal) according to the voltage of the floating diffusion 118.

[0044] The gate of reset transistor 120 is connected to the reset line RST. The drain of reset transistor 120 is connected to the power supply, and the source is connected to the source of transfer transistor 116 and floating diffuser 118. When the reset line RST is high, the floating diffuser is reset, that is, the voltage of the floating diffuser becomes the voltage of the power supply.

[0045] First, the reset line RST is set high, and reset transistor 120 is turned on, as well as reset floating diffuser 118. This also turns on the output of source follower transistor 124 and reset bit line 128. Next, after RST returns low and after a predetermined exposure time, transfer control line TX is turned on. This turns on transfer transistor 116, and the charge stored in photodiode 114 is read out through floating diffuser 118 to bit line 128 as an analog image signal.

[0046] The digital signal converted from the analog image signal read from the bit line 128 as described above is processed in the image signal processor (ISP) 16. Here, the image signal processor 16 has a black level processing unit 18 in which a black level is set.

[0047] Specifically, in this system, for each pixel in the pixel array 10, a black level reference (a predetermined black level reference, e.g., LUT(x,y)) is stored in memory 18a. For example, when the power is turned on, the image signals (black levels) of all pixels are captured under conditions of no light incidence, and these image signals are stored in memory 18a by mapping the pixel positions.

[0048] Next, for the output from each pixel during imaging, a predetermined black level reference stored in memory 18a can be used to obtain the black level compensated and / or corrected output, such as OUT(x,y).

[0049] Figure 3 The process of correcting the output OUT(x,y) is illustrated. In one embodiment, at least one light-blocking region comprises two light-blocking regions located at two positions immediately adjacent to the imaging region. The output OUT(x,y) of each pixel in the imaging region can be obtained by the following formula:

[0050] OUT(x,y)={IN(x,y)-AVG(R1,R2)}+{LUT(x,y)-AVG(R1ref,R2ref)}*slope+C.

[0051] Here, the meanings of the above terms are as follows.

[0052] IN(x,y) is the output from the pixel at coordinates (x,y) in the imaging region during imaging. It is the value before correction. IN(x,y) is the value corresponding to the incident light level at pixel (x,y) (which is related to the amount of charge generated in the photodiode of the pixel), or the voltage value corresponding to the amount of charge in the floating diffusion. When there is no light incident, the value represents the black level, and when light is incident, the value is the incident light amount plus the black level.

[0053] AVG(R1,R2) represents the average output value of pixels in two shaded regions acquired during imaging. AVG(R1,R2) should be the average black level, which is the output of the pixels in the shaded regions.

[0054] LUT(x,y) is a predetermined black level reference stored in memory 18a for each pixel in the imaging region. It is measured under certain conditions. For example, it can be the value when the image is captured under conditions of maximum heat generation (e.g., at the maximum frame rate). The maximum frame rate is, for example, 24 frames per second (fps), in which case the heat generation of the signal processing circuitry increases.

[0055] AVG(R1ref, R2ref) is a baseline average output value of pixels in two shaded areas acquired before imaging, under certain conditions. Since the value of each pixel is not required, only the average value can be stored in memory 18a.

[0056] In one embodiment, the slope is {AVG(R2)–AVG(R1)} / {AVG(R2ref)–AVG(R1ref)}, where AVG(R1) is the average output value of a pixel in one of the two shaded regions (R1) acquired during imaging, and AVG(R2) is the average output value of a pixel in the other of the two shaded regions (R2) acquired during imaging, AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions (R1) acquired before imaging under certain conditions, and AVG(R2ref) is the baseline average output value of a pixel in the other of the two shaded regions (R2) acquired before imaging under certain conditions.

[0057] C is Figure 3 The constant is not shown in the diagram. Constant C is the adjustment value to be set so that the black level value is appropriate in signal processing.

[0058] The memory 18a can store {LUT(x,y)-AVG(R1ref,R2ref)} for each pixel.

[0059] In this way, in one embodiment, the above equation can be used to obtain the output OUT(x,y) of the black level corrected during imaging. Specifically, this system uses a slope as a variable. The slope is the ratio of the value {AVG(R2)-AVG(R1)} at imaging time to the value {AVG(R2ref)-AVG(R1ref)} in a specific state (reference state), and thus represents the deviation of the value (e.g., black level) of the shaded area at different locations (in this case, the deviation between the top and bottom areas). If the deviation value {AVG(R2)-AVG(R1)} at imaging time is small, the slope value will also be small, and the black level correction amount will also be small.

[0060] The black level processing unit 18 can use a preferred black level reference (e.g., LUT(x,y)) and a slope to correct the output of each pixel in the imaging region. The slope can be calculated by the black level processing unit 18 by the average output value of pixels in at least one shaded region of the pixel array acquired during imaging and the reference average output value of pixels in at least one shaded region of the pixel array acquired before imaging under certain conditions.

[0061] In one embodiment, the slope is determined by the average output value of pixels in two shaded regions acquired during imaging and the baseline average output value of pixels in two shaded regions acquired before imaging under certain conditions.

[0062] When the temperature of the pixel array 10 is uneven due to heat from the signal processing circuitry located below the pixel array 10, the temperature at one location of the pixel array 10 will be higher and the temperature at another location will be lower. Therefore, the extent of the heat effect generated by the signal processing circuitry in the pixel array 10 can be evaluated by the black level of the light-shielding areas (R1, R2) located at two peripheral locations of the pixel array 10.

[0063] As used in this example, it is appropriate to use light-blocking regions located on two opposite sides of the pixel array 10. In the case where four light-blocking regions are located on each side of the pixel array 10, the average of two slopes detected from top to bottom or from left to right of the light-blocking regions can be used, or a larger value can be adopted.

[0064] Figure 4 A photograph is shown illustrating an example of temperature distribution or variation on the pixel array 10 of the image sensor 100. In the figure, bright areas are high-temperature areas. In this example, the temperature is high in the lower right area. In this case, black level correction using the slope described above is effective.

[0065] In the above explanation, in one embodiment, the slope is calculated using the following formula:

[0066] Slope = {AVG(R2) – AVG(R1)} / {AVG(R2ref) – AVG(R1ref)}.

[0067] However, in one embodiment, the following equation can also be used for correction:

[0068] Slope = {AVG(R2) + AVG(R1)} / {AVG(R2ref) + AVG(R1ref)}.

[0069] The slope can be calculated using the two formulas described above, and the user can select one of the formulas. For example, the system can be configured to accept and set a signal from an external source to select one of the two slopes. The system can use both slopes to output an image, and then select one of the slopes via an external command.

[0070] Similarly, in one embodiment described above, the following formula is used.

[0071] {IN(x,y)-AVG(R1,R2)}+{LUT(x,y)-AVG(R1ref,R2ref)}.

[0072] However, in one embodiment, the following formula may be more appropriate based on the value stored in memory 18a.

[0073] {IN(x,y)-AVG(R1,R2)}-{LUT(x,y)-AVG(R1ref,R2ref)}.

[0074] Users can compare the two results from these two formulas and choose one of them.

[0075] Therefore, the output OUT(x,y) of each pixel in the imaging region can be obtained by the following formula:

[0076] OUT(x,y)={IN(x,y)-AVG(R1,R2)}-{LUT(x,y)-AVG(R1ref,R2ref)}*slope+C.

[0077] Similarly, the slope can be calculated using the two formulas mentioned above, and the user can choose one of the formulas.

[0078] Similarly, the slope should generally not be too large. In other words, it can be considered a few percent of "1". Therefore, it is recommended to set upper and lower limits for the slope, and to cancel black level correction or issue a warning if the slope exceeds the limits. For example, the slope could have a lower limit of 5% and an upper limit of 30%.

[0079] Although the invention has been described herein with respect to exemplary embodiments and the best mode for practicing the invention, it will be apparent to those skilled in the art that various embodiments, adaptations, and numerous modifications, improvements, and sub-combinations of the invention may be made without departing from the spirit and scope of the invention.

[0080] The terminology used in the appended claims should not be construed as limiting the invention to the specific embodiments disclosed in the specification and claims. In fact, the scope will be determined entirely by the appended claims, which should be interpreted in accordance with established principles of claim interpretation. This specification and the accompanying drawings should accordingly be considered illustrative rather than limiting.

Claims

1. An image sensor, comprising: A pixel array comprising at least one light-shielding region where light does not enter and an imaging region where light enters, wherein each pixel of the pixel array comprises a photoelectric conversion element; A black level processing unit is used to correct the output of each pixel in the imaging region of the pixel array; as well as A memory for storing a predetermined black level reference for each pixel in the imaging region of the pixel array; The black level processing unit is configured to: The slope is calculated by using the average output value of the pixels in the at least one shaded area of ​​the pixel array acquired during imaging and the baseline average output value of the pixels in the at least one shaded area of ​​the pixel array acquired before imaging under certain conditions. as well as The output of each pixel in the imaging region is corrected using the predetermined black level reference and the slope.

2. The image sensor according to claim 1, wherein the slope value has a lower limit and an upper limit.

3. The image sensor of claim 2, wherein the lower limit of the slope value is 5% and the upper limit of the slope value is 30%.

4. The image sensor of claim 1, wherein the at least one light-shielding area comprises two light-shielding areas at two locations adjacent to the imaging area, and the light-shielding area at one location and the light-shielding area at the other location are positioned across the imaging area.

5. The image sensor of claim 4, wherein the output of each pixel in the imaging region is corrected by the black level processing unit using the following equation: Slope + C; in: OUT(x,y) is the output of the pixel at coordinates (x,y) in the imaging region; IN(x,y) is the output of the pixel at coordinates (x,y) before correction; AVG(R1,R2) is the average output value of the pixels in the two shaded areas acquired during imaging. LUT(x,y) is the predetermined black level reference for the pixel at coordinates (x,y); AVG(R1ref,R2ref) is the baseline average output value of the pixels in the two shaded areas acquired before imaging, under certain conditions; as well as C is a constant.

6. The image sensor according to claim 5, wherein the slope is determined as: {AVG(R2) – AVG(R1)} / {AVG(R2ref) – AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

7. The image sensor according to claim 5, wherein the slope is determined as: {AVG(R2) + AVG(R1)} / {AVG(R2ref) + AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

8. The image sensor of claim 4, wherein the output of each pixel in the imaging region is corrected by the black level processing unit using the following equation: Slope + C; in: OUT(x,y) is the output of the pixel at coordinates (x,y) in the imaging region; IN(x,y) is the output of the pixel at coordinates (x,y) before correction; AVG(R1,R2) is the average output value of the pixels in the two shaded areas acquired during imaging. LUT(x,y) is the predetermined black level reference for the pixel at coordinates (x,y); AVG(R1ref,R2ref) is the baseline average output value of the pixels in the two shaded areas acquired before imaging, under certain conditions; as well as C is a constant.

9. The image sensor according to claim 8, wherein the slope is determined as: {AVG(R2) – AVG(R1)} / {AVG(R2ref) – AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

10. The image sensor of claim 8, wherein the slope is determined as: {AVG(R2) + AVG(R1)} / {AVG(R2ref) + AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

11. A method for correcting the output of pixels in a pixel array of an image sensor, wherein the pixel array includes an imaging region and two light-shielding regions spanning the imaging region, and the pixels are located in the imaging region, the method comprising: The output of the pixel is corrected by the black level processing unit of the image sensor using the following equation: Slope + C; in: OUT(x,y) is the output of the pixel at coordinates (x,y); IN(x,y) is the output of the pixel at coordinates (x,y) before correction; AVG(R1,R2) is the average output value of the pixels in the two shaded areas acquired during imaging. LUT(x,y) is a predetermined black level reference for the pixel at coordinates (x,y); AVG(R1ref,R2ref) is the baseline average output value of the pixels in the two shaded areas obtained before imaging under certain conditions; The slope is determined by the average output value of pixels in the two shaded regions acquired during imaging and the baseline average output value of pixels in the two shaded regions under certain conditions acquired before imaging; and C is a constant.

12. The method for correcting the output of pixels in a pixel array of an image sensor according to claim 11, wherein the slope is determined as: {AVG(R2) – AVG(R1)} / {AVG(R2ref) – AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging. AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

13. The method for correcting the output of pixels in a pixel array of an image sensor according to claim 11, wherein the slope is determined as: {AVG(R2) + AVG(R1)} / {AVG(R2ref) + AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging. AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

14. A method for correcting the output of pixels in a pixel array of an image sensor, wherein the pixel array includes an imaging region and two light-shielding regions spanning the imaging region, and the pixels are located in the imaging region, the method comprising: The output of the pixel is corrected by the black level processing unit of the image sensor using the following equation: Slope + C; in: OUT(x,y) is the output of the pixel at coordinates (x,y); IN(x,y) is the output of the pixel at coordinates (x,y) before correction; AVG(R1,R2) is the average output value of the pixels in the two shaded areas acquired during imaging. LUT(x,y) is a predetermined black level reference for the pixel at coordinates (x,y); AVG(R1ref,R2ref) is the baseline average output value of the pixels in the two shaded areas obtained before imaging under certain conditions; The slope is determined by the average output value of pixels in the two shaded regions acquired during imaging and the baseline average output value of pixels in the two shaded regions under certain conditions acquired before imaging; and C is a constant.

15. The method for correcting the output of pixels in a pixel array of an image sensor according to claim 14, wherein the slope is determined as: {AVG(R2) – AVG(R1)} / {AVG(R2ref) – AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging. AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

16. The method for correcting the output of pixels in a pixel array of an image sensor according to claim 14, wherein the slope is determined as: {AVG(R2) + AVG(R1)} / {AVG(R2ref) + AVG(R1ref)}; in: AVG(R1) is the average output value of a pixel in one of the two shaded regions acquired during imaging. AVG(R2) is the average output value of a pixel in one of the two shaded regions acquired during imaging; AVG(R1ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions; as well as AVG(R2ref) is the baseline average output value of a pixel in one of the two shaded regions acquired before imaging, under certain conditions.

Images