Elimination of blooming artifacts in high dynamic range imaging

Abstract

Image sensors, imaging systems, and methods for eliminating blooming artifacts in high dynamic range imaging. The image sensor includes a pixel array and a controller. The controller is configured to detect when a first pixel signal is at or below a barrier value. The first pixel signal is generated during a first exposure. The controller is also configured to compare neighboring pixel signals with the barrier value. Furthermore, the controller is configured to determine a first image value for the center pixel based on a second pixel signal if at least one of the neighboring pixel signals is above the barrier value. The second pixel signal is generated during a second exposure, which is shorter than the first exposure. The controller is also configured to set the first image value to the first pixel signal if any of the neighboring pixel signals is at or below the barrier value.

Description

BACKGROUND

[0001] Image sensors are used in electronic devices, such as mobile phones, cameras, and computers, to capture images. Specifically, an electronic device is provided with an array of image sensor pixels arranged in a grid pattern. Each image sensor pixel receives incident photons, such as light, and converts them into electrical signals. A column circuit arrangement is coupled to each column to read out sensor signals from each image sensor pixel. SUMMARY

[0002] High dynamic range (HDR) images can be created by combining pixel signals generated by the image sensor pixels during two readouts with different exposure times. In some cases, when the image sensor pixels with the highest sensitivity to the spectrum of incident light become saturated, charge overflow occurs onto neighboring image sensor pixels. This overflow charge (sometimes called blooming or bloom charge) introduces nonlinearity into the signals of the affected pixels and can lead to unwanted image artifacts (sometimes called blooming artifacts) in HDR images.The present disclosure therefore provides image sensors, imaging systems and methods which, among other things, eliminate or mitigate blooming artifacts in HDR imaging by detecting image sensor pixels that are likely to be affected by blooming and replacing their long exposure signals with other signals determined on the basis of their short exposure signals.

[0003] The present disclosure provides an image sensor which, in one implementation, includes a pixel array and a controller. The pixel array includes a plurality of image sensor pixels. The controller is configured to detect whether a first pixel signal is at or below a barrier value. The first pixel signal is generated by a central pixel of the plurality of image sensor pixels during a first exposure time. The controller is also configured to compare a plurality of neighboring pixel signals with the barrier value. The plurality of neighboring pixel signals is generated during the first exposure time. The controller is further configured to determine a first image value for the central pixel based on a second pixel signal if at least one of the plurality of neighboring pixel signals is above the barrier value.The second pixel signal is generated by the center pixel during a second exposure duration that is shorter than the first. The controller is also configured to set the first image value to the first pixel signal if any of the multiple neighboring pixel signals is at or below the barrier value.

[0004] The present disclosure also provides an imaging system which, in one implementation, includes a lens system, an image sensor, and an imaging controller. The image sensor is functionally related to the lens system. The image sensor includes a pixel array containing a plurality of image sensor pixels. The imaging controller is configured to detect whether a first pixel signal is at or below a barrier value. The first pixel signal is generated by a central pixel of the plurality of image sensor pixels during a first exposure time. The imaging controller is also configured to compare a plurality of neighboring pixel signals with the barrier value. The plurality of neighboring pixel signals is generated during the first exposure time.The imaging control is further configured to determine a first image value for the center pixel based on a second pixel signal if at least one of the plurality of neighboring pixel signals is above the barrier value. The second pixel signal is generated by the center pixel during a second exposure duration that is shorter than the first exposure duration. The imaging control is also configured to set the first image value to the first pixel signal if any of the plurality of neighboring pixel signals is at or below the barrier value.

[0005] The present disclosure further provides a method for eliminating blooming artifacts in high dynamic range imaging. The method includes detecting whether a first pixel signal is at or below a barrier value. The first pixel signal is generated by a central pixel of a plurality of image sensor pixels contained in a pixel array during a first exposure time. The method also includes comparing a plurality of neighboring pixel signals with the barrier value. The plurality of neighboring pixel signals is generated during the first exposure time. The method further includes determining a first image value for the central pixel based on a second pixel signal if at least one of the plurality of neighboring pixel signals is above the barrier value.The second pixel signal is generated by the central pixel during a second exposure duration that is shorter than the first. The procedure also includes adjusting the first image value to the first pixel signal if any of the multiple neighboring pixel signals is at or below the barrier value. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] For a detailed description of exemplary implementations, please refer to the attached drawings, where: Fig. 1A is a block diagram of an example of an imaging system according to some implementations; Fig. 1B is a diagram of an example of an imaging system that, according to some implementations, is installed in a vehicle; Fig. 2 is a partial schematic and a partial block diagram of an example of an image sensor according to some implementations; Fig. 3 is a schematic representation of an example of a circuit arrangement in an image sensor pixel according to some implementations; Fig. 4A is a top view of an example of a pixel array with a Bayer color filter array under green light according to some implementations; Fig. 4B is a top view of an example of a pixel array with a Bayer color filter array under red light according to some implementations; Fig. 5A is a diagram of an example of a long exposure image, which, according to some implementations, is generated by a section of a pixel array; Fig. 5B is a diagram of an example of a short exposure image, which, according to some implementations, is generated by a section of a pixel array; Fig. 6 is a flowchart of an example of a procedure for determining a long exposure corrected signal value for an image sensor pixel according to some implementations; Fig. 7A to 7E are diagrams of examples of long exposure images with different combinations of neighboring image sensor pixels, which are to be examined to determine whether a middle pixel is likely to be affected by blooming according to some implementations; Fig. 8 is a flowchart of an example of a procedure for determining a digital lateral overflow value for an image sensor pixel according to some implementations; Fig. 9 is a flowchart of an example of a procedure for determining a long exposure corrected signal value for an image sensor pixel with noise filtering according to some implementations; Fig. 10 is a graphical representation of examples of long-exposure and short-exposure signal values ​​generated by red and green image sensor pixels under green light at different amounts of photon flux according to some implementations; and Fig. Figure 11 shows a flowchart of an example of a procedure for eliminating blooming artifacts in high dynamic range imaging according to some implementations. DEFINITIONS

[0007] Various terms are used to refer to specific system components. Different companies may refer to a component by different names—this document does not intend to differentiate between components that differ in name but not in function. In the following discussion and in the claims, the terms "including" and "comprehensive" are used in an open sense and should therefore be interpreted as meaning "including, but not limited to...". Similarly, the term "couple" or "couples" is intended to mean either an indirect or a direct connection. Therefore, if a first device is coupled to a second device, this connection may be made by a direct connection or by an indirect connection via other devices and connections.

[0008] Terms defining elevation, such as "above," "below," "upper," and "lower," are intended to be spatial terms in relation to the direction of light incident on a pixel array and / or image pixel. Incoming light is to be considered as interacting with or passing through objects and / or structures that are "above" and "upper" before interacting with or passing through objects and / or structures that are "below" or "lower." Therefore, these spatial terms may not have a relationship to the direction of gravity.

[0009] With regard to electrical devices, whether individual or as part of an integrated circuit, the terms "input" and "output" refer to electrical connections to the electrical devices and are not to be understood as verbs requiring an action. For example, a differential amplifier such as an operational amplifier may have a first differential input and a second differential input, and these "inputs" define electrical connections to the operational amplifier and are not to be understood as requiring input signals for the operational amplifier.

[0010] "Activation" means changing the state of a Boolean signal. Boolean signals can be activated high or with a higher voltage, and Boolean signals can be activated low or with a lower voltage, at the discretion of the circuit designer. Similarly, "deactivation" means changing the state of the Boolean signal to a voltage level opposite to the activated state.

[0011] "Light" or "color" refers to visible light in the range between approximately 380 and 700 nanometers (nm). "Light" or "color" also includes invisible light, such as infrared light in the range between approximately 800 nm and 1 millimeter. "Light" or "color" also includes invisible light, such as ultraviolet light in the range of approximately 100 to 400 nm.

[0012] “Control” means, alone or in combination, individual circuit components, an application-specific integrated circuit (ASIC), one or more microcontrollers with control software, a reduced instruction set computer (RISC) with control software, a digital signal processor (DSP), one or more processors with control software, a programmable logic device (PLD), a field-programmable gate array (FPGA) or a programmable system-on-a-chip (PSOC) configured to read inputs and, in response to the inputs, drive outputs. DETAILED DESCRIPTION

[0013] The following explanation is directed towards various implementations of the invention. Although one or more of these implementations may be preferred, the disclosed implementations should not be interpreted or otherwise used in such a way as to limit the scope of protection of the present disclosure, including the claims. Furthermore, the person skilled in the art understands that the following description has broad application and that the discussion of any particular implementation is merely to be understood as an example of that implementation and is not intended to imply that the scope of protection of the present disclosure, including the claims, is limited to that implementation.

[0014] Several examples address systems and methods for eliminating blooming artifacts in high dynamic range (HDR) imaging. Specifically, several examples focus on image sensors and associated imaging systems that detect, based on pixel signals from neighboring image sensor pixels, when image sensor pixels are likely to be affected by blooming. In particular, several examples address image sensors and associated imaging systems that replace the long-exposure signals generated by image sensor pixels affected by blooming with other signals determined based on the short-exposure signals generated by those pixels. An exemplary system is now considered in this patent specification to provide the reader with an overview.

[0015] Fig. Figure 1A shows an example of an imaging system 100. Specifically, the imaging system 100 can be a portable electronic device such as a camera, mobile phone, tablet computer, webcam, video camera, video surveillance system, or video game system with imaging capabilities. In other cases, the imaging system 100 can be an automotive imaging system.

[0016] The in Fig. Figure 1A illustrates an imaging system 100 that includes a camera module 102, which can be used to convert incident light into digital image data. The camera module 102 can include one or more lenses 104 and one or more corresponding image sensors 106. The lenses 104 can be fixed and / or adjustable. During image acquisition, light from a scene can be focused by the lenses 104 onto the image sensor 106. The image sensor 106 can include circuitry for converting analog pixel data into corresponding digital image data to be provided to the imaging controller 108. If desired, the camera module 102 can be equipped with an array of lenses 104 and an array of corresponding image sensors 106.

[0017] The imaging controller 108 can include one or more integrated circuits. The imaging circuits can include image processing circuits, microprocessors, and storage devices such as random-access memory and non-volatile memory. The imaging controller 108 can be implemented using components that are separate from the camera module 102 and / or that form part of the camera module 102, for example, circuits that form part of the image sensor 106. Digital image data acquired by the camera module 102 can be processed and stored using the imaging controller 108. Processed image data can, if desired, be made available to external devices such as a computer, an external display, or other devices using wired and / or wireless communication paths connected to the imaging controller 108.

[0018] Fig. Figure 1B shows another example of the Imaging System 100. This is shown in Fig. 1B Illustrated imaging system 100 includes an automobile or vehicle 110. The vehicle 110 is shown for illustrative purposes as a passenger car, but the imaging system 100 can be other types of vehicles, including commercial vehicles, on-road vehicles, and off-road vehicles. Commercial vehicles can include buses and semi-trucks. Off-road vehicles can include tractors and harvesters. In the example of Fig. 1B The vehicle 110 includes a forward-looking camera module 102, arranged to capture images of scenes in front of the vehicle 110. Such a forward-looking camera module 102 can be used for any suitable purpose, such as lane keeping assist, collision warning systems, adaptive cruise control systems, autonomous driving systems, and proximity detection. The vehicle 110 further includes a rear-facing camera module 102, arranged to capture images of scenes behind the vehicle 110. Such a rear-facing camera module 102 can be used for any suitable purpose, such as collision warning systems,

[0019] Reversing video, autonomous driving systems, proximity detection, monitoring the position of overtaking vehicles, and reversing. The vehicle 110 further includes a side-view camera module 102, arranged to capture images of scenes beside the vehicle 110. Such a side-view camera module 102 can be used for any suitable purpose, such as blind spot monitoring, collision warning systems, autonomous driving systems, monitoring the position of overtaking vehicles, lane change detection, and proximity detection. In situations where the imaging system 100 is a vehicle, the imaging controller 108 can be a controller of the vehicle 110. The discussion now turns in more detail to the image sensor 106 of the camera module 102.

[0020] Fig. Figure 2 shows an example of the image sensor 106. In particular, it shows Fig. 2, that the image sensor 106 can comprise a substrate 200 made of semiconductor material (for example, silicon) encapsulated within a package to form an enclosed semiconductor device or product. Bond pads or other connection points of the substrate 200 couple to terminals of the image sensor 106, such as a serial communication channel 202 coupled to a first terminal 204, and capture input 206 coupled to a second terminal 208. Additional terminals are present, such as ground, common, or power terminals, but these are omitted to avoid unnecessary complexity. While a single substrate 200 is shown, in other implementations multiple substrates can be combined to form the image sensor 106, creating a multi-chip module.

[0021] The in Fig. The illustrated image sensor 106 includes a pixel array 210 with a multitude of image sensor pixels 212 arranged in rows and columns. The pixel array 210, which is an example of an “array of pixels”, can, for example, include hundreds or thousands of rows and columns of image sensor pixels 212.

[0022] Control and readout of the pixel array 210 can be implemented by an image sensor controller 214, which is coupled to a row controller 216 and a column controller 218. The row controller 216 can receive row addresses from the image sensor controller 214 and supply corresponding row control signals to image sensor pixels 212, such as reset, row selection, charge transfer, double conversion gain, and readout control signals. The row control signals can be communicated via one or more conductors, such as row control paths 220.

[0023] The column controller 218 can be coupled to the pixel array 210 by means of one or more conductors, such as column lines 222. Column controllers can sometimes be referred to as column control circuits, readout circuits, and / or column decoders.

[0024] The column lines 222 can be used to read pixel signals from the image sensor pixels 212 and to supply the image sensor pixels 212 with bias currents and / or bias voltages. If desired, during pixel readout operations, a pixel row in the pixel array 210 can be selected using the row control 216, and pixel signals generated by image sensor pixels 212 in this pixel row can be read out along the column lines 222.The column controller 218 can include a sample-and-hold circuit arrangement for sampling and temporarily storing pixel signals read from the pixel array 210, an amplifier circuit arrangement, an analog-to-digital converter (ADC) circuit arrangement, a bias circuit arrangement, a column memory, a flip-flop circuit arrangement for selectively enabling or disabling the column circuit arrangement, or another circuit arrangement coupled to one or more columns of image sensor pixels 212 in the pixel array 210 for operating the image sensor pixels 212 and for reading pixel signals from the image sensor pixels 212. The ADC circuit arrangement in the column controller 218 can convert analog pixel values ​​received from the pixel array 210 into corresponding digital image data.The column control 218 can, for example, transmit digital image data to the image sensor control 214 and / or the imaging control 108 via the serial communication channel 202. Fig. 1A) deliver.

[0025] Fig. Figure 3 shows an example of a circuit arrangement in one of the image sensor pixels 212. The image sensor pixels 212 may have fewer, additional, or different components in different configurations than those shown in Figure 3. Fig. 3 illustrated. In particular, it shows Fig. 3, that each of the image sensor pixels 212 can include a photodetector 302 (for example, a photodiode). A positive pixel supply voltage, such as supply voltage VAAPIX, can be provided at a positive power supply terminal 304. A ground supply voltage, such as a reference voltage Vss, can be provided at a ground terminal 306. Incoming light is collected by the photodetector 302, in some cases after the light passes through a color filter structure (not shown). The photodetector 302 converts the light into an electrical charge.

[0026] Before an image is acquired, the reset control signal RST can be acknowledged. The reset control signal RST turns on a reset transistor 308 and resets a charge storage node (CS node) 310 to a voltage equal to or near the supply voltage VAAPIX. The reset control signal RST can then be nulled to turn off the reset transistor 308. After the reset is complete, the transfer gate control signal TX can be acknowledged to turn on the transfer transistor 312. When the transfer transistor 312 is turned on, the charge generated by the photodetector 302 in response to the incident light is transferred to the charge storage node 310. The charge storage node 310 has a capacitance that can be used to store the charge transferred by the photodetector 302.The signal associated with the charge stored in the charge storage node 310 is buffered by a source-follower transistor 314. A row-selector transistor 316 connects the source-follower transistor 314 to one of the column lines 222.

[0027] When it is desired to read the value of the charge stored in charge storage node 310, a control signal RS is acknowledged. The readout value can be, for example, the value of charge storage node 310 represented by the signal at the source terminal S of source follower transistor 314. When the control signal RS is acknowledged, the row selector transistor 316 is turned on, and an output signal Vout, representing the magnitude of the charge stored in charge storage node 310, is generated on an output path 318. The output signal Vout is an example of a "pixel signal." When the control signal RS is acknowledged, one of the column lines 222 can be used to route the output signal Vout from the image sensor pixel 212 to the readout circuitry, such as the column control 218 in [reference missing]. Fig. 2, to lead.

[0028] When the photodetector 302 of one of the image sensor pixels 212 is exposed to incident light, charge begins to accumulate in a photodiode well of the photodetector 302. Under certain circumstances, more charge can be generated than the amount that the photodetector 302 can hold, for example, if one of the image sensor pixels 212 is exposed to extremely bright light. In other words, one of the image sensor pixels 212 becomes saturated when the charge exceeds the full-well capacity of its photodetector 302. When one of the image sensor pixels 212 is saturated, charge can overflow to neighboring pixels. Fig. Figure 4A, for example, shows a top view of a first array 402 of pixels with a Bayer color filter array (red-green-green-blue) under green light. When the green pixels of the first array 402 are saturated, the charge spills over to the neighboring red and blue pixels of the first array 402, as indicated by the arrows in Figure 4A. Fig. 4A illustrates this. As a further example, it shows Fig. Figure 4B shows a top view of a second array 404 of pixels with a Bayer color filter array under red light. When the red pixels of the second array 404 are saturated, charge flows to the neighboring green and blue pixels of the second array 404, as indicated by the arrows in Figure 4B. Fig. Figure 4B illustrates this. The spilled charge (sometimes called blooming or bloom charge) adds to the photo signal, and as a result, the pixel signal is no longer proportional to the illumination level. As described in more detail below, the non-linearity caused by the bloom charge can lead to unwanted image artifacts (sometimes called bloom artifacts) in HDR images.

[0029] HDR images can be created by combining pixel signals from two readouts of the pixel array 210 at different exposure times. In some implementations, the image sensor 106, for example, can include a digital lateral overflow (DLO) pipeline that determines a combined linearized HDR response based on pixel signals from a long-duration and a short-duration readout. Pixel signals generated by image sensor pixels 212 that have become saturated are not proportional to the illuminance. Thus, the DLO pipeline truncates the pixel signals of the long and short readouts that are above a predetermined barrier and assumes that the pixel signals below the predetermined barrier are proportional to the illuminance.When image sensor pixels 212 are affected by blooming, they can generate pixel signals that are lower than the predetermined barrier but higher than their photosignal, for example, due to charge spillover from one or more neighboring pixels that have become saturated. Although they are not proportional to the illuminance, these pixel signals are not clipped because their values ​​are below the predetermined barrier. Thus, a combined HDR image created using these non-linear pixel signals can exhibit color artifacts, especially after color processing. In other words, blooming can cause color artifacts in combined HDR images. Blooming can also cause noise with fixed patterns (sometimes referred to as "structural noise").

[0030] (referred to as "noise"), occurs when pixels are saturated at different light levels, such as when there are fluctuations in full-well capacity.

[0031] As described above, the DLO pipeline determines a combined linearized HDR response based on pixel signals for long and short readouts. Fig. 5A is a diagram of a first time frame 502 (sometimes referred to as the "T1 frame") generated by a section of the pixel array 210. Fig. 5B is a diagram of a second time frame 504 (sometimes referred to as a "T2 frame") generated by a section of the pixel array 210. The exposure of the second time frame 504 (an example of a "second exposure") is lower than the exposure of the first time frame 502 (an example of a "first exposure"). Exposure is the product of integration time and light intensity. Thus, in some implementations, the exposure time of the second time frame 504 (an example of a "second exposure duration") is shorter than the exposure time of the first time frame 502 (an example of a "first exposure duration"). Alternatively or additionally, the image sensor 106 may include one or more attenuation filters to reduce the intensity of the incident light for the second time frame 504.Alternatively or additionally, the image sensor 106 can use a detector with a smaller area to reduce the intensity of the incident light for the second time frame 504. As described in more detail below, the image sensor 106 is configured to determine when a first pixel signal T1(c) generated by a middle pixel in the first time frame 502 is likely to be affected by blooming, based on two or more of a plurality of neighboring pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8) generated by a plurality of neighboring pixels in the first time frame 502. The middle pixel can be any pixel in the pixel array.

[0032] Fig. Figure 6 is a flowchart of an example of a procedure 600 for determining a first image value T1*(c) for the middle pixel in the first time frame 502 according to some implementations. For a given pixel array, the procedure 600 can be performed for one or more pixels in the given pixel array. For the sake of simplicity, the procedure 600 is shown in Fig. Figure 6 is shown and described as a series of processes. However, these processes can occur in different sequences and / or simultaneously and / or in conjunction with other processes not shown or described herein.

[0033] At block 602, the first pixel signal T1(c) is compared to a barrier value B1. The first pixel signal T1(c) is generated by the center pixel in the first frame 502. The barrier value B1 can include a predetermined threshold above which the center pixel is considered saturated or nearly saturated. For example, in an image with a 12-bit data format and a maximum signal level of 4,095 digital numbers (DN), the center pixel may be saturated or near saturated if the first pixel signal T1(c) is above 3,600 DN; therefore, B1 can be set to 3,600 DN. If the first pixel signal T1(c) is above the barrier value B1, the first frame value T1*(c) at block 604 is set to the barrier value B1.Alternatively, if the first pixel signal T1(c) is at or below the barrier value B1, two or more of the neighboring pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8) are compared with the barrier value B1. The neighboring pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8) are generated by the neighboring pixels in the first time frame 502.

[0034] In some implementations, the neighbor pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8), generated by all eight neighbor pixels, are compared to the barrier value B1. This can be done, for example, through shading in Fig. As shown in Figure 7A, the neighbor pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8), generated by all eight of the neighbor pixels, can each be compared to the barrier value B1. However, in some situations, it is not necessary to check the neighbor pixel signals generated by some of the neighbor pixels. For example, a pixel signal generated by a green pixel may not be affected by blooming if one or more of its neighboring green pixels are saturated. Thus, in some implementations, fewer than all eight of the neighbor pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8) are checked. As indicated by the shading in Figure 7A, this is not necessary. Fig. As illustrated in Figure 7B, for example, the neighbor pixel signals T1(4) and T1(5), generated by the two neighboring pixels located next to the middle pixel in the same row as the middle pixel, can each be compared to the barrier value B1. As a further example, as illustrated by the shading in Fig. Figure 7C illustrates how the neighboring pixel signals T1(2) and T1(7), generated by the two neighboring pixels positioned next to the middle pixel on the rows above and below it, are compared to the barrier value B1. As a further example, how shading in Fig. Figure 7D illustrates how the neighbor pixel signals T1(2), T1(4), T1(5), and T1(7), generated by the four neighboring pixels not diagonally adjacent to the central pixel, are each compared to the barrier value B1. As a further example, how shading in Fig. Figure 7E illustrates comparing the neighbor pixel signals T1(1), T1(3), T1(6) and T1(8), which are generated by the four neighbor pixels positioned diagonally adjacent to the middle pixel, with the barrier value B1.

[0035] Returning to Block 606 in Fig. 6. The neighboring pixel signals T1(2), T1(4), T1(5), and T1(7) of the four neighboring pixels that are not diagonally adjacent to the middle pixel are each compared to the barrier value B1. If all of the neighboring pixel signals T1(2), T1(4), T1(5), and T1(7) are at or below the barrier value B1, the first image value T1*(c) at block 608 is set to the value of the first pixel signal T1(c).

[0036] Alternatively, if one of the neighboring pixel signals T1(2), T1(4), T1(5), and T1(7) is above the barrier value B1, the first pixel signal T1(c) is likely affected by blooming. As described above, pixels affected by blooming can produce pixel signals that are not proportional to the illuminance. However, since the exposure time of the second frame 504 is shorter than the exposure time of the first frame 502, the second pixel signal T2(c), generated by the center pixel, cannot be affected by blooming. Therefore, if one of the neighboring pixel signals T1(2), T1(4), T1(5), and T1(7) is above the barrier value B1, the first frame value T1*(1) at block 610 is set to the product of the second pixel signal T2(c) and an exposure ratio G1. The exposure ratio G1 can be determined based on the exposure durations and / or gain values ​​of the first time frame 502 and the second time frame 504.If the gain values ​​for the first frame (502) and the second frame (504) are the same, the exposure ratio G1 can be the ratio of the exposure time of the first frame (502) to the exposure time of the second frame (504). For example, if the exposure time of the first frame (502) is twice as long as the exposure time of the second frame (504) and the gain value for both frames is the same, the exposure ratio G1 can be set to two.

[0037] Fig. Figure 8 is a flowchart of an example of a procedure 800 for determining a DLO image value T. DLO (c) for the middle pixel in the first time frame 502 according to some implementations. For a given pixel array, procedure 800 can be performed for one or more pixels in the given pixel array. For the sake of simplicity, procedure 800 is shown in Fig. Figure 8 illustrates and describes a series of processes. However, these processes can occur in different sequences and / or simultaneously and / or in conjunction with other processes not illustrated or described herein.

[0038] At block 802, the first image value T1*(c) is determined. The first image value T1*(c) can be determined using one of the methods described herein (for example, the method 600 described above). Next, at block 804, a second image value T2*(c) is determined as the lower of the second pixel signal T2(c) and the barrier value B1. At block 806, a summed image value T SUM (c) for a combined HDR image is determined by adding the first image value T1*(c) and the second image value T2*(c). In block 806, a summed image value T SUM(c) for a combined HDR image is determined by adding the first image value T1*(c) and the second image value T2*(c). The gain factor G2 can be determined based on the integration time and / or the gain values. In some implementations, HDR images are constructed from more than two exposures. With N exposures, the images T1 to T2 can be N-1 They can be treated similarly to image T1 in the two-exposure example described herein. The barrier level can be set for images T1 to T1. N-1 not the same, for example B 1_1 to B 1_N-1 , and the image T N can be treated in a similar way to T2 in the two-exposure example described herein.

[0039] In some implementations, a noise threshold can be implemented to avoid reducing the quality of the resulting HDR image. For example, if the second pixel signal T2(c) is low, its noise level may be high because the background noise dominates. Using a low second pixel signal to determine a first image value for one of the image sensor pixels 212 that is affected by blooming can reduce the signal-to-noise ratio (SNR) of the combined HDR image. Thus, a noise threshold can be implemented to avoid using low second pixel signals. In some implementations, the first pixel signal T1(c) is compared to a noise threshold B2 to determine when the second pixel signal T2(c) is low.If the first pixel signal T1(c) is less than the noise threshold B2, the first image value T1*(c) cannot be determined using the second pixel signal T2(c), even if one or more of the neighboring pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8) are above the barrier value B1. Instead, the first image value T1*(c) can be set to the value of the first pixel signal T1(c) if the first pixel signal T1(c) is below the noise threshold B2 and one or more of the neighboring pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8) are above the barrier value B1. The noise threshold B2 is less than the barrier value B1.

[0040] Fig. Figure 9 is a flowchart of an example of Procedure 900 for determining the first image value T1*(c) for the middle pixel in the first time frame 502 with noise filtering according to some implementations. For a given pixel array, Procedure 900 can be performed for one or more pixels in the given pixel array. For the sake of simplicity, Procedure 900 is shown in Fig. 9 is shown and described as a series of processes. However, the processes can occur in different sequences and / or simultaneously and / or with other processes not shown and described herein.

[0041] At block 902, the first pixel signal T1(c) is compared to the barrier value B1. If the first pixel signal T1(c) is above the barrier value B1, the first image value T1*(c) at block 904 is set to the barrier value. Alternatively, if the first pixel signal T1(c) is at or below the barrier value B1, the first pixel signal T1(c) at block 906 is compared to the noise threshold B2. If the first pixel signal T1(c) is below the noise threshold B2, the first image value T1*(c) at block 908 is set to the value of the first pixel signal T1(c). Alternatively, if the first pixel signal T1(c) is at or above the noise threshold B2, one or more of the neighboring pixel signals T1(1), T1(2), T1(3), T1(4), T1(5), T1(6), T1(7), and T1(8) are compared with the barrier value B1.For example, at block 910, the neighboring pixel signals T1(1), T1(3), T1(6), and T1(8) of the four neighboring pixels positioned diagonally adjacent to the center pixel are each compared to the barrier value B1. If any of the neighboring pixel signals T1(1), T1(3), T1(6), and T1(8) is above the barrier value B1, the first image value T1*(1) at block 912 is set as the product of the second pixel signal T2(c) and the exposure ratio G1. Alternatively, if any of the neighboring pixel signals T1(1), T1(3), T1(6), and T1(8) is at or below the barrier value B1, the first image value T1*(c) at block 908 is set to the first pixel signal T1(c).

[0042] Fig. Figure 10 illustrates representations of T1 and T2 signals generated by a red pixel and a green pixel of pixel array 210 under green light at different photon flux levels. The photon flux is the number of photons striking pixel array 210 during a unit of time. Fig. 10. The T1 signal generated by the green pixel (referred to herein as the "T1 green pixel signal") increases linearly with the photon flux until the T1 green pixel signal reaches the barrier value B1 and is clipped (for example, as above with respect to block 604 of Fig. 6 described). Furthermore, it increases in Fig. 10. The T1 signal generated by the red pixel (referred to herein as the “T1 red pixel signal”) is non-linear with the photon flux when the T1 signal of the green pixel is limited to the barrier value B1 and the T1 signal of the red pixel is below the barrier value B1. As in Fig. As illustrated in Figure 10, however, the T2 signal generated by the red pixel (referred to herein as the "T2 red pixel signal") is linear across the different photon flux levels at which the T1 signal of the red pixel is non-linear. Using the systems and methods described herein, the non-linear portion of the T1 red pixel signal is replaced by a portion of the T2 red pixel signal that is proportional to the illuminance. The result is, as shown in Fig. Figure 10 illustrates that the initial image values ​​for the red pixel increase linearly with the photon flux until the signal of the red pixel T1 reaches the barrier value B1 and is clipped.

[0043] Fig. Figure 11 is a flowchart of an example of Method 1100 for eliminating blooming artifacts in high dynamic range imaging according to some implementations. For the sake of simplicity, Method 1100 is shown in Fig.The processes are shown in Figure 11 and described as a series of operations. However, these operations can occur in different sequences and / or simultaneously and / or in conjunction with other operations not shown or described herein. In Block 1102, a first pixel signal is detected that is at or below the barrier value B1. The first pixel signal is generated by a central pixel during a first exposure duration. In Block 1104, a multitude of neighboring pixel signals are compared to the barrier value. The multitude of neighboring pixel signals is generated during the first exposure duration. In Block 1106, a first image value for the central pixel is determined based on a second pixel signal if at least one of the multitude of neighboring pixel signals is above the barrier value. The second pixel signal is generated by the central pixel during a second exposure duration that is shorter than the first exposure duration.In block 1108, the first image value is set to the first pixel signal if each of the multitude of neighboring pixel signals is at or below the barrier value. Procedure 1100 can be performed for one or more additional pixels in a pixel array than the center pixel.

[0044] Many of the electrical connections in the drawings are shown as direct couplings without intermediate devices, but are not explicitly identified as such in the preceding description. Nevertheless, this paragraph serves as a reference in the claims for electrical connections without intermediate device(s) shown in the drawing, in order to designate any electrical connection as "directly coupled".

[0045] The foregoing discussion is intended to illustrate the principles and various implementations of the present invention. Numerous variations and modifications will be apparent to the person skilled in the art once the foregoing disclosure is fully understood. It is intended that the following claims be interpreted as encompassing all such variations and modifications.

Claims

[1] Image sensor comprising: a pixel array including a plurality of image sensor pixels; and a controller configured to: for one or more of the plurality of image sensor pixels: Detecting that a first pixel signal is at or below a barrier value, wherein the first pixel signal is generated by a pixel of the plurality of image sensor pixels during a first exposure duration, the pixel being a center pixel, Comparing a plurality of neighboring pixel signals with the barrier value, wherein the plurality of neighboring pixel signals are generated during the first exposure period, and Determining a first image value for the center pixel based on a second pixel signal if at least one of the plurality of neighboring pixel signals is above the barrier value, wherein the second pixel signal is generated by the center pixel during a second exposure duration that is shorter than the first exposure duration. [2] The image sensor of claim 1, wherein, to determine the first image value based on the second pixel signal, the controller is further configured to set the first image value to a product of the second pixel signal and an exposure ratio between the second exposure duration and the first exposure duration. [3] The image sensor of claim 1, wherein the controller is further configured to: Comparing the first pixel signal with a noise threshold value that is lower than the barrier value, and Set the first image value to the first pixel signal if the first pixel signal is below the noise threshold. [4] The image sensor of claim 1, wherein the controller is further configured to set the first image value to the first pixel signal when each of the plurality of neighboring pixel signals is at or below the barrier level. [5] The image sensor of claim 1, wherein the plurality of neighboring pixel signals are generated by image sensor pixels positioned adjacent to the center pixel in a row or a column of the pixel array. [6] The image sensor of claim 1, wherein the plurality of neighboring pixel signals are generated from image sensor pixels positioned diagonally adjacent to the central pixel. [7] The image sensor of claim 1, wherein the controller is further configured to generate a high dynamic range image based on the first image value for each of the one or more of the plurality of image sensor pixels. [8] The image sensor of claim 1, wherein the controller is further configured to read out the first pixel signal, the second pixel signal, and the plurality of neighboring pixel signals. [9] An imaging system comprising: a lens system; an image sensor in a functional relationship to the lens system and comprising a pixel array including a plurality of image sensor pixels; and an imaging controller configured to: for one or more of the plurality of image sensor pixels: Detecting that a first pixel signal is at or below a barrier value, wherein the first pixel signal is generated by a central pixel during a first exposure, Comparing a plurality of neighboring pixel signals with the barrier value, wherein the plurality of neighboring pixel signals are generated during the first exposure, Determining a first image value for the central pixel based on a second pixel signal if at least one of the plurality of neighboring pixel signals is above the barrier value, wherein the second pixel signal is generated from the central pixel during a second exposure which is less than the first exposure, and Setting the first image value to the first pixel signal when each of the plurality of neighboring pixel signals is at or below the barrier value. [10] The imaging system of claim 9, wherein, to determine the first image value based on the second pixel signal, the imaging controller is further configured to set the first image value to a product of the second pixel signal and an exposure ratio between the second exposure and the first exposure. [11] The imaging system of claim 9, wherein the imaging controller is further configured to: Comparing the first pixel signal with a noise threshold value that is lower than the barrier value, and Set the first image value to the first pixel signal if the first pixel signal is below the noise threshold. [12] The imaging system of claim 9, wherein the plurality of neighboring pixel signals are generated by two of the plurality of image sensor pixels positioned adjacent to the center pixel in a row or a column of the pixel array. [13] The imaging system of claim 9, wherein the plurality of neighboring pixel signals are generated by four of the plurality of image sensor pixels positioned diagonally adjacent to the central pixel. [14] The imaging system of claim 9, wherein the plurality of neighboring pixel signals are generated by eight of the plurality of image sensor pixels positioned diagonally around the central pixel. [15] The imaging system of claim 9, wherein the imaging controller is further configured to generate a high dynamic range image based on the first image value for each of the one or more of the plurality of image sensor pixels. [16] The imaging system of claim 9, wherein the imaging controller is further configured to read the first pixel signal, the second pixel signal, and the plurality of neighboring pixel signals from the image sensor. [17] A method for eliminating blooming artifacts in high dynamic range imaging, the method comprising: for one or more of the plurality of image sensor pixels included in a pixel array: Detecting that a first pixel signal is at or below a barrier value, the first pixel signal being generated by a central pixel during a first exposure duration; Comparing a plurality of neighboring pixel signals with the barrier value, wherein the plurality of neighboring pixel signals are generated during the first exposure period; Determining a first image value for the central pixel based on a second pixel signal if at least one of the plurality of neighboring pixel signals is above the barrier value, wherein the second pixel signal is generated by the central pixel during a second exposure duration that is shorter than the first exposure duration; and Setting the first image value to the first pixel signal when each of the plurality of neighboring pixel signals is at or below the barrier value. [18] The method of claim 17, wherein determining the first image value based on the second pixel signal further includes setting the first image value to a product of the second pixel signal and an exposure ratio between the second exposure duration and the first exposure duration. [19] The method of claim 17, further comprising: Comparing the first pixel signal with a noise threshold value that is less than the barrier value; and Set the first image value to the first pixel signal if the first pixel signal is below the noise threshold. [20] The method of claim 17, further comprising generating a high dynamic range image based on the first image value for each of the one or more of the plurality of image sensor pixels.

Images