High dynamic range image sensor with dual carrier pixel architecture

Abstract

To provide a high dynamic range image sensor.SOLUTION: A high dynamic range camera device has an image sensor implementing a readout circuit for reading out, for individual pixels of the sensor, two values based on two different types of charge carriers generated in a response to a light exposure. The two readout values are read out from two floating diffusion nodes of a pixel photodiode designed to collect two types of charge carriers (electrons and holes). In embodiments, a first readout value is read out by a first readout circuit that provides a linear output with respect to a current generated in the photodiode, and a second readout value is read out by a second readout circuit that provides a logarithmic output with respect to the current. The two readout values may be selected or averaged to generate a high dynamic range pixel data based on varying light conditions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates to integrated circuit image sensors and camera devices employing such image sensors.

[0002] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 63 / 549,402, entitled "High Dynamic Range Image Sensor with Dual-Carrier Pixel Architecture," filed February 2, 2024, the disclosure of which is incorporated herein by reference in its entirety. [Background technology]

[0003] Description of Related Art High-dynamic-range (HDR) image sensors are used in cameras to capture scenes without saturation, even when the scene exhibits a wide range of signal intensities. The dynamic range of an image sensor can be measured by the ratio of the pixel's full well capacity (FWC) (the maximum number of carriers the pixel can store without saturating) to the pixel's electronic readout noise floor (readout noise). In a typical complementary metal-oxide semiconductor (CMOS) image sensor, the pixel's FWC is limited by the capacitance of the embedded photodiode (PPD) within the pixel. Higher capacitance allows for a larger FWC. However, higher capacitance also increases the maximum potential of the PPD, making charge transfer more difficult. Therefore, the FWC of PPD-based pixels is typically limited to a few thousand to tens of thousands of electrons, depending on the pixel size. This limits the dynamic range to a <100 dB level, assuming a readout noise of 1 e or greater. To overcome these limitations and achieve higher conversion gain, several pixel and sensor architectures and techniques have been attempted, including multiple exposures with different exposure times, lateral overflow (LOFIC) pixels, and multiple conversion gain pixels. However, all of these techniques suffer from drawbacks such as slow readout speed, high power consumption, fixed pattern noise (FPN), nonlinearity, among others. New approaches are needed in this field to overcome the limitations of current pixel architectures and further extend the dynamic range of image sensors with less compromise to other sensor performance. [Brief explanation of the drawings]

[0004] [Figure 1] 1 illustrates an image sensor implementing pixels having a dual-carrier pixel architecture to provide high dynamic range pixel data, according to some embodiments.

[0005] [Figure 2] 1A and 1B show cross-sectional and schematic diagrams of a pixel of an image sensor providing dual carrier readout according to some embodiments.

[0006] [Figure 3] 10 illustrates the signal output of a pixel that provides dual-carrier readout, according to some embodiments.

[0007] [Figure 4] 10A-10C illustrate timing diagrams of the operation of a pixel that provides dual-carrier readout according to some embodiments.

[0008] [Figure 5] 1 illustrates a process using pixels performed in a dual-carrier pixel architecture to generate image data, according to some embodiments.

[0009] [Figure 6] 1 illustrates a process of calibrating pixels performed in a dual-carrier pixel architecture to generate image data, according to some embodiments.

[0010] [Figure 7] FIG. 1 is a block diagram of an exemplary CMOS image sensor architecture that may be used to implement an image sensor employing a dual-carrier pixel architecture, according to some embodiments.

[0011] [Figure 8] 1 is a block diagram of an exemplary image capture device (e.g., a camera) having an image sensor that provides dual-carrier readout, according to some embodiments.

[0012] [Figure 9] FIG. 1 is a schematic diagram of an exemplary mobile device implementing one or more cameras with image sensors that provide dual-carrier readout, according to some embodiments.

[0013] [Figure 10]1 is a block diagram of an exemplary computing system that includes or hosts an image capture device (e.g., a camera) having an image sensor that provides dual-carrier readout, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0014] To address current challenges in image sensors supporting high dynamic range imaging at the state of the art, various embodiments of an image sensor integrated circuit having a dual-carrier pixel architecture that enables dual-carrier readout from pixels of the image sensor are disclosed.

[0015] In embodiments of the disclosed pixel architecture, different types of charge carriers (e.g., both electrons and holes) generated as a result of exposure to light are collected by the pixel photodiode. In some embodiments, during the pixel's integration time, electrons are collected by an n-type doped well (NW) in the pixel substrate, and holes are collected by a p-type doped well (PW) in the pixel substrate. Both the NW and PW in the substrate may be allowed to float during the integration time so that photon-generated carriers generate voltage signals on two floating diffusion nodes (e.g., the NW as FD1 and the PW as FD2). In some embodiments, electrons collected in FD1 are read out using a linear readout architecture with low readout noise, low FPN, and high linearity, but limited full-well capacity. Meanwhile, holes in FD2 are read out using a logarithmic readout architecture to efficiently compress the signal and extend the pixel's FWC by delaying pixel saturation, but with relatively high readout noise. Depending on the embodiment, the linear / logarithmic readout of electrons / holes may be swapped so that a linear output is produced for holes and a logarithmic output is produced for electrons.

[0016] In some embodiments, the substrates of the image sensor pixels are electrically isolated from each other using full-size deep trench isolation (FTDI). The sidewall interfaces of the FDTI can be passivated with a high-k material with a bias voltage or fixed charge to promote the flow of hole carriers to the PW or FD2. In some embodiments, the FD2 node can be composed of a heavily p-doped material so that it can collect hole carriers in an electric field.

[0017] In some embodiments, the logarithmic readout circuit does not employ a transfer gate. In some embodiments, the logarithmic readout circuit implements a reset transistor that allows leakage current from the floating diffusion node (which acts as a capacitor for the collected charge carriers in FD2). The leakage current is used to measure the voltage of FD2. The reset transistor can discharge FD2 by pulling the node to ground voltage, preparing the node for the next integration in response to exposure.

[0018] In some embodiments, the readout circuitry may require a static substrate bias voltage to be applied, which is incompatible with the operation of the floating diffusion nodes FD1 and FD2. Therefore, in some embodiments, the pixel readout circuitry may be implemented on a substrate separate from the PPD substrate. The two substrates may be metallurgically bonded (e.g., face-to-face) after fabrication.

[0019] The two output values read out from the pixels are finally combined (e.g., selected or merged) to create a high dynamic range output for each pixel. In some embodiments, the electron signal (VE) output is used for the low illumination signal range, and the hole signal (VH) output is used for the high illumination signal range. The combining can be performed on the image sensor integrated circuit or by a separate device, such as a separate image signal processor. In some embodiments, when the linear pixel value is considered saturated (e.g., when the measured intensity of the exposure light exceeds a saturation threshold), the logarithmic pixel value is used. In some embodiments, the saturation threshold can be different for different pixels in the pixel array. In some embodiments, multiple thresholds can be used to implement several saturation layers or ranges, and the two pixel values are combined differently depending on the saturation layer or range (e.g., by adjusting the relative weight of the two values in the averaging of the two values).

[0020] The disclosed dual-carrier pixel architecture achieves several technical advantages over current state-of-the-art image sensors. First, the architecture enables HDR pixel output, providing a low-noise output for low-light conditions while extending the sensor's range for high-light conditions. Second, the disclosed architecture requires only a single exposure to generate two readouts, thereby achieving faster readout times and sensor frame rates. Third, the disclosed architecture does not require additional power to extend the pixel's FWC, which reduces the device's overall power requirements and reduces issues such as output nonlinearity caused by high power. Additionally, the disclosed architecture does not employ overly complex readout circuitry (such as additional transfer gates used in lateral-overflow pixel architectures), which further reduces readout time and power consumption and mitigates other performance issues associated with these conventional pixel architectures. These advantages and features of the disclosed dual-carrier pixel architecture are described in further detail below in conjunction with the drawings.

[0021] FIG. 1 shows an image sensor 100 implementing pixels with a dual-carrier pixel architecture to provide high dynamic range pixel data 182, according to some embodiments.

[0022] As shown, the diagram illustrates an image sensor integrated circuit 100 that implements a pixel array 100 including pixels capable of detecting light. The pixels generate electrical signals in response to the detected light, and the electrical signals are read out from the pixel array 110 via readout circuitry 120. Each pixel in the pixel array 100 includes a light-detecting element, such as a photodiode 140. Exposure 130 causes the photodiode 140 to generate a current 142, which is characterized by the flow of both negative charge carriers 144 (electrons) and positive charge carriers 146 (holes) within the photodiode. As shown, both types of carriers 144 and 146 are collected by the photodiode, here at two floating diffusion nodes FD1 150 and FD2 152. In this example, negative carriers are collected by FD1 and positive carriers are collected by FD2. The two floating diffusion nodes FD1 and FD2 can be designed to collect charge carriers during an integration time without an external bias voltage or in an electric field.

[0023] As shown, the electrical states of the floating diffusion nodes FD1 and FD2 are then read out by pixel readout circuitry 160, which includes an FD1 readout circuit 162 and an FD2 readout circuit 164. The outputs of readout circuits 162 and 164 may be voltage readouts corresponding to the amount of charge carriers stored in the floating diffusion nodes FD1 and FD2. One of the two readout circuits (e.g., FD1 readout 162) performs a linear readout, which produces output values that are linear with respect to the amount of current produced by exposure to light, while the other readout circuit (e.g., FD2 readout 164) performs a logarithmic readout, which produces output values that are logarithmic with respect to the amount of current produced by exposure to light.

[0024] The two readout values generated by the two readout circuits 162 and 164 are then provided to a pixel value combiner component 170, which combines the two values based on a comparison of the exposure 130 intensity with one or more saturation thresholds 172. In some embodiments, the pixel value combiner 170 may simply select one of the readout values to use to generate the image data 180. For example, a logarithmic output value may be used when the exposure 130 saturates the floating diffusion node associated with the linear output value (e.g., when the exposure intensity exceeds the saturation threshold 172). In some embodiments, the two pixel values may be averaged based on the saturation level of the exposure. The averaging may be a weighted average using relative weights for the two pixel values. In some embodiments, multiple saturation thresholds 170 may be used to define several saturation layers or ranges, and the combiner component 170 may adjust the weights of the two values based on the saturation layer or range determined for the exposure 130. For example, as the exposure intensity reaches the next saturation layer, the linear value is weighted less and the logarithmic value is weighted more. In some embodiments, the saturation threshold 172 may be different for different pixels in the pixel array 110, which may be determined based on an initial calibration process. The results of the combiner component 170 are ultimately used to generate HDR pixel data 182 for pixels in the image data 180 of the scene captured by the image sensor 100.

[0025] In some embodiments, pixel value combining component 170 may be implemented as part of image sensor integrated circuit 100, either as part of pixel readout circuit 160 or as an on-sensor digital block, such that the image sensor outputs only one output value per pixel. In other embodiments, pixel value combining component 170 may be located external to image sensor integrated circuit 100, for example on a separate image signal processor.

[0026] It should also be noted that while the diagram shows a single pixel readout circuit 160 for an individual pixel, in various embodiments, the readout circuit 160 may be shared by several adjacent pixels in a pixel group. In some such embodiments, the pixel value combiner 170 may perform its function based on the readout value of the entire group. For example, in some embodiments, the saturation threshold 172 may be compared to the average exposure intensity observed for the entire group. In some embodiments, the pixel value selection or averaging decision of the combiner 160 may be made for all pixels in the pixel group.

[0027] FIG. 2 shows a cross-sectional view and a schematic diagram of a pixel of an image sensor that provides dual carrier readout 162 and 164, according to some embodiments.

[0028] As shown, the top part of the figure shows a cross section of a photodiode 140. As explained, the photodiode includes two floating diffusion nodes FD1 150 and FD2 152 that are read out by FD1 readout circuitry 162 and FD2 readout circuitry 164, respectively, shown in the bottom part of the figure. In some embodiments, the top and bottom parts of the figure may be implemented on two separate substrates because the readout circuits may operate under substrate bias voltages that are incompatible with the operation of the floating diffusion nodes.

[0029] As shown, photodiode 140 includes an n-type doped well 220 and a p-type doped well 230, which result in the flow of charge carriers within the photodiode in response to exposure to light 130. The FD1 node is implemented as an n-type region surrounded by p-type doped well 230. FD1 may be gated by a transfer gate TX1 240, which controls the diffusion of electrons into FD1.

[0030] As shown, the FD2 node is a heavily p-doped region 232 connected to a p-doped well 230. The p-doping promotes the flow of positive charge carriers into the region. The FD2 node is configured to collect holes generated by exposure to light during the integration phase, for example, when the photodiode is floating or ungrounded. As described, in some embodiments, the FD2 node can collect holes without the application of an external bias voltage. In this example, the collected positive charge carriers form the basis of a logarithmic readout.

[0031] As shown, deep trench isolation (DTI) structures 210a and 210b are used to electrically isolate pixels from other pixels in the pixel array. Pixels may be isolated using full trench deep isolation (FTDI), in which the substrate of each pixel is completely isolated from top to bottom. In some embodiments, DTI 210 may be a polysilicon trench that can be electrically biased. A negative bias voltage can be applied to the wall interface of DTI 210 to control the flow of holes to the FD2 node. In some embodiments, DTI 210 may be used to implement a transfer gate that can be electrically driven to control the flow of carriers to the FD2 node.

[0032] As shown, the FD1 readout circuit 162 comprises an n-channel metal-oxide-semiconductor (NMOS) logic device capable of conducting electrons. The FD1 readout circuit 162 includes a reset transistor RST1 260 that can be used to discharge carriers stored at the FD1 node. The circuit also includes a select transistor SEL1 that provides control for timed sampling of the voltage at FD1. The FD1 readout circuit 162 is controllable to generate an output signal VE 270, which is a voltage readout that is linear with respect to the magnitude of the current generated in the photodiode 140 as a result of the exposure 130.

[0033] As shown, the FD2 readout circuit comprises a p-channel metal-oxide-semiconductor (PMOS) logic device capable of conducting holes. The FD2 readout circuit 164 includes a reset transistor RST2 280 and a select transistor SEL2 282. The reset transistor RST2 is controllable to pull the FD2 node to ground, discharging the FD2 node. The reset transistor RST2 is also configured to allow leakage current from FD2 when the FD2 node holds a stored charge and is used to measure the voltage at FD2. This voltage readout, VH 290, is logarithmic with respect to the magnitude of the current generated in the photodiode 140 as a result of the exposure 130.

[0034] FIG. 3 shows the signal output of a pixel that provides dual-carrier readout, according to some embodiments.

[0035] The top graph of the figure shows the VE (electronic) signal 270 as a function of the current generated in the photodiode 140. The VE output 270 can be expressed mathematically as: VE=I phe ×T int ×CG×A sf1 In the formula, I phe is the photodiode electron current, and T int represents the photodiode integration time, CG is the pixel conversion gain, and A sf1 is the source follower voltage gain. The gain values CG and A sf1 is a value that is fixed by the system design. As shown by the equation and the graph, the VE signal 270 is a linear function 310 of the amount of electron current 144 generated in the photodiode.

[0036] The graph at the bottom of the figure shows the VH (hole) signal 290 as a function of the current generated in the photodiode 140. The VH output 290 can be expressed mathematically as: VH=K×A sf2 ×ln(I phh / I0) In the formula, I phh is the photodiode hole current, and A sf2 is the source follower voltage gain, I0 is a leakage current control parameter, and K is a constant. The values K and I0 are process and dimension dependent parameters of the RST2 NMOS 280. The product K×A sf2 , which depends on the characteristics of the readout circuitry and can be calibrated through a calibration process. As shown by the equation and graph, the VH signal 290 is a logarithmic function 320 of the amount of hole current 146 generated in the photodiode.

[0037] FIG. 4 shows a timing diagram of the operation of a pixel that provides dual-carrier readout, according to some embodiments.

[0038] As shown, at time T0, a shutter operation 410 is performed on the pixel. During the shutter operation, both the TX1 NMOS 240 and the RST2 PMOS 280 are turned on to clear the electrons and holes stored in the pixel. As previously mentioned, turning on the RST2 PMOS pulls the FD2 node to ground.

[0039] Following the shutter operation 410, both TX1 and RST2 are turned off, allowing the pixel to enter an integration phase, in which electrons generated by incident light are collected in FD1 and holes generated by incident light are collected in FD2. In some embodiments, during the integration phase, both the FD1 and FD2 nodes are kept floating. The readout phase 420 begins after the integration phase.

[0040] At time T1, the FD2 voltage is sampled to obtain a first signal VH_Sig 442. SEL2 is turned on to enable sampling. Due to the sub-threshold / leakage current characteristics of the RST2 PMOS, the hole current is converted to a voltage signal VH at the FD2 node. The RST2 PMOS transistor then turns on, pulling the FD2 node and the substrate of the pixel to a stable ground.

[0041] At time T2, the voltage on the FD2 node is sampled again to obtain a second signal, VH_Rst 444, which is used as a reference voltage readout for FD2. The VH pixel value 440 is determined as the voltage difference between the two signals, VH_Sig and VH_Rst. In high illumination conditions, the voltage difference between VH_Sig and VH_Rst can be interpreted logarithmically as photohole current intensity. Subsequently, the RST1 NMOS transistor turns on and off to reset the FD1 node.

[0042] At time T3, the voltage at the FD1 node is sampled to obtain VE_Rst 432 as a reference readout. SEL1 is turned on to enable sampling. Following this, the TX1 NMOS is turned on, allowing charge transfer from the photodiode to the FD1 node. At time T4, the FD1 node is sampled again to obtain VE_Sig 434. The VE pixel value 430 is determined as the voltage difference between the two signals, VE_Sig and VE_Rst. In low lighting conditions, the voltage difference between VE_Rst and VE_Sig can be linearly converted to the number of accumulated electrons per integration time.

[0043] Note that in this example, there is no transfer gate used for hole-side (VH signal) readout. Therefore, FD2 is directly diffused with holes as a result of exposure. This transfer gate-less approach may incorporate more readout noise into the output signal because there is no correlated double sampling to cancel out the noise. However, because the design simplifies the readout circuitry and the VH pixel value 440 is only used for high lighting conditions, a small amount of increased noise may be considered acceptable.

[0044] It should also be noted that in other embodiments of dual-carrier pixel architectures, variations in the readout process may be performed. For example, in some embodiments, the VE signal readout may be obtained before the VH signal readout. In some embodiments, the VH and VE values may be read out together in parallel. In some embodiments, a saturation assessment may be performed before the readout so that either the VH or VE values are not sampled if not needed.

[0045] FIG. 5 illustrates a process for using pixels implemented in a dual-carrier pixel architecture (eg, as shown in FIGS. 1 and 2) to generate image data, according to some embodiments.

[0046] The process begins at operation 510, where a current is generated in a photodiode of a pixel (e.g., photodiode 140) in response to an exposure (e.g., exposure 130). The pixel may be part of a pixel array in an image sensor (e.g., image sensor integrated circuit 100) configured to capture successive frames of a scene based on repeated exposures.

[0047] In operation 520, a first type of charge carrier (e.g., electrons) of the current is collected at a first floating diffusion node (e.g., FD1 150) of the pixel. In some embodiments, the FD1 node may be a negative region within the photodiode surrounded by a p-type doped well (e.g., p-type doped well 230). In some embodiments, the diffusion of FD1 may be controlled by a transfer gate (e.g., TX1 240).

[0048] In operation 530, a second type of charge carrier (e.g., holes) of the current is collected at a second floating diffusion node (e.g., FD2 152) of the pixel. In some embodiments, the FD2 node may be a heavily p-doped region (e.g., region 232) in the photodiode connected to the p-doped well that is more heavily doped than the rest of the p-doped well. The p-doping of FD2 causes holes to flow into the node in response to light exposure. In some embodiments, the diffusion of FD2 may occur directly as a result of light exposure without being controlled by a transfer gate. In some embodiments, the diffusion of FD2 may occur without an external bias. In some embodiments, the diffusion of FD2 may occur when another structure in the photodiode (e.g., DTI 210) is biased. In some embodiments, another structure(s) in the photodiode may be used to implement a transfer gate that controls the diffusion of holes into FD2.

[0049] In operation 540, a first readout value of the FD1 node (e.g., VE 270) is output via a first readout circuit (e.g., FD1 readout 162). The first readout value is linear with respect to the amount of current generated in the photodiode as a result of the exposure to light, as described in connection with FIG.

[0050] In operation 550, a second readout value (e.g., VH 290) of the FD2 node is output via a second readout circuit (e.g., FD2 readout 164). The second readout value is logarithmic with respect to the amount of current generated in the photodiode as a result of the exposure to light, as described in connection with FIG. 3. In some embodiments, the logarithmic readout circuit may implement a transistor (e.g., RST2 280) that allows leakage current from the FD2 node. This leakage current is used to measure the voltage at FD2.

[0051] In operation 560, a first pixel value (e.g., VE pixel value 430) is determined using the first readout, and a second pixel value (e.g., VH pixel value 440) is determined using the second readout. As described in connection with FIG. 4, the two pixel values 430 and 440 can be determined from two readouts (e.g., readouts 432 and 434 for VE pixel value 430 and readouts 442 and 444 for VH pixel value 440) from each respective readout circuit, and then the difference of the two readouts can be taken. For logarithmic readouts, a linearization step can be performed to convert the logarithmic readout signal to a linear signal.

[0052] Operation 560 then determines how the two pixel values should be used to generate image data corresponding to the exposure (e.g., HDR pixel data 182) based on one or more saturation threshold(s) (e.g., threshold 172). In some embodiments, the determination may involve selecting whether to use the first pixel value, the second pixel value, or both to generate the image data. For example, in some embodiments, the second pixel value (e.g., a logarithmic value) may be used when the exposure exceeds the saturation threshold (e.g., when the pixel observes light in high lighting conditions). As another example, the first and second pixel values may be combined to generate the image data, for example, using an averaging function. The averaging may employ weights (e.g., linear weights) for the two pixel values that are adjustable based on the saturation level of the pixel (e.g., based on one of several saturation layers or ranges). The combining function may also perform per-pixel corrections such as gain, offset, linearity, etc.

[0053] Depending on the embodiment, the saturation threshold(s) may be set globally for all pixels on the pixel array or may be different for individual pixels in the pixel array. The saturation threshold may be determined based on an initial calibration process. Depending on the embodiment, operation 560 may be performed on the image sensor (e.g., as part of the readout circuitry or downstream digital blocks) or by a separate device different from the image sensor (e.g., on a separate image signal processor).

[0054] FIG. 6 illustrates a process 600 for calibrating pixels performed in a dual-carrier pixel architecture (eg, as shown in FIGS. 1 and 2) to generate image data, according to some embodiments.

[0055] In some embodiments, due to variations in the electrical characteristics of the logarithmic side readout circuit, particularly the RST2 PMOS 280, the output of the readout circuit will vary from pixel to pixel, which can lead to problems such as FPN. To mitigate such problems, a pixel-by-pixel calibration process 600 is used to eliminate (or at least reduce) the readout variations.

[0056] During the calibration process, readouts from the two readout circuits are performed for a single exposure. The calibration process assumes that the amount of electron and hole current generated by the exposure is the same, and that a linear readout accurately reflects the amount of current generated. These assumptions are then used to calculate one or more calibration parameters that reconcile the logarithmic readout with the linear readout. In some embodiments, the calibration is performed using the equations for VE and VH discussed above, and I for the test exposure. phe =I phh Assuming that the linear readout value VE is I phe is assumed to be the given correct value. Therefore, the calculation begins with calculating I using the VE equation and the observed VE. phe and then solve for the observed VH values and I phh to solve the calibration parameters of the VH equation, which is Iphe In some embodiments, the calibration process is performed by adjusting the parameter K×A used in the VH formula. sf2 Only two test exposures per pixel are used to calibrate VH. In general, calibration techniques can use three or more pairs of (VE, VH) test readouts and can be used to independently calibrate multiple variables of the VH signal.

[0057] Referring to the figure, in operation 610, the process adjusts the light intensity for a test exposure and exposes the pixel under test to light. In operation 620, a pair of VH and VE readings is sampled from the pixel for the test exposure. Operation 630 repeats operations 610 and 620 until a sufficient number of test readout pairs (VH, VE) have been collected. Readout pairs with saturated VE readings are discarded because these readings are not reliable for calibration purposes. The light intensity for each test exposure may be selected randomly or using some heuristic to optimize calibration accuracy and / or speed.

[0058] In operation 640, assuming the VE readout is correct, the collected (VH,VE) pairs are used to calculate one or more calibration values for the VH readout circuit. As explained, the calculations can assume that the hole and electron currents produced by a single exposure are the same, and this assumption is used to calculate the calibration parameter(s) for VH.

[0059] In operation 650, the calibration value(s) determined from operation 640 are saved to persistent storage. These calibration values may be stored in the image sensor itself or in a separate device such as a separate image signal processor. These calibration value(s) are used after the calibration process to adjust or correct subsequent readouts from the VH readout circuitry.

[0060] FIG. 7 is a block diagram of an exemplary CMOS image sensor architecture that may be used to implement an image sensor 100 employing a dual-carrier pixel architecture, according to some embodiments.

[0061] The pixel array 110 may include a number of pixels arranged in an M×N array. Individual pixels in the pixel array may be implemented according to the dual-carrier pixel architecture described in connection with Figures 1 and 2. For example, each pixel in the array may implement two floating diffusion nodes (FD1 and FD2) configured to collect different types of carriers.

[0062] The row addressing and row driver circuit 710 generates readout signals to the pixel array 110, which may include transfer gate (TG) control signals, row select (RS) signals, and reset drain (RD) control signals. In some embodiments, the column readout circuit 720 may be configured to perform per-pixel gain selection as described herein and may include analog-to-digital circuitry for sampling and / or digitizing output values read out from the pixel array 110. The circuit 720 may be implemented with multiple A / D converters configured to perform column-parallel readout. In some embodiments, the circuit 720 may be configured such that the readout circuit associated with each column bus may have a separate analog-to-digital converter (ADC), although in some embodiments, pairs of columns may share an ADC. As shown, the column readout circuit 720 in this example implements a VE readout circuit 722 and a VH readout circuit 724, which implement FD1 and FD2 readout circuits for individual pixels or groups of pixels in the pixel array 100. As discussed, in some embodiments, the pixel array 110 and readout circuits 710 and 720 may be implemented on separate substrates, so that a substrate bias voltage can be applied to the readout substrate without interfering with the operation of the floating diffusion nodes of the pixels.

[0063] The peripheral circuits 730 are used to perform functions separate from the readout circuits 710 and 720. In some embodiments, the peripheral circuits 730 may implement timing and control circuitry used to control both the row addressing and row driver circuitry 710 and the column readout circuitry 720. For example, the timing and control circuitry controls the row addressing and row driver circuitry 710 to select the appropriate rows for readout and can provide timing control signals according to, for example, a rolling shutter readout or a global shutter readout. As shown, the timing and control circuitry 730 may also provide communications capabilities to communicate or interface with a host 750 (e.g., a processor associated with a camera system comprising an image sensor), which, in some implementations, may specify various control information.

[0064] In some embodiments, the signals on the column readout bus are sampled and digitized by circuitry within the column readout circuit 720, and the digitized pixel values provided by the ADC may be provided to a line buffer that may be used to temporarily store digital signals from the column readout circuit 720 for use by an image processor implemented by the digital block(s) 740. In general, any number of line buffers may be included, and for example, each line buffer may be capable of storing digital signals representing the charge signals that may be read out from each pixel in a given row of pixels in the pixel array 110. The image processor may be used to process the digital signals held in the line buffers to generate output image data that may be provided to a device external to the image sensor. In some embodiments, image processing may be performed on a separate integrated circuit as the image sensor integrated circuit 100.

[0065] FIG. 8 is a block diagram of an exemplary image capture device 800 (eg, a camera device) having an image sensor that provides dual-carrier readout, according to some embodiments.

[0066] 8 , in some embodiments, image capture device 800 can include one or more lenses 802 and an image sensor 804, which is an embodiment of image sensor 100 described above. In some embodiments, image capture device 800 can capture light from the environment (e.g., exposure 130), which can pass through lens 802 and reach image sensor 804. In some embodiments, image sensor 804 can implement multiple pixels (e.g., in pixel array 110), at least some of which implement a dual-carrier readout architecture as described above. In some embodiments, image sensor 504 is a CMOS image sensor. In some embodiments, image capture device 800 can be a video camera capable of capturing successive frames of a scene.

[0067] In some embodiments, image capture device 800 may include a filter 806 (e.g., an infrared blocking filter) disposed between lens 802 and image sensor 804 to block some light from reaching image sensor 804. As shown in this example, image sensor 804 and filter 806 may be mounted on a substrate 808, and image sensor 804 may be positioned upside down to receive backside illumination. Alternatively, in some embodiments, front side illumination may be implemented on image sensor 804.

[0068] As explained, the dual-carrier pixel readout architecture described herein enables camera 800 to capture HDR images at relatively high frame rates and using less power compared to camera devices that use image sensors that do not employ the dual-carrier pixel readout architecture. As will be appreciated by those skilled in the art, the advantages created by the dual-carrier pixel design form the basis of features that are essential drivers of demand for cameras in the marketplace.

[0069] FIG. 9 is a schematic diagram of an example mobile device 900 implementing one or more cameras 904 having image sensors that provide dual-carrier readout, according to some embodiments.

[0070] In some embodiments, the mobile device 900 may be a multi-function user device such as a smartphone, personal computer, laptop, notebook, tablet, slate, pad, or netbook computer, handheld computer, digital camera, video recording device, augmented reality (AR) and / or virtual reality (VR) headset, handheld video game device, among others. In some embodiments, the mobile device 900 may be battery-powered so that it can operate for a period of time without access to a fixed power source (e.g., a wall power socket). In these types of mobile devices, the voltage output of the battery may decrease over time, resulting in some powered components of the device performing poorly or becoming inoperable after a period of use without recharging. As explained, the dual-carrier pixel readout architecture described herein enables the camera 904 on the mobile device to capture HDR image data using less power. Thus, a mobile device employing a dual-carrier pixel readout architecture can operate longer without power recharge or can employ smaller batteries with reduced charge capacity. As shown, in some embodiments, the dual-carrier pixel readout architecture enables the mobile device to operate multiple camera devices 904 simultaneously. Thus, the advantages enabled by the dual carrier image sensor designs described herein form the basis of features that are essential drivers of the demand for these mobile devices in the marketplace.

[0071] In some embodiments, mobile device 900 may include a display system 902 (e.g., including a display and / or a touch-sensitive surface) and / or one or more cameras 904. In some non-limiting embodiments, display system 902 and / or one or more front-facing cameras 804a may be provided on the front side of device 900. Additionally or alternatively, one or more rear-facing cameras 904b may be provided on the rear side of device 900. In various embodiments, the location(s) and / or configuration(s) of camera(s) 904 may differ from that shown in FIG. 9 . Depending on the embodiment, some or all of the cameras 904 may be the same or similar to one another. For example, some of the cameras 904 may implement a dual-carrier pixel readout architecture, as described herein. Some of the cameras 904 may be video cameras configured to capture HDR image data, as described in connection with FIG. 8 .

[0072] Depending on the embodiment, the mobile device 900 may operate several electrically powered components, including, among others, memory 906 (e.g., comprising an operating system 908 and / or application(s) / program instructions 910), one or more processors and / or controllers 912 (e.g., CPU(s), memory controller(s), display controller(s), and / or camera controller(s), etc.), and / or one or more sensors 916 (e.g., orientation sensor(s), proximity sensor(s), and / or position sensor(s), etc.). In some embodiments, the device 900 may implement a network interface for communicating with one or more other devices and / or services, such as computing device(s) 918, cloud service(s) 920, etc., via one or more networks 922. For example, device 900 may include a wireless interface that enables device 900 to transmit data to and receive data from wireless network(s) using various communication standards, protocols, and / or technologies.

[0073] FIG. 10 is a block diagram of an exemplary computing system 1000 that includes or hosts an image capture device (e.g., one or more cameras 1008) having an image sensor that provides dual-carrier readout, according to some embodiments.

[0074] Computer system 1000 may execute methods for controlling the operation of camera 1008 and / or for performing image processing of images captured using camera 1008. In some embodiments, camera device 800 of FIG. 8 or mobile device 900 of FIG. 9 may execute some or all of the functional components of computer system 1000 described herein. Embodiments of computer system 1000 may be configured to execute any or all of the embodiments described above.

[0075] In various embodiments, computer system 1000 may be any of a variety of types of device, including, but not limited to, a personal computer system, a desktop computer, a laptop, a notebook, a tablet, a slate, a pad, or a netbook computer, a mainframe computer system, a handheld computer, a workstation, a network computer, a camera, a set-top box, a mobile device, an augmented reality (AR) and / or virtual reality (VR) headset, a consumer device, a video game console, a handheld video game device, an application server, a storage device, a television, a video recording device, a peripheral device such as a switch, modem, router, or generally any type of computing or electronic device.

[0076] In the illustrated embodiment, computer system 1000 includes one or more processors 1002 coupled to system memory 1004 via an input / output (I / O) interface 1006. Computer system 1000 further includes one or more cameras 1008 coupled to I / O interface 1006. Computer system 1000 further includes a network interface 1010 coupled to I / O interface 1006, and one or more input / output devices 1012, such as a cursor control device 1014, a keyboard 1016, and a display(s) 1018. While in some cases, embodiments may be implemented using a single instance of computer system 1000, it is contemplated that in other embodiments, multiple such systems, or multiple nodes comprising computer system 1000, may be configured to host different portions or instances of an embodiment. For example, in one embodiment, some elements may be implemented via one or more nodes of computer system 1000 that are different from the nodes implementing other elements.

[0077] In various embodiments, computer system 1000 may be a uniprocessor system including one processor 1002, or a multiprocessor system including multiple processors 1002 (e.g., two, four, eight, or another suitable number). Processor 1002 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 1002 may be a general-purpose or embedded processor implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable instruction set architecture (ISA). Also, in some embodiments, one or more of processors 1002 may include additional types of processors, such as a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or the like. In a multiprocessor system, each of processors 1002 may generally, but not necessarily, implement the same ISA. In some embodiments, computer system 1000 may be implemented as a system-on-a-chip (SoC). For example, in some embodiments, the processor 1002, memory 1004, I / O interface 1006 (e.g., fabric), etc. may be implemented in a single SoC with multiple components integrated on a single chip. For example, the SoC may include multiple CPU cores, a multi-core GPU, a multi-core neural engine, a cache, one or more memories, etc. integrated on a single chip. In some embodiments, the SoC implementation may implement a reduced instruction set computing (RISC) architecture, or any other suitable architecture.

[0078] The system memory 1004 may be configured to store program instructions 1020 accessible by the processor 1002. In various embodiments, the system memory 1004 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash-type memory, or any other type of memory. Additionally, existing camera control data 1022 in the memory 1004 may include any of the information or data structures described above. In some embodiments, the program instructions 1020 and / or data 1022 may be received, sent, or stored on different types of computer-accessible media, or on similar media separate from the system memory 1004 or the computer system 1000. In various embodiments, some or all of the functionality described herein may be performed via such a computer system 1000.

[0079] In one embodiment, I / O interface 1006 may be configured to coordinate I / O traffic between processor 1002, system memory 1004, and any peripheral devices within the device, including other peripheral interfaces such as network interface 1010 or input / output devices 1012. In some embodiments, I / O interface 1006 may perform any necessary protocol, timing, or other data conversions to convert data signals from one component (e.g., system memory 1004) into a format suitable for use by another component (e.g., processor 1002). In some embodiments, I / O interface 1006 may include support for devices attached via various types of peripheral buses, such as, for example, variants of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard. In some embodiments, the functionality of I / O interface 1006 may be split between two or more separate components, such as, for example, a northbridge and a southbridge. Additionally, in some embodiments, some or all of the functionality of the I / O interface 1006 , such as the interface to the system memory 1004 , may be incorporated directly into the processor 1002 .

[0080] Network interface 1010 may be configured to allow data to be exchanged between computer system 1000 and other devices (e.g., carrier or agent devices) attached to network 1024, or between nodes of computer system 1000. Network 1024, in various embodiments, may include one or more networks, including, but not limited to, a local area network (LAN) (e.g., an Ethernet or enterprise network), a wide area network (WAN) (e.g., the Internet), a wireless data network, some other electronic data network, or some combination thereof. In various embodiments, network interface 1010 may support communication over a wired or wireless general-purpose data network, such as, for example, any suitable type of Ethernet network. It may also support communication over a telecommunications / telephone network, such as an analog voice network or a digital fiber communications network, a storage area network, such as a Fibre Channel SAN, or any other suitable type of network and / or protocol.

[0081] The input / output devices 1012, in some embodiments, may include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for inputting or accessing data by one or more computer systems 1000. Multiple input / output devices 1012 may be present within computer system 1000 or may be distributed on various nodes of computer system 1000. In some embodiments, similar input / output devices may be separate from computer system 1000 and may interact with one or more nodes of computer system 1000 through wired or wireless connections, such as via network interface 1010.

[0082] Those skilled in the art will appreciate that computer system 1000 is merely exemplary and is not intended to limit the scope of the embodiments. In particular, computer systems and devices may include any combination of hardware or software capable of performing the depicted functions, including computers, network devices, Internet appliances, PDAs, wireless telephones, pagers, etc. Computer system 1000 may also be connected to other devices not shown, or alternatively, may operate as a stand-alone system. Additionally, functionality provided by the illustrated components may, in some embodiments, be combined in fewer components or distributed among additional components. Similarly, in some embodiments, the functionality of some of the illustrated components may not be provided, and / or other additional functionality may be available.

[0083] Those skilled in the art will also understand that while various items are shown as being stored in memory or on a storage device during use, these items, or portions thereof, may be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments, some or all of the software components may execute in memory on another device and communicate with the illustrated computer system via computer-to-computer communications. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or portable item to be read by an appropriate drive, various examples of which are described above. In some embodiments, instructions stored on a computer-accessible medium separate from computer system 800 may be transmitted to computer system 800 via a transmission medium or signal, such as an electrical, electromagnetic, or digital signal, conveyed over a communications medium, such as a network and / or a wireless link. Various embodiments may further include receiving, sending, or storing instructions and / or data to be executed on a computer-accessible medium in accordance with the preceding description. Generally speaking, a computer-accessible medium may include a non-transitory computer-readable storage medium or memory medium, such as a magnetic medium or an optical medium, e.g., a disk or DVD / CD-ROM, a RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), a volatile or non-volatile medium, such as a ROM, etc. In some embodiments, a computer-accessible medium may include a transmission medium or a signal, such as an electrical, electromagnetic, or digital signal, conveyed over a communication medium, such as a network and / or a wireless link.

[0084] The methods described herein may, in different embodiments, be implemented in the form of software, hardware, or a combination thereof. In addition, the order of method blocks may be changed, and various elements may be added, reordered, combined, omitted, modified, etc. Various modifications and variations may be made as would be apparent to one of ordinary skill in the art having the benefit of this disclosure. The various embodiments described herein are illustrative and not limiting. Many variations, modifications, additions, and improvements are possible. Accordingly, multiple variations may be provided for components described herein as a single instance. Boundaries between various components, operations, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific exemplary configurations. Other allocations of functionality are contemplated and may be included within the scope of the following claims. Finally, structures and functions presented as separate components in exemplary configurations may be implemented as combined structures or components. These and other variations, modifications, additions, and improvements may be included within the scope of the embodiments, as defined by the following claims.

Claims

1. 1. An image sensor, comprising: a plurality of pixels, each of said pixels comprising: (a) a photodiode that generates a current in response to exposure to light; (b) a first floating diffusion (FD1) node that collects charge carriers of a first type in said current; (c) a second floating diffusion (FD2) node that collects charge carriers of a second type in the current; and a first readout circuit for the pixel that outputs a first readout value of the FD1 node that is linear with respect to the amount of current generated in the photodiode; a second readout circuit for the pixel that outputs a second readout value of the FD2 node that is logarithmic with respect to the amount of current generated in the photodiode; The second readout is used to generate image data corresponding to the exposure when the first signal exceeds a saturation threshold.

2. The FD1 node collects electrons, The image sensor of claim 1 , wherein the FD2 node collects holes.

3. 10. The image sensor of claim 1, wherein the pixels are electrically isolated using full-size deep trench isolation (FDTI).

4. the FDTI is implemented using a polysilicon trench; The image sensor of claim 3 , wherein the polysilicon trench is biased to control carrier flow to the FD2 node.

5. The image sensor of claim 1 , wherein the FD2 node includes a p-type doped semiconductor region of the photodiode that collects holes generated by the photodiode when the photodiode is in a floating state.

6. the first readout circuit includes a first transfer gate that controls when charge carriers from a p-type doped well are transferred to the FD1 node; The image sensor of claim 1 , wherein the second readout circuit does not employ a transfer gate for the FD2 node.

7. The second readout circuit includes a reset transistor, the reset transistor Leakage current from the FD2 node allows the accumulated charge at the FD2 node to be measured; 10. The image sensor of claim 1, controllable in response to a reset signal to pull the FD2 node to ground voltage, discharging the FD2 node.

8. obtaining two readouts from the FD2 node, the readouts including a first value before the reset of the FD2 node and a second value after the reset; The image sensor of claim 1 , configured to determine a pixel value for the pixel based on a difference between the first value and the second value.

9. the pixel array is mounted on a first substrate; The image sensor of claim 1 , wherein the second readout circuitry is implemented on a second substrate that is different from the first substrate.

10. 2. The image sensor of claim 1, configured to select between a first pixel value determined using the FD1 node and a second pixel value determined using the FD2 node for use in generating the image data.

11. 2. The image sensor of claim 1, configured to determine that the exposure received by the pixel is within a specified range, and to accordingly combine pixel values determined from the FD1 node and pixel values determined from the FD2 node for use in generating the image data.

12. The image sensor of claim 1 , wherein different pixels in the pixel array have different saturation thresholds.

13. 1. A camera device, comprising: Lenses and 1. An image sensor, comprising: a plurality of pixels, each of said pixels comprising: (a) a photodiode that generates a current in response to light exposure received through said lens; (b) a first floating diffusion (FD1) node that collects charge carriers of a first type in said current; (c) a second floating diffusion (FD2) node that collects charge carriers of a second type in the current; and a first readout circuit for the pixel that outputs a first readout value of the FD1 node that is linear with respect to the amount of current generated in the photodiode; a second readout circuit for the pixel that outputs a second readout value of the FD2 node that is logarithmic to the amount of current generated in the photodiode; an image signal processor, the image signal processor comprising: a camera device configured to use the second readout to generate image data corresponding to the exposure when the first signal exceeds a saturation threshold.

14. the camera device is a mobile device; The camera device of claim 13 , wherein the camera device is a video camera configured to capture video data.

15. 1. A method comprising: receiving an exposure at a pixel array of pixels; generating a current in a photodiode of one of the pixels in response to the exposure to light; collecting charge carriers of a first type of the current at a first floating diffusion (FD1) node of the pixel; collecting a second type of charge carrier of the current at a second floating diffusion (FD2) node of the pixel; outputting a first readout of the FD1 node via a first readout circuit of the pixel, the first readout being linear with respect to the amount of current generated in the photodiode; outputting a second readout of the FD2 node via a second readout circuit of the pixel, the second readout being logarithmic with respect to the amount of current generated in the photodiode; selecting, based on a saturation threshold, a first pixel value determined using the first readout or a second pixel value determined using the second readout to generate image data corresponding to the exposure; A method comprising:

16. obtaining two readouts from the FD2 node, the readouts including a first value before the reset of the FD2 node and a second value after the reset; The method of claim 15 , further comprising: determining the second pixel value based on a difference between the first value and the second value.

17. The image sensor The method further includes using a reset transistor of the second readout circuit to control operation of the second readout circuit, wherein using and controlling includes: allowing leakage current from the FD2 node to measure accumulated charge at the FD2 node; and pulling the FD2 node to a ground voltage to reset the FD2 node.

18. the first readout circuit and the second readout circuit are implemented on an image sensor integrated circuit device; 16. The method of claim 15, wherein the selecting the first pixel value or the second pixel value to use to generate the image data is performed in an image processor separate from the image sensor integrated circuit device.

19. 16. The method of claim 15, further comprising: determining that another exposure received by the pixel is within a specified range; and responsively combining pixel values determined from the FD1 node and the FD2 node to generate image data corresponding to the another exposure.

20. performing a calibration of the pixels, obtaining a linear readout of the pixel via the first readout circuit corresponding to a test exposure, the linear readout being linear with respect to the amount of electron current generated in the photodiode in response to the test exposure; obtaining a logarithmic readout of the pixel via the second readout circuit corresponding to the test exposure, the logarithmic readout being logarithmic with respect to the amount of hole current generated in the photodiode in response to the test exposure; calculating one or more calibration values for the second readout circuit based on pairs of linear and logarithmic readout values obtained from a plurality of test exposures, the calculation solving for the one or more calibration values assuming that the amount of hole current is equal to the amount of electron current for each test exposure; 16. The method of claim 15, further comprising: saving the one or more calibration values in persistent storage, the one or more calibration values being used to adjust subsequent readouts from the second readout circuit after the calibration.

Images