Autofocus system and method for imaging sensor with multiple conversion gains

Abstract

Methods and devices for imaging with autofocus based on phase difference, PD, use three or more PD gain configurations. A dual conversion gain (DCG) imaging sensor, which includes a plurality of PD pixels, receives (802) light and selects (804) one of three or more DCG PD gain configurations to autofocus, AF, a lens associated with the DCG imaging sensor. The three or more DCG PD gain configurations are determined based on a single gain configuration for the plurality of PD pixels. A PD generator module associated with the DCG imaging sensor applies (806) the selected one of the three or more DCG PD gain configurations to the DCG imaging sensor.

Description

AUTOFOCUS SYSTEM AND METHOD FOR IMAGING SENSOR WITH MULTIPLE CONVERSION GAINSFIELD OF THE DISCLOSURE

[0001] This document generally describes methods and devices operating to acquire an image of an object such as (but not limited to) a smartphone camera that has a dual conversion gain (DCG) imaging sensor with an autofocus capability.BACKGROUND

[0002] Traditional DCG image pixels in a DCG imaging sensor of an imaging device (e.g., camera, smartphone, etc.) are configured to generate two pixel outputs, characterized by different conversion gains. One pixel output has a low-conversion gain (LCG) configuration, and the other pixel output has a high-conversion gain (HCG) configuration. A High Dynamic Range (HDR) mode of the imaging device uses the two LCG and HCG configurations for capturing accurate images in high and low ambient illumination conditions. While the LCG configuration is suited for good to high illumination conditions and the HCG configuration is suited for low illumination conditions, the HDR mode uses (combines) both configurations for obtaining accurate images.

[0003] The imaging device also has an autofocus (AF) mode. The AF mode relies on pixel-based phase difference (PD) measurements generated by PD pixels. PD pixels, different from the DCG image pixels, are configured to measure a phase difference between a left image and a right image of the same object. Based on the measured phase difference, a procedure associated with the AF mode calculates how much to move a lens and in what direction, to focus the lens of the camera.

[0004] The AF mode uses one of the two pixel outputs produced by the DCG sensor in the HDR mode for generating DCG PD gain configurations. The traditional DCG PD gain configurations only include an LCG PD configuration and an HCG PD configuration. The DCG PD gain configurations are based on the same output of the PD pixels but combining different number of pixels (instead of using different conversiongains). However, the LCG and HCG PD gain configurations are insufficient for the AF mode to handle various brightness ranges in multiple scenes, from low to high illumination conditions.SUMMARY

[0005] According to an embodiment, an AF control system of the imaging device uses, in addition to the LCG and HCG PD configurations, a mid-conversion gain (MCG) PD configuration. A DCG-HDR configuration (i.e. , LCG and HCG configurations) refers to an output of a single image pixel while a DCG PD gain configuration (i.e., LCG, MCG, and HCG PD configurations) refers to an output of a combination of sub-pixels of the PD pixels. In one embodiment, each DCG PD gain configuration uses the same DCG-HRD configuration.

[0006] The MCG PD configuration combines the output from a few sub-pixels of the PD pixels. The number of sub-pixels for this configuration is larger than the number of sub-pixels used by the LCG PD configuration and smaller than the number of sub-pixels used by the HCG PD configuration. More than one MCG PD configuration may be introduced. In this way, the MCG PD configuration improves sensitivity and signal quality achieved by the imaging device by increasing the number of sub-pixels read out when compared to the legacy DCG PD systems.

[0007] The DCG PD gain configurations are implemented on the DCG sensor side. In one embodiment, each DCG PD gain configuration uses only left or only right sub-pixels of the PD pixels. In another embodiment, each of the DCG PD gain configurations uses the same DCG-HDR conversion gain per pixel.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate one or more embodiments and, together with the description, explain these embodiments.

[0009] FIG. 1 is a block diagram of an imaging device having plural PD pixels and a DCG controller for generating, selecting, and applying one of three or more DCG PD gain configurations according to an embodiment.

[0010] FIG. 2 is a functional diagram of the imaging device of FIG. 1 illustrating various modules of the DCG controller according to an embodiment.

[0011] FIGs. 3A and 3B illustrate sub-pixel combinations used for generating LCG and HCG PD gain configurations, respectively.

[0012] FIGs. 4A to 4C illustrate sub-pixel combinations used for generating LCG, MCG, and HCG PD gain configurations, respectively, according to an embodiment.

[0013] FIG. 5 is a table that compares the DCG PD gain configurations and their signals according to an embodiment.

[0014] FIG. 6 is a data flow of the DCG controller implemented at an AF control system according to an embodiment.

[0015] FIG. 7 is a schematic diagram of states corresponding to the three DCG PD gain configurations and associated state transitions according to an embodiment.

[0016] FIG. 8 is a flowchart of a method for generating, selecting, and applying one of the three or more DCG PD gain configurations in an imaging device according to an embodiment.DETAILED DESCRIPTION

[0017] Methods and devices described in this section embody techniques related to a PD-based AF procedure performed by an imaging device that includes a DCG sensor. The associated AF module varies the number of sub-pixels / pixels that are binned together and using a same pixel conversion gain, to differentiate an output signal level, i.e. , the brightness. These features are discussed now in more detail.

[0018] The embodiment descriptions in this section refer to the accompanying drawings. The same reference numbers in different drawings identify the same or similar elements. The detailed descriptions do not preclude other embodiments within the scope of the appended claims. The embodiments are not limited to the described configurations but may be extended to other arrangements.

[0019] FIG. 1 schematically illustrates a cross-section of an imaging device 100, e.g., a camera. Camera 100 may be a standalone camera, a smartphone camera, a tablet camera, a car-mounted camera, a medical camera, or any other type of camera that includes an imaging sensor having plural image pixels configured with DCG. Such an imaging sensor is called a “DCG sensor” in this disclosure. Camera 100 includes a lens unit 110 and a camera body 120. The lens unit 110 includes at least a lens 112 for focusing incoming light 113 into one or more pixels 124. The lens unit 110 may further include additional lenses, a diaphragm, a zoom actuator, a focus actuator, and associated drivers (not shown). As these units are known in the art, their description is omitted herein. The lens unit 110 may also include a dedicated processor and associated memory (not shown) for supporting the above noted functions.

[0020] The camera body 120 houses an imaging sensor 122 (DCG sensor) and the imaging sensor 122 includes plural image pixels 123. The imaging sensor 122 may be configured as a Bayer sensor. The imaging sensor 122 may be a complementary metal- oxide sensor (CMOS) and the image pixels 123 may have any known structure used in the camera field. The imaging sensor 122 further includes PD pixels 124. The image and PD pixels may be interleaved. The PD pixels 124 are configured to receive light beams 113 and to individually output signals independent of the other PD pixels. The PD pixels 124 are further configured, different from a traditional image pixel 123, to record a left image and a right image (and / or top and bottom images) of a same object (not shown) forcalculating a defocus amount by an imaging plane PD AF method, called in this disclosure “PDAF.” Each of an imaging pixel 123 and a PD pixel 124 includes at least a photodetector (not shown) which is configured to transform the incoming light into electrical charges. The pixels also include corresponding electronics 126 (shown only for the PD pixel 124), i.e., capacitors, transistors, etc. for storing and manipulating the electrical charges. In one embodiment, a pixel includes two capacitors electrically separated by a transistor. The first capacitor is used alone for storing the electrical charge under low illumination while the second capacitor is off. When the ambient illumination becomes high, the transistor switches on the second transistor to increase the electrical charge storage capabilities of the pixel. By switching on and off the second capacitor, the imaging pixel 123 achieves the HCG for low illumination (when only the first capacitor is used) and the LCG for high illumination (when both the first and second capacitors are used). The PD pixel 124 may also have the capability to be configured to select and use one of the LCG and HCG. However, the traditional image pixel 123 lacks the capacity to determine a left or right image of the same object. Other methods / configurations for storing the electrical charges may be used as long as the imaging sensor 122 is a DCG sensor.

[0021] The camera body 120 further includes a processor 128 which is configured with a DCG controller 130. Processor 128 is located at the DCG sensor 122 (e.g., directly connected to the sensor or physically located on the sensor). Other processors may be present in the camera body 120, e.g., remote from the DCG sensor 122. The DCG controller 130 controls the DCG sensor 122, for example, switching on and off a transistor associated with the second capacitor of the PD pixel 124 (or imaging pixel 123) for selecting one of the LCG or HCG configurations. The DCG controller 130 may include plural modules (software, hardware, or a combination of them), for example, a DCG PD generator module 132 and a PD selector module 134. The processor 128 is further configured with an AF control system 136. The AF control system 136 (also called “PD based AF control system” or “PDAF control system”) may be associated with a phasedifference AF module, an AF core, and a sensor driver, which are discussed later in more detail. The PDAF control system 136 is responsible for autofocusing the image, i.e., controlling a position of the lens 112 using a calculated defocus amount. The AF core mayperform a focus detection using a phase-difference detecting method with the PD pixels 124, separate from the imaging pixels 123.

[0022] A DCG sensor is traditionally used in a camera for high dynamic range (HDR) procedures. The HDR involves taking multiple exposures of the same scene / object at different brightness levels (i.e. , using LCG and HCG pixel configurations discussed above) and then combining these images using software schemes to generate the final image. The final image is generated either at the DCG sensor 122 or at an application processor (AP) level for the traditional imaging pixel pipeline. A ratio of the conversion gains for the two different pixel configurations (LCG and HCG) is physically fixed by the imaging sensor manufacturer, for example, to 1 to 2n(where n = 1 , 2, 3, 4, ... ). The output bit depth of the combined sensor data (final image) is 11 to 16 bits, depending on the DCG ratio. The signal ratio between HCG and LCG is typically 8:1 . For both 10-bit pixel outputs, with an 8:1 ratio, it is possible to achieve a 13-bit extended dynamic range in iDCG mode (10 + 3 bits from the gain ratio). Note that the LCG and HCG configurations for the image pixels 123 are used for the HDR procedures discussed above.

[0023] The DCG sensor 122 is configured to also provide two (only two) distinct DCG PD gain configurations for the AF control system 136. As discussed above, the AF control system 136 is configured to autofocus the lens 112 of the camera 100 to ensure that the subject in the frame appears sharp and clear. The two DCG PD gain configurations include a low gain conversion per frame for PD purposes, called in this disclosure “LCG PD,” and a high gain conversion per frame for PD purposes, called in this disclosure “HCG PD.” Thus, the LCG and HCG configurations are different from the LCG PD and HCG PD configurations. The LCG and HCG configurations are used to HDR purposes (i.e., image formation) while the LCG PD and HCG PD configurations are used for AF purposes (finding the optimal focus position of the lens). Because of the limited number (only two) of DCG PD gain configurations for the AF capability, the AF control system is limited in finding the optimal focus position.

[0024] More specifically, to consistently obtain the best tonal PD data, the AF control system examines two different types of PD gain configurations and switch between them whenever one is deemed unreliable. In addition, the binary selection of DCG PDframes is not suitable for handling various brightness ranges in various scenes, such as from dark to bright through a mid-range zone. While the LCG PD is better suited for bright light conditions and the HCG PD is better suited for dark conditions, there is currently no suitable DCG PD gain configuration for the mid-range light conditions.

[0025] Further, using only the HCG-HDR information for PD purposes provides the best dark object AF capability, but loses most of the scene information because the majority area of the sensor is already saturated by the HCG conversion. On the other hand, if only the LCG-HDR information is used for AF purposes, it can cover all type of ambient illuminations, but the quality of dark objects is very poor compared to either HCG- HDR information or normal operation mode PD (i.e., a non-iDCG mode, where full PD is used), so it limits the AF capability in dark conditions and / or for dark objects in any ambient light conditions.

[0026] Although some existing sensors provide both LCG PD and HCG PD through combining sparse PD (AF method that uses fewer PD pixels), for example, these sensors provide HCG PD at sparse PD and LCG PD at full PD (another AF method that use dense PD pixels), the density of HCG PD is very small and has a severe quality drop at low-light conditions because of a lack of density for HCG PD.

[0027] To address the AF limitations discussed above for the traditional DCG sensors, according to the embodiment illustrated in FIG. 2, the DCG controller 130 includes a DCG PD generator module 132 and an optimal PD configuration selector module 134 that are configured to generate, select, and apply one of three or more DCG PD gain configurations. Processor 128, which is located at the DCG sensor 122 side, hosts both of these modules. In this embodiment, the DCG PD generator module 132 applies (i.e., generates at the DCG sensor) one of the three or more DCG PD gain configurations, based on optimal PD configuration selected by the PD configuration selector module 134. The PD configuration selector module 134 chooses the optimal PD configuration based on the ambient light conditions, as discussed later. For this purpose, the PD configuration selector module 134 receives from the PD pixels 124 of the sensor 122, signals indicative of the left and right images (called in this disclosure the "dual PD images” or simply “dual PD”) and determines the optimal PD configuration (one of three or more DCG gain configurations) to be used. The PD configuration selector sends then theselected optimal PD configuration to the DCG PD generator 132, to apply (i.e. , configure the DCG sensor) the optimal PD configuration to the DCG sensor 122. FIG. 2 also shows actuator 114 receiving an optical focus lens position from the disparity estimator (PDAF control system) 136. The PDAF control system 136 receives the dual PD from the sensor 122 and calculates, as previously discussed, the phase difference between the left and right images. Based on these calculations, the PDAF control system 136 generates the optimal focus lens position, which is transmitted to the actuator 114, as schematically illustrated in FIG. 2.

[0028] For a better understanding of the differences between the traditional and newly introduced DCG PD gain configurations, FIGs. 3A and 3B schematically illustrate the two traditional DCG PD gain configurations and FIGs. 4A to 4C schematically illustrate the three or more DCG PD gain configurations. More specifically, FIG. 3A illustrates a PD pixel 124 that includes four unit pixels 124A to 124D. Each unit pixel includes four subpixels 324 to 330 (only the sub-pixels of a single unit pixel are labelled in the figure for simplicity, but each unit pixel has the same structure). Each sub-pixel generates an independent signal, based on the amount of electrical charges caused by the incoming light. Thus, the PD pixel 124 in this embodiment is capable of generating 16 different and independent signals, one per sub-pixel. However, the PDAF control system may bin the signals so that a single signal per unit pixel 124A to 124D is generated or a single signal per the entire pixel 124 is generated. For AF purposes, the output of each sub-pixel is used. For image purposes, for the traditional image pixel 123, all the outputs from all the sub-pixels are binned together so that only one output per pixel is produced. Different from the traditional image pixel 123, the PD pixel 124 has the capability of producing an electrical signal corresponding to a left image of an object and a different electrical signal corresponding to a right image of the same object. In quad unit pixels, i.e., a unit pixel that has four sub-pixels as in FIG. 3A, it is possible to also generate different electrical signals for a top image and bottom image of the same object. The PDAF control system estimates the defocus based on the phase difference of the left (or top) and right (or bottom) images and calculates a focusing distance to be applied by the actuator 114 to the lens 112.

[0029] For the scenario illustrated in FIG. 3A, the electrical signals from two subpixels 324 corresponding to unit pixels 124C and 124D are summed up and read out asthe LCG PD for AF purposes. For the scenario illustrated in FIG. 3B, the electrical signals from eight sub-pixels 324, 328 corresponding to all four unit pixels 124A to 124D are summed up and read out as the HOG PD for AF purposes. Note that both figures illustrate the summation of the electrical signals for the left sub-pixels. In one embodiment, it is possible to use the same configuration for the right sub-pixels.

[0030] For the traditional DCG sensors, a ratio of PDs between LCG and HCG PD gain configurations (e.g., 1 :4 in FIGs. 3A and 3B because of the selection of 2 sub-pixels in FIG. 3A and 8 sub-pixels in FIG. 3B) is consistent with normal imaging pixel pipelines (i.e., the DCG-HDR configurations). However, as earlier discussed, this traditional configuration restricts PD pixel selection to only LCG PD output and HCG PD output, which is not satisfactory for the mid-range illumination conditions.

[0031] Thus, the embodiments illustrated in FIGs. 4A to 4C redefine the DCG PD gain configurations by adding at least one new DCG PD gain configuration and potentially also changing the number of sub-pixels that are used for the LCG PD configuration. More specifically, the LCG PD configuration shown in FIG. 4A is redefined to include the electrical signal output from a single sub-pixel 324 (left-bottom sub-pixel) instead of two sub-pixels from two different unit pixels as in FIG. 3A. FIG. 4C illustrates the HCG PD gain configuration, which is identical to the configuration illustrated in FIG. 3B. The PD gain ratio between these two configurations is now 1 :8 instead of 1 :4 as in the embodiment of FIGs. 3A and 3B.

[0032] FIG. 4B introduces a mid-conversion gain (MCG) PD gain configuration. The MCG PD gain configuration combines the electrical signal outputs from more than one and less than eight sub-pixels belonging to the four unit pixels 124A to 124D of the PD pixel 124. In this embodiment, the MCG PD gain configuration combines the output from four sub-pixels 324, one from each of the unit pixels 124A to 124D. In a different embodiment, the MCG PD gain configuration combines the output from a different number of pixels, as long as the 1 to 2nratio of the gains with the LCG PD gain configuration is maintained, where n is an integer equal to one or larger. For the configurations shown in FIGs. 4A to 4C, the ratio of the gains is 1 :4:8. Note that these configurations are implemented on the sensor side. These configurations help to reduce the gap between the legacy gain configurations that are limited by the binary selection of HCG PD and LCG PD output fromthe DCG sensor. In this respect, the MCG PD gain configuration improves sensitivity and signal quality relative to traditional PD configurations by doubling the number of pixels read out when compared to the legacy LCG PD gain configuration, which only reads out two pixels from a quad pixel group. Additionally, the MCG PD gain configuration reduces the risk of pixel saturation more than HCG-HDR.

[0033] To compare the different DCG PD gain configurations of FIGs. 3A to 4C and their respective characteristics, the table in FIG. 5 presents their signals for the raw image (image pixel in the table) and PD frames. The table shows the binning, LCG, and HCG for the raw image and the binning, LCG PD, MCG PD, and HCG PD configurations for the PD mode. The table also shows the ratio of the various configurations noted above relative to the binning configuration in each of the raw image part of the table and the PD part of the table.

[0034] The integration of the DCG PD generator module 132 and PD selector module 134 with the DCG controller 130 and the AF control system 136 is now discussed with regard to FIG. 6. This figure illustrates the AF core 650 interacting with the sensor driver 652, PDAF 136, and DCG controller 130. The AF core 650 is a core block responsible for analyzing various types of metering statistics, including high-pass filtered images and scene information. It also manages interfaces for different types of AF algorithms and schedules them for AF system work. The PDAF 136 is responsible for calculating the phase differences between the left and right (or top and bottom) frames generated by the PD pixels 124. The sensor driver 652 is responsible for applying one or more voltages to the DCG sensor 122, and also for reading the various DCG PD gain configurations from the sensor. The sensor driver 652 is also responsible for applying an updated PD configuration to the sensor. The updated PD configuration is selected, from three or more PD configurations, by the PD configuration selector 134, and this updated PD configuration is generated at the DCG sensor 122 by the DCG PD generator module 132.

[0035] As shown in FIG. 6, the DCG controller 130 generates and selects the next DCG PD gain configuration (also called in this disclosure “PD configuration”) based on one or more factors, which include a PD average, saturation ratio, and an updated PD configuration. These factors are discussed in more detail with regard to FIG. 7. The DCGcontroller 130 receives the PD average and the saturation ratio from the PDAF 136. The DCG controller 130 receives the updated PD configuration from the sensor driver 652. In one embodiment, all these factors are passing through the AF core 650, i.e. , the AF core is transparent to these factors. The DCG controller 130 sends the generated next PD configuration to the AF core 650, which in turn provides this information to the sensor driver 652. The sensor driver 652 uses the next PD configuration to generate an updated PD configuration and applies the updated PD configuration to the sensor 122. The sensor 122 provides PD raw data or statistics associated with the PD raw data to the PDAF 136. In one embodiment, the sensor manufacturer provides the statistics that may be stored at any of the modules illustrated in FIG. 6. The statistics describe particular features of the DCG sensor, for example, light sensitivity per pixel, number of PD pixels 124, number of image pixels 123, light thresholds of the photodetectors associated with the pixels, etc. FIG. 6 also shows the AF core 650 providing a saturation threshold (per pixel) to the PDAD 136.

[0036] FIG. 7 illustrates a state decision / transition between states associated with the three or more DCG PD gain configurations. Only three gain configurations are shown in the figure for simplicity. However, more than three gain configurations may be used, i.e., plural MCG PD configurations may be implemented. Each of the three DCG PD gain configurations are associated with a state counter “state_counter,” whose value is increased by one unit each time such a state is considered. A threshold value “trhd” is used to limit the switching between two DCG PD gain configurations. For example, if the ambient light conditions quickly change from low to high to low, it is undesirable to have the AF control system quickly change states based on the three or more DCG PD gain configurations. Thus, the “thrd” value is used to limit how quickly the system is allowed to change its state. For example, the AF core may use a counter (the state_counter) that has its value increased by one any time that a change happens between two of the three DCG PD gain configurations. If there is no change, although new PD average and saturation ratio values have been received, the counter is reset to zero. In one embodiment, the “thrd” has a value of 3. Other values may be used.

[0037] FIG. 7 also shows that factors used for changing one state (associated with one DCG PD gain configuration) to another state (associated with another DCG PD gainconfiguration) are: (1 ) a PD average within a region of interest, ROI, (labeled “underexposed” in the figure), which is associated with a brightness level within the ROI, and (2) a saturation ratio (labeled “saturated” in the figure). The two factors are determined from statistical data extracted from the PD raw data. The saturation ratio provides an indication of the number of pixels that exceed a predefined threshold within the ROI, which can be utilized to infer the saturation status of the object within the ROI. The saturation ratio helps the AF core to determine whether a current PD state is suitable for reliably performing PDAF. In addition, the brightness level is utilized as a significant metric for determining the next optimal PD configuration estimate.

[0038] FIG. 7 shows that if the ROI is underexposed, and the counter is larger than the threshold thrd, the DCG controller changes the LCG PD configuration to the HCG PD configuration. The same conditions may apply for changing the LCG PD configuration to the MCG PD configuration, but the values of the PD average and saturation for the different transitions may be different. In this embodiment, to avoid a drastic change, the system is configured to not allow an HCG PD gain configuration change directly to the LCG PD gain configuration. The system is configured to first change from the HCG PD gain configuration to the MCG PD gain configuration when the PD saturation is below a certain first saturation threshold and the counter is above a given threshold. If this new configuration is not suitable, the system is further configured to then change from the MCG PD gain configuration to the LCG PD gain configuration, when the PD saturation is below a certain second saturation threshold and the counter is above the given threshold.

[0039] In one embodiment, the most desirable state is the HCG PD gain configuration, which ensures the highest signal reliability of the PD procedure. Based on the brightness statistics in the AF ROI given over dual PD raw data, any configuration illustrated in FIG. 7 is conditionally transitioned to another configuration as discussed above. The transition conditions / thresholds may be set up in the same manner for all DCG PD gain configurations. Based on the example illustrated in FIG. 7, there are two directional transitions between each state: low to high, and high to low. For the condition to transition to the higher gain state from the low gain state, the parameter is_underexposed can be predicted based on the difference in sensitivities of the two gains of the configurations. On the other hand, the readiness of the transition to the lower gain ischecked by the saturation ratio of the pixels in the ROI. In this embodiment, the is_underexposed parameter is given by is_underexposed = (PD_average x gain_ratio) < threshold! , and the is_saturated parameter is given by is_saturated = PD_saturation_ratio < threshold2. The gain_ratio is defined as the ratio between (1 ) DCG PD gain of the next configuration / state and (2) the DCG PD gain of the next configuration / state. Other definitions may be used for these parameters.

[0040] A method 800 that uses the configurations of FIGs. 4A to 4C and 7 and the AF control system of FIG. 6 is now discussed with regard to FIG. 8. The method 800, which is an imaging method with autofocus based on PD, has a DCG imaging sensor including a plurality of PD pixels, and the PD pixels receive 802 light and transform the light into electrical charges and corresponding signals. A PD selector module of a DCG controller selects 804 one of three or more DCG PD gain configurations to autofocus, AF, a lens associated with the DCG imaging sensor. The three or more DCG PD gain configurations are based on a single gain configuration for the plurality of PD pixels. A DCG PD generator module applies 806 the selected one of the three or more DCG PD gain configurations to the DCG imaging sensor. An AP module optionally calculates 808 a focus distance based on the selected one or the three or more DCG PD gain configurations and instructs an actuator to move the lens with the focus distance relative to the DCG imaging sensor for autofocusing the lens.

[0041] In one embodiment, the single gain configuration of the plurality of PD pixels is a high conversion gain, HCG, configuration, and the plurality of PD pixels further includes a low conversion gain, LCG, configuration. The three or more DCG PD gain configurations include a low-conversion gain, LCG, PD configuration, a mid-conversion gain, MCG, PD configuration, and a high-conversion gain, HCG, PD configuration. The single gain configuration of the plurality of PD pixels is defined by electronics associated with the plurality of PD pixels while the three or more DCG PD gain configurations are defined by combining outputs of PD sub-pixels of the plurality of PD pixels.

[0042] In another embodiment, a PD pixel of the plurality of PD pixels includes four unit pixels 124A, 124B, 124C, 124D, and each unit pixel includes four sub-pixels 324, 326. The LCG PD configuration uses a single sub-pixel 324 of one unit pixel of the PD pixel, the MCG PD configuration uses a combination of more than one and less than eight sub-pixels belonging to the four unit pixels of the PD pixel, and the HCG PD configuration uses two sub-pixels from each unit pixel of the PD pixel. In one variation of this embodiment, the MCG PD configuration uses four sub-pixels, one from each unit pixel of the PD pixel.

[0043] In one embodiment, the step of selecting 804 includes selecting the one of the three or more DCG PD gain configurations based on a PD average and a saturation ratio. The PD average is an average brightness of left or right images in a region of interest, ROI, and the saturation ratio is indicative of how many pixels in the ROI are saturated.

[0044] The method 800 of FIG. 8 may be implemented in imaging device 100 as now discussed. Imaging device includes a DCG sensor that has a plurality of PD pixels configured to transform an incoming light beam into electrical charges. The imaging device further includes a lens configured to focus the incoming light beam onto the plurality of PD pixels, an actuator configured to move the lens along an axis to autofocus the lens, and a processor hosting a DCG controller. The DCG controller is operable to, select one of three or more DCG PD gain configurations for the electrical charges, the three or more DCG PG gain configurations being based on applying a single gain configuration to the plurality of PD pixels, the one of the three or more DCG PD gain configurations selected according to an ambient illumination, and apply the selected one of three or more DCG PD gain configurations to the DCG sensor.

[0045] Reference throughout this section to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout the specification are not necessarily all referring to the same embodiment. Further, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.

[0046] Numerical adjectives “first”, “second”, and “third” do not imply any order (are not ordinals) but are markers to distinguish separate instances of similar elements. References to the singular (e.g., “a” or “an”, “the”) should include the plural unless clearly indicated otherwise.

[0047] As used herein, a phrase referring to “at least one of’ or “one or more of’ a list of items refers to any combination of those items, including single members. For example, “at least one of: a, b, or c” is intended to cover the possibilities of: a only, b only, c only, a combination of a and b, a combination of a and c, a combination of b and c, and a combination of a and b and c.

[0048] Although the features and elements of the present embodiments are described in the embodiments in particular combinations, each feature or element can be used alone without the other features and elements of the embodiments or in various combinations with or without other features and elements disclosed herein. The methods or flowcharts may be implemented in a computer program, software or firmware tangibly embodied in a computer-readable storage medium for execution by a specifically programmed computer or processor.

Claims

WHAT IS CLAIMED IS:1 . An imaging method with autofocus based on phase difference, PD, the method comprising: receiving (802) light at a dual conversion gain, DCG, imaging sensor (122) including a plurality of PD pixels (124); selecting (804), at a PD selector module (134) of a DCG controller (130), one of three or more DCG PD gain configurations to autofocus, AF, a lens (112) associated with the DCG imaging sensor (122), the three or more DCG PD gain configurations being based on a single gain configuration for the plurality of PD pixels; and applying (806), with a DCG PD generator module (132), the selected one of the three or more DCG PD gain configurations to the DCG imaging sensor.

2. The method of Claim 1 , wherein the single gain configuration of the plurality of PD pixels is a high conversion gain, HCG, configuration, and the plurality of PD pixels further includes a low conversion gain, LCG, configuration.

3. The method of Claim 1 or 2, wherein the three or more DCG PD gain configurations include a low-conversion gain, LCG, PD configuration, a mid-conversion gain, MCG, PD configuration, and a high-conversion gain, HCG, PD configuration.

4. The method of any of Claims 1 to 3, wherein the single gain configuration of the plurality of PD pixels is defined by electronics associated with the plurality of PD pixels while the three or more DCG PD gain configurations are defined by combining outputs of PD sub-pixels of the plurality of PD pixels.

5. The method of Claim 3, wherein a PD pixel (124) of the plurality of PD pixels includes four unit pixels (124A, 124B, 124C, 124D), and each unit pixel includes four sub-pixels (324, 326), and the LCG PD configuration uses a single sub-pixel (324) of one unit pixel of the PD pixel,the MCG PD configuration uses a combination of more than one and less than eight sub-pixels belonging to the four unit pixels of the PD pixel, and the HCG PD configuration uses two sub-pixels from each unit pixel of the PD pixel.

6. The method of Claim 5, wherein the MCG PD configuration uses four subpixels, one from each unit pixel of the PD pixel.

7. The method of any of Claims 1 to 6, wherein the selecting comprises: selecting the one of the three or more DCG PD gain configurations based on aPD average and a saturation ratio.

8. The method of Claim 7, wherein the PD average is an average brightness of left or right images in a region of interest, ROI, and the saturation ratio is indicative of how many pixels in the ROI are saturated.

9. An imaging device (100) comprising: a dual conversion gain, DCG, sensor (122) including a plurality of phase difference, PD, pixels (124) configured to transform an incoming light beam into electrical charges; a lens (112) configured to focus the incoming light beam onto the plurality of PD pixels (124); an actuator (114) configured to move the lens (112) along an axis to autofocus, AF, the lens (112); and a processor (128) hosting a DCG controller (130), the DCG controller (130) operable to, select one of three or more DCG PD gain configurations for the electrical charges, the three or more DCG PG gain configurations being based on applying a single gain configuration to the plurality of PD pixels (124), the one of the three or more DCG PD gain configurations selected according to an ambient illumination; andapply the selected one of three or more DCG PD gain configurations to the DCG sensor (122).

10. The imaging device of Claim 9, further comprising: calculating a focus distance based on the selected one of three or mre DCG PD gain configurations to autofocus the lens.11 . The imaging device of any of Claim 9 or 10, wherein the single gain configuration of the plurality of PD pixels is a high-conversion gain, HCG, configuration, and each of the plurality of PD pixels further includes a low-conversion gain, LCG, configuration.

12. The imaging device of any of Claims 9 to 11 , wherein the three or more DCG PD gain configurations include a low-conversion gain, LCG, PD configuration, a midconversion gain, MCG, PD configuration, and a high-conversion gain, HCG, PD configuration.

13. The imaging device of any of Claims 9 to 12, wherein the LCG and HCG configurations of the plurality of PD pixels are defined by electronics associated with the plurality of PD pixels while the three or more DCG PD gain configurations are defined by combining outputs of PD sub-pixels of the plurality of PD pixels.

14. The imaging device of Claim 12, wherein a PD pixel of the plurality of PD pixels includes four unit pixels, and each unit pixel includes four sub-pixels, and the LCG PD configuration uses a sub-pixel of one unit pixel of the PD pixel, the MCG PD configuration uses a combination of more than one and less than eight sub-pixels belonging to the four unit pixels of the PD pixel, and the HCG PD configuration uses two sub-pixels from each unit pixel of the PD pixel.

15. The imaging device of Claim 14, wherein the MCG PD configuration uses four sub-pixels, one from each unit pixel of the PD pixel.

16. The imaging device of any of Claims 9 to 15, wherein a PD configuration selector module of the DCG controller selects the one of three or more DCG PD gain configurations based on a PD average and a saturation ratio, and a DCG PD generator applies the selected one of three tor more DCG PG gain configurations to the DCG sensor.

17. The imaging device of Claim 16, wherein the PD average is an average brightness of left or right images in a region of interest, ROI, and the saturation ratio is indicative of how many pixels in the ROI are saturated.

18. A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a processor, implement a method with autofocus based on a phase difference, PD, the medium comprising instructions for: selecting (804), at a PD selector module (134) of a DCG controller (130) of a dual conversion gain, DCG, imaging sensor (122), one of three or more DCG PD gain configurations to autofocus, AF, a lens (112) associated with the DCG imaging sensor (122), the three or more DCG PD gain configurations being based on a single gain configuration of a plurality of PD pixels (124) of the DCG imaging sensor (122); and applying (806), with a DCG PD generator module (132), the selected one of the three or more DCG PD gain configurations to the DCG imaging sensor.

19. The medium of Claim 18, wherein the single gain configuration of the plurality of PD pixels is a high conversion gain, HCG, configuration, and the plural PD pixels further include a low conversion gain, LCG, configuration.

20. The medium of Claim 18 or 19, wherein the three or more DCG PD gain configurations include a low-conversion gain, LCG, PD configuration, a mid-conversion gain, MCG, PD configuration, and a high-conversion gain, HCG, PD configuration.

Images