Post-shutter autofocus for high-resolution image capture

Abstract

A method includes sensing a low-resolution image stream captured by an image sensor operating in a low-resolution mode. The method includes detecting a shutter activation signal while sensing the low-resolution image stream. The method includes performing a post-shutter autofocus operation in response to detecting the shutter activation signal. Performing the post-shutter autofocus operation includes (i) analyzing one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position, (ii) mapping the low-resolution lens position to a high-resolution lens position, and (iii) determining, based on the confidence value, an offset to shift the high-resolution lens position. The method includes switching the image sensor from the low-resolution mode to a high-resolution mode and adjusting the high-resolution lens position by the offset to capture a high-resolution image.

Description

Post- Shutter Autofocus for High-Resolution Image CaptureBACKGROUND

[0001] Mobile devices may include one or more image sensors (e.g., cameras) to capture images of a surrounding environment. Typically, an image sensor can capture (i) low- resolution images of the surrounding environment or (ii) high-resolution images of the surrounding environment. As a non-limiting example, when operating in a low-resolution mode, the image sensor may capture a low-resolution image of the surrounding environment. However, when operating in a high-resolution mode, the image sensor may capture a high- resolution image of the surrounding environment.

[0002] Due to the increased image size, an image sensor typically utilizes a relatively large amount of memory and power when capturing a high-resolution image when compared to capturing a low-resolution image. To alleviate the resource demand (e.g., the memory footprint and power consumption) associated with capturing a high-resolution image, autofocusing operations may be performed while the image sensor streams a low-resolution output and, upon activation of a shutter button, the image sensor can dynamically switch to the high- resolution mode to capture a high-resolution image using auto-focusing parameters (e.g., lens position parameters) determined during the low-resolution stream. However, using low- resolution auto-focusing parameters in the high-resolution mode may result in a less than optimal output (e.g., blurry images).SUMMARY

[0003] A device may include an image sensor (e.g., a camera) that is configured to (i) capture low-resolution images while operating in a low-resolution mode and (ii) capture high- resolution images while operating in a high-resolution mode. To alleviate power consumption and a memory footprint associated with continuously operating in the high-resolution mode, to capture a high-resolution image, the image sensor may (i) operate in the low-resolution mode (e.g., stream a low-resolution sensor output) and, upon detection of a shutter button, (ii) dynamically switch to operate in the high-resolution mode to capture the high-resolution image.

[0004] In bright conditions (e.g., when the environmental lighting is relatively bright), static autofocus mapping between the low-resolution mode and the high-resolution mode may be performed. For example, a low-resolution lens position for a focus target in the low- resolution sensor output can be statically mapped to a corresponding high-resolution lensposition in the high-resolution mode for the high-resolution image capture. However, in low light conditions (e.g., when the environment lighting is relatively dark), a post-shutter autofocus operation (e.g., a post-shutter autofocus algorithm) may be implemented to further adjust the high-resolution lens position, ensuring a higher quality image capture.

[0005] To perform the post-shutter autofocus operation, one or more post-shutter autofocus frames (e.g., low-resolution frames) are captured after detection of the shutter button but prior to switching to the high-resolution mode. The post-shutter autofocus frames are analyzed to determine (i) the low-resolution lens position for the focus target and (ii) a confidence value associated with the low-resolution lens position. The image sensor may map the low-resolution lens position to the corresponding high-resolution lens position. Based on the confidence value, the image sensor may determine (i) whether to shift the high-resolution lens position to improve the high-resolution image capture and (ii) an offset for shifting the high-resolution lens position to improve the high-resolution image capture. For example, a smaller confidence value may indicate there is a large margin of error, and as a result, the high- resolution lens position may be shifted by a relatively large offset to ensure that the focus target in the resulting high-resolution image is sharp. A larger confidence value may indicate that there is a small margin of error, and as a result, the high-resolution lens position may be shifted by a relatively small offset (or shifting may be bypassed) to ensure that the focus target in the resulting high-resolution image is sharp. After determining whether, and how much, to shift the high-resolution lens position, the image sensor may switch to the high-resolution mode, implement the determined shift, and capture the high-resolution image.

[0006] In a first example embodiment, a method of capturing a high-resolution image includes sensing, by an image controller, a low-resolution image stream captured by an image sensor operating in a low-resolution mode. The method also includes detecting, by the image controller, a shutter activation signal while sensing the low-resolution image stream. The method also includes performing a post-shutter autofocus operation at least in response to detecting the shutter activation signal. Performing the post-shutter autofocus operation includes analyzing one or more frames of the low-resolution image stream to determine a low- resolution lens position for a focus target and a confidence value associated with the low- resolution lens position. Performing the post-shutter autofocus operation also includes mapping the low-resolution lens position to a high-resolution lens position. Performing the post-shutter autofocus operation also includes determining, based at least on the confidence value, an offset to shift the high-resolution lens position. The method also includes switching the image sensor from the low-resolution mode to a high-resolution mode. The method alsoincludes adjusting the high-resolution lens position by the offset to capture the high-resolution image while the image sensor is operating in the high-resolution mode.

[0007] In a second example embodiment, a device includes a memory, an image sensor, and an image controller coupled to the image sensor and to the memory. The image controller is configured to sense a low-resolution image stream captured by the image sensor operating in a low-resolution mode. The image controller is also configured to detect a shutter activation signal while sensing the low-resolution image stream. The image controller is also configured to perform a post-shutter autofocus operation at least in response to detecting the shutter activation signal. To perform the post-shutter autofocus operation, the image controller is configured to analyze one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low- resolution lens position. To perform the post-shutter autofocus operation, the image controller is also configured to map the low-resolution lens position to a high-resolution lens position. To perform the post-shutter autofocus operation, the image controller is also configured to determine, based at least on the confidence value, an offset to shift the high-resolution lens position. The image controller is also configured to switch the image sensor from the low- resolution mode to a high-resolution mode. The image controller is also configured to adjust the high-resolution lens position by the offset to capture a high-resolution image while the image sensor is operating in the high-resolution mode.

[0008] In a third example, a non-transitory computer-readable medium includes instructions that, when executed by an image controller of a device, cause the image controller to perform operations. The operations include sensing a low-resolution image stream captured by an image sensor operating in a low-resolution mode. The operations also include detecting a shutter activation signal while sensing the low-resolution image stream. The operations also include performing a post-shutter autofocus operation at least in response to detecting the shutter activation signal. Performing the post-shutter autofocus operation includes analyzing one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position. Performing the post-shutter autofocus operation also includes mapping the low- resolution lens position to a high-resolution lens position. Performing the post-shutter autofocus operation also includes determining, based at least on the confidence value, an offset to shift the high-resolution lens position. The operations also include switching the image sensor from the low-resolution mode to a high-resolution mode. The operations also includeadjusting the high-resolution lens position by the offset to capture a high-resolution image while the image sensor is operating in the high-resolution mode.

[0009] In a fourth example embodiment, a computer program product includes a computer hardware storage device having stored therein computer-executable program code for adjusting a facial region of interest. The computer-executable program code, when executed by a computer, causes the computer to sense a low-resolution image stream captured by an image sensor operating in a low-resolution mode. The computer-executable program code, when executed by the computer, causes the computer to detect a shutter activation signal while sensing the low-resolution image stream. The computer-executable program code, when executed by the computer, causes the computer to perform a post-shutter autofocus operation at least in response to detecting the shutter activation signal. Performing the post-shutter autofocus operation includes analyzing one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position. Performing the post-shutter autofocus operation also includes mapping the low-resolution lens position to a high-resolution lens position. Performing the post-shutter autofocus operation also includes determining, based at least on the confidence value, an offset to shift the high-resolution lens position. The computerexecutable program code, when executed by the computer, causes the computer to switch the image sensor from the low-resolution mode to a high-resolution mode. The computerexecutable program code, when executed by the computer, causes the computer to adjust the high-resolution lens position by the offset to capture a high-resolution image while the image sensor is operating in the high-resolution mode.

[0010] In a fifth example embodiment, a system may include various means for carrying out each of the operations of the first example embodiment.

[0011] These, as well as other embodiments, aspects, advantages, and alternatives, will become apparent to those of ordinary skill in the art by reading the following detailed description, with reference where appropriate to the accompanying drawings. Further, this summary and other descriptions and figures provided herein are intended to illustrate embodiments by way of example only and, as such, that numerous variations are possible. For instance, structural elements and process steps can be rearranged, combined, distributed, eliminated, or otherwise changed, while remaining within the scope of the embodiments as claimed.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 illustrates a device, in accordance with examples described herein.

[0013] Figure 2A illustrates an example graphical representation of shifting a high- resolution lens position out of blurred image range in low-light conditions using the postshutter autofocus operation when a low-resolution confidence focus is relatively high, in accordance with examples described herein.

[0014] Figure 2B illustrates an example graphical representation of shifting a high- resolution lens position out of blurred image range in low-light conditions using the postshutter autofocus operation when a low-resolution confidence focus is relatively low, in accordance with examples described herein.

[0015] Figure 2C illustrates an example graphical representation of potentially shifting a high-resolution lens position out of blurred image range in low-light conditions using the post-shutter autofocus operation when a low-resolution confidence focus is low, in accordance with examples described herein.

[0016] Figure 3 illustrates an example of a process of selectively performing a postshutter autofocus operation, in accordance with examples described herein.

[0017] Figure 4 illustrates a process for performing a post-shutter autofocus operation, in accordance with examples described herein.

[0018] Figure 5 is a diagram illustrating training and inference phases of a machine learning model, in accordance with examples described herein.

[0019] Figure 6 illustrates a flow chart, in accordance with examples described herein.DETAILED DESCRIPTION

[0020] Example methods, devices, and systems are described herein. It should be understood that the words “example” and “exemplary” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or feature described herein as being an “example,” “exemplary,” and / or “illustrative” is not necessarily to be construed as preferred or advantageous over other embodiments or features unless stated as such. Thus, other embodiments can be utilized and other changes can be made without departing from the scope of the subject matter presented herein.

[0021] Accordingly, the example embodiments described herein are not meant to be limiting. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations.

[0022] Further, unless context suggests otherwise, the features illustrated in each of the figures may be used in combination with one another. Thus, the figures should be generally viewed as component aspects of one or more overall embodiments, with the understanding that not all illustrated features are necessary for each embodiment.

[0023] Particular embodiments are described herein with reference to the drawings. In the description, common features are designated by common reference numbers throughout the drawings. In some figures, multiple instances of a particular type of feature are used. Although these features are physically and / or logically distinct, the same reference number is used for each, and the different instances are distinguished by addition of a letter to the reference number. When the features as a group or a type are referred to herein (e.g., when no particular one of the features is being referenced), the reference number is used without a distinguishing letter. However, when one particular feature of multiple features of the same type is referred to herein, the reference number is used with the distinguishing letter. For example, referring to Figure 1, post-shutter autofocus frames are illustrated and associated with reference numbers 162 A and 162B. When referring to a particular post-shutter autofocus frame, such as the postshutter autofocus frame 162A, the distinguishing letter “A” is used. However, when referring to any arbitrary post-shutter autofocus frame or to the post-shutter autofocus frames as a group, the reference number 162 is used without a distinguishing letter.

[0024] Additionally, any enumeration of elements, blocks, or steps in this specification or the claims is for purposes of clarity. Thus, such enumeration should not be interpreted to require or imply that these elements, blocks, or steps adhere to a particular arrangement or are carried out in a particular order. Unless otherwise noted, figures are not drawn to scale.I. Overview

[0025] The techniques described herein improve autofocus for a high-resolution image capture. In particular, the techniques described herein improve the autofocus for a high- resolution image capture in low light conditions while reducing power consumption at a device and relaxing a memory footprint at the device.

[0026] Typically, compared to capturing low-resolution images (e.g., twelve (12) megapixel images), capturing high-resolution images (e.g., fifty (50) megapixel images) consumes a relatively large amount of power and memory due to the increased image size. In some scenarios, to reduce the power consumption and the memory footprint associated with capturing high-resolution images, a device may stream a low-resolution sensor output (e.g., a low-resolution image stream) and dynamically switch to a high-resolution sensor output (e.g., a high-resolution image stream) after detecting a shutter activation signal (e.g., activation ofthe shutter). For example, after detecting a shutter has been activated, the device may dynamically switch from a low-resolution mode to a high-resolution mode and capture a high- resolution image in the high-resolution mode.

[0027] However, in these scenarios, conventional autofocus methodologies may not be feasible for the high-resolution image stream due to the dynamic sensor mode switch. In particular, if autofocus mapping from the low-resolution image stream to the high-resolution image stream is used in low light conditions, image sharpness during the high-resolution image capture may be compromised, resulting in a blurry image. The image sharpness may be compromised because, in low light conditions, there may be a margin of error associated with the autofocus mapping correlation between the low-resolution image stream and the high- resolution image stream.

[0028] To illustrate, one non-limiting example of an autofocus metric may be a lens position of an image sensor lens. In some scenarios, for the low-resolution image stream, there may be a parabolic relationship between the lens position and the image sharpness of the low- resolution sensor output when focusing a particular target (e.g., a focus target). For example, as the lens position moves along an axis in a particular direction, an image sharpness of the focus target in the low-resolution image stream may increase until an ideal focus (e.g., peak sharpness) is achieved. As used herein, the ideal focus for the low-resolution image stream may be achieved when the lens position is in a “low-resolution focus position.” If the lens position continues to move along the axis in the particular direction after the lens position is in the low-resolution focus position, the image sharpness of the focus target in the low-resolution image stream may decrease.

[0029] However, when the device dynamically switches to the high-resolution mode to produce a high-resolution sensor output in low light conditions, there may be a drop in focus sharpness when mapping the low-resolution focus position to a high-resolution focus position. For example, during the high-resolution mode, as the lens position moves along the axis in the particular direction, the image sharpness of the focus target in the high-resolution image stream may increase. However, as the lens position reaches a “high-resolution focus position” (e.g., a lens position mapped from the low-resolution focus position), the image sharpness of the focus target in the high-resolution image stream may drastically decrease (e.g., dip) such that, when the lens position is at the high-resolution focus position, the image sharpness of the focus target in the high-resolution image stream is relatively low (e.g., at a sharpness notch). In some scenarios, when the lens position is at the high-resolution focus position, the image sharpness of the focus target in the high-resolution image stream may be lower than the image sharpnessof the focus target in the low-resolution image stream when the lens position is at the low- resolution focus position. During the high-resolution mode, if the lens position continues to move along the axis in the particular direction after the lens position is in the high-resolution focus position, the image sharpness of the focus target in the high-resolution image stream may drastically increase (e.g., rise from the dip) and then gradually decrease. Thus, in the high- resolution mode, there may be a lens position range around the high-resolution focus position that results in a drastic decrease in image sharpness of the focus target. As used herein, this lens position range in the high-resolution mode that results in the drastic decrease in image sharpness of the focus target may be referred to as the “blurred image range.”

[0030] The techniques described herein may be used to ensure that the lens position does not fall within the blurred image range in the high-resolution mode. To illustrate, after detection of the shutter, which triggers the device dynamically switching to the high-resolution mode, the device may determine whether to perform a post-shutter autofocus (PSAF) operation (e.g., algorithm). The PSAF operation may be used to shift (e.g., offset) the lens position (from the high-resolution focus position) out of the blurred image range to ensure a relatively high image focus target sharpness value for a high-resolution image captured in the high-resolution mode.

[0031] In some embodiments, the device may determine whether to perform the PSAF operation based on lighting conditions. For example, if the lighting conditions are relatively bright, the device may bypass performance of the PSAF operation and perform static mapping between the low-resolution mode and the high-resolution mode. To illustrate, if the lighting conditions are relatively bright, after detection of the shutter, the device perform static mapping to determine an equivalent lens position (for clearly capturing the focus target) in the high- resolution mode to the low-resolution focus position, ensuring that a captured high-resolution image has a relatively high focus target image sharpness.

[0032] However, if the lighting conditions are relatively dark, the device may perform the PSAF operation. The PSAF operation may be performed to ensure that the lens position does not fall within the blurred image range in the high-resolution mode. By ensuring that the lens position does not fall within the blurred image range, the PSAF operation may result in improved image sharpness for the focus target. To perform the PSAF operation, when the shutter is activated, a small group of PSAF frames may be analyzed to identify the low- resolution focus position (e.g., the low-resolution lens position for the focus target) and a confidence value associated with the low-resolution focus position. The PSAF frames may correspond to low-resolution frames (e.g., twelve (12) megapixel frames) captured in the low-resolution mode just prior to dynamically switching to the high-resolution mode. The confidence value may indicate the likelihood (e.g., the confidence) that the low-resolution focus position of the lens in the PSAF frames can adequately capture the focus target.

[0033] In some embodiments, to identify the focus target and the confidence value via the PSAF operation, the PSAF frames (or data associated with the PSAF frames) can be provided to a machine-learning network. The machine-learning network may process the data to identify (e.g., predict) the focus target and the confidence value.

[0034] The confidence value may be correlated to a margin of error that is used to determine a particular offset to shift the high-resolution focus position of the lens to ensure, during the high-resolution mode, the lens is out of the blurred image range. In particular, a smaller confidence value may indicate there is a large margin of error, and as a result, the device may need to shift the high-resolution focus position by a relatively large offset amount to ensure that the lens position is outside of the blurred image range. Shifting the lens position by a large offset amount may reduce the image sharpness of a high-resolution image. Thus, in some scenarios, shifting the lens position may be bypassed if the offset amount is too large, as it may reduce the image sharpness of the high-resolution image.

[0035] However, by performing the PSAF operation, the confidence value may be relatively high. As a result, the high-resolution focus position of the lens may be shifted (e.g., offset) by a relatively small amount to ensure that the lens position is outside of the blurred image range and while also ensuring that lens position is relatively close to the low-resolution focus position, such that the image sharpness of the focus target in a high-resolution image is relatively high. For example, a larger confidence value may indicate there is a small margin of error, and as a result, the device may need to shift the lens position by a smaller offset amount (from the low-resolution focus position) to ensure that the lens position is outside of the blurred image range. Shifting the lens position by a small offset amount may preserve the image sharpness of the focus target when dynamically switching from the low-resolution mode to the high-resolution mode in low-light conditions.

[0036] In some scenarios, the offset may also be determined based on other parameters in addition to the confidence value, such as a depth of field in the PSAF frames, a sensor pixel size, and / or an optical spectrum of the PSAF frames. The device may adjust the lens position based on the offset amount and switch to the high-resolution mode to capture a high-resolution image.II. Example Device

[0037] Figure 1 illustrates a diagram of a device 100, in accordance with examples described herein. In some implementations, the device 100 may be a mobile device, such as a mobile phone, a tablet, a laptop computer, etc. As described below, the device 100 may be configured to perform a post-shutter autofocus operation during low light conditions to capture a high-resolution image 190 that has a sharp focus on a focus target 170.

[0038] The device 100 includes an image controller 102, a memory 104 coupled to the image controller 102, and an image sensor 106 coupled to the image controller 102. The memory 104 can be a non-transitory computer-readable medium that stores instructions 103 that are executable by the image controller 102 to perform the operations described herein. Specifically, the instructions 103 can be executable to cause the image controller 102 to perform a post-shutter autofocus operation during low light conditions to capture the high- resolution image 190 that has a sharp focus on the focus target 170.

[0039] The image sensor 106 may be configured to generate image data 107 and provide the image data 107 to the image controller 102. In some scenarios, the image data 107 can be generated without capturing an image. For example, light indicative of a scene may enter through a lens 109 of the image sensor 106, and a signal output by the image sensor 106 may be processed to generate the image data 107. As described below, depending on the mode 136, 138 of the image sensor 106, the image data 107 may correspond to a low-resolution image stream 132 or a high-resolution image stream 134.

[0040] In some implementations, the image controller 102 may be integrated into a processor, such as a central processing unit (CPU). In some implementations, the image controller 102 may be integrated into the image sensor 106 and coupled to a processor. Thus, as described herein, the operations performed by image controller 102 may be performed by a standalone processor or by an image controller integrated into the image sensor 106. It should be understood that additional components (e.g., circuitry, hardware, etc.) can be coupled to the image controller 102 (or to the processor). As non-limiting examples, a display screen can be coupled to the image controller 102, a transceiver can be coupled to the image controller 102, an auxiliary device interface can be coupled to the image controller 102, one or more additional sensors can be coupled to the image controller 102, etc. The components depicted in Figure 1 are merely for illustrative purposes and should not be construed as limiting.

[0041] As described herein, the image controller 102 may improve autofocus for the focus target 170 during a high-resolution image capture. In particular, the image controller 102 may improve the autofocus for the focus target 170 during a high-resolution image capture inlow light conditions while reducing power consumption at the device 100 and relaxing a memory footprint at the device 100. For example, as described below, the image controller 102 may stream a low-resolution image stream 132, instead of a high-resolution image stream 134, to reduce power consumption and relax the memory footprint at the device 100. When a shutter is triggered in low light conditions, the image controller 102 may (i) perform a postshutter autofocus operation to map a low-resolution autofocus lens position for the focus target 170 to a high-resolution autofocus lens position, (ii) adjust (e.g., shift) the high-resolution autofocus lens position to avoid a scenario where the mapped high-resolution autofocus lens position would result in a blurred image (e.g., a blurred focus target), (iii) switch to a high- resolution image stream 134, and (iv) capture the high-resolution image using the adjusted high-resolution autofocus lens position.

[0042] To illustrate, the image controller 102 includes a shutter detection unit 110, an image stream buffer 112, a lens controller 114, a sensor mode controller 116, a light detection unit 118, a high-resolution image generator 120, and a post-shutter autofocus unit 122. The image controller 102 may also include other components not depicted in Figure 1. As nonlimiting examples, the image controller 102 may include a low-resolution image generator, an image exposure unit, an aperture unit, etc. According to some implementations, one or more components of the image controller 102 can be implemented using dedicated circuitry. As non-limiting examples, one or more components of the image controller 102 can be implemented using application-specific integrated circuits (ASICs) or field-programmable gate array (FPGA) devices. According to some implementations, one or more components of the image controller 102 can be implemented using software. As a non-limiting example, the image controller 102 can execute the instructions 103 stored in the memory 104 to perform the operations of one or more components of the image controller 102.

[0043] The sensor mode controller 116 may be configured to control a resolution (e.g., a mode) of the image sensor 106. To illustrate, the sensor mode controller 116 may set the image sensor 106 to operate in a low-resolution mode 136 or a high-resolution mode 138. When the image sensor 106 is operating in the low-resolution mode 136, the image sensor 106 may capture the low-resolution image stream 132 that is (temporally) stored in the image stream buffer 112. In some implementations, the low-resolution image stream 132 may correspond to a twelve (12) megapixel image stream. When the image sensor 106 is operating in the high-resolution mode 138, the image sensor 106 may capture the high-resolution image stream 134 that is (temporally) stored in the image stream buffer 112. In some implementations, the high-resolution image stream 134 may correspond to a fifty (50)megapixel image stream. To conserve power and relax the memory footprint at the device 100, to capture high -resolution images, the sensor mode controller 116 may operate the image sensor 106 in the low-resolution mode 136 and switch to the high-resolution mode 138 at the time of capture.

[0044] The shutter detection unit 110 may be configured to detect a shutter activation signal 130. For example, when a user of the device 100 presses the shutter button, the device 100 can generate the shutter activation signal 130 that is detected by the shutter detection unit 110. To reduce power consumption associated with continuously operating the image sensor 106 in the high-resolution mode 138 to generate the high-resolution image stream 134, the image controller 102 may sense the low-resolution image stream 132 until the shutter detection unit 110 detects that a shutter is activated. Thus, the shutter detection unit 110 may detect the shutter activation signal 130 while the image controller 102 is sensing the low-resolution image stream 132.

[0045] Prior to, or in response to, detection of the shutter activation signal 130, the light detection unit 118 may be configured to determine whether a lighting condition proximate to the image sensor 106 corresponds to a bright lighting condition 140 or a dark lighting condition 142 (e.g., a low light condition). As a non-limiting example, the light detection unit 118 may determine a brightness value associated with a surrounding environment and compare the brightness value to a brightness threshold. If the brightness value satisfies (e.g., is greater than or equal to) the brightness threshold, the light detection unit 118 may determine that the lighting condition corresponds to the bright lighting condition 140. However, if the brightness value fails to satisfy (e.g., is less than) the brightness threshold, the light detection unit 118 may determine that the lighting condition corresponds to the dark lighting condition 142.

[0046] After detection of the shutter activation signal 130, prior to switching to the high-resolution mode 138 and capturing the high-resolution image 190, the image controller 102 may determine whether to perform a post-shutter autofocus operation. The determination of whether to perform the post-shutter autofocus operation may be based on the lighting conditions.

[0047] For example, if a bright lighting condition 140 exists, the image controller 102 may bypass performance of the post-shutter autofocus operation and may perform static mapping between the low-resolution image stream 132 and the high-resolution image stream 134. Thus, in response to detecting the shutter activation signal 130 and determining that the lighting condition corresponds to the bright lighting condition 140, the image controller 102 may be configured to map the focus target 170 in the low-resolution image stream 132 to thefocus target 170 in the high-resolution image stream 134 and capture the resulting high- resolution image 190.

[0048] However, if the dark lighting condition 142 exists, the image controller 102 may perform a post-shutter autofocus operation to map the focus target 170 in the low-resolution image stream 132 to the focus target 170 in high-resolution image stream 134. Performing the post-shutter autofocus operation may preserve the image sharpness of the focus target 170 in the high-resolution image 190 in dark lighting conditions 142 by ensuring that, in the high- resolution mode 138, the position of the lens 109 does not fall within a blurred image range that results in a drastic decrease in image sharpness of the focus target 170, as described in greater detail with respect to Figure 2 A.

[0049] The post-shutter autofocus unit 122 may be configured to perform the postshutter autofocus operation (e.g., the post-shutter autofocus algorithm). The post-shutter autofocus unit 122 may include a post-shutter autofocus frame generator 150, a parameter generation unit 152, a lens position determination unit 154, an offset determination unit 160, a focus target identifier 156, a confidence value unit 158, and an adjusted high-resolution lens determination unit 166. In some implementations, one or more of the operations performed by the post-shutter autofocus unit 122 may be performed using one or more machine-learning networks 165.

[0050] To perform the post-shutter autofocus operation, the post-shutter autofocus frame generator 150 may be configured to sense one or more post-shutter autofocus frames 162 after detection of the shutter activation signal 130. For example, in the implementation depicted in Figure 1, the post-shutter autofocus frame generator 150 may generate a postshutter autofocus frame 162 A and a post-shutter autofocus frame 162B in response to detection of the shutter activation signal 130. Although two (2) post-shutter autofocus frames 162 are depicted in Figure 1, in other implementations, the post-shutter autofocus frame generator 150 may generate additional (or fewer) post-shutter autofocus frames 162. As a non-limiting example, in some implementations, the post-shutter autofocus frame generator 150 may generate five (5) post-shutter autofocus frames 162 after detection of the shutter activation signal 130. In other implementations, the post-shutter autofocus frame generator 150 may generate a single post-shutter autofocus frame 162 after detection of the shutter activation signal 130. The post-shutter autofocus frames 162 may be generated from the low-resolution image stream 132 sensed from the image sensor 106 while operating in the low-resolution mode 136. Thus, to perform the post-shutter autofocus operation, the sensor mode controller 116 does not immediately switch from the low-resolution mode 136 to the high-resolutionmode. Instead, the sensor mode controller 116 continues to operate the image sensor 106 in the low-resolution mode 136 to facilitate generation of the post-shutter autofocus frames 162.

[0051] The post-shutter autofocus unit 122 may be configured to analyze the postshutter autofocus frames 126 (e.g., frames of the low-resolution image stream 132) to determine a low-resolution lens position 192 for capturing the focus target 170 in sharp contrast and a confidence value 172 associated with the low-resolution lens position 192. For example, the focus target identifier 156 may identify the focus target 170 in one or more of the postshutter autofocus frames 162. The focus target 170 may indicate a region within the postshutter autofocus frames 162 in which to focus. The lens position determination unit 154 may determine the lens position (e.g., the low-resolution lens position 192) of the lens 109 that would result in capturing the focus target in sharp contrast. The confidence value unit 158 may determine a confidence value 172 that indicates the likelihood that the low-resolution lens position 192 of the lens 109 can adequately capture the focus target 170 (e.g., capture the focus target 170 in sharp contrast).

[0052] After the low-resolution lens position 192 is determined, the lens position determination unit 154 may be configured to map the low-resolution lens position 192 to a high-resolution lens position 194. For example, the lens position determination unit 154 may determine the lens position in the high-resolution mode 138 that is substantially equivalent to the low-resolution lens position 192.

[0053] However, in some instances, during the high-resolution mode 138 in a dark lighting condition 142, if the lens 109 is in the high-resolution lens position 194 that is substantially equivalent to the low-resolution lens position 192, the focus target 170 may be blurry during a high-resolution image capture. For example, if the image sensor 106 captures twelve (12) megapixel images while operating in the low-resolution mode 136 and captures fifty (50) megapixel images while operating in the high-resolution mode 138, each pixel in the low-resolution mode 136 may be represented by approximately four (4) pixels in the high- resolution mode 138. Thus, if the focus target 170 is a particular low-resolution pixel when the lens 109 is in the low-resolution lens position 192 during the low-resolution mode 136, there may not be a single high-resolution pixel in the high-resolution mode 138 that directly maps to the particular low-resolution pixel. For example, when the lens 109 is in the high- resolution lens position 194 during the high-resolution mode 138, a portion (e.g., a subset) of the focus target 170 may be in focus. To compensate, the post-shutter autofocus unit 122 may adjust (e.g., shift 176) the high-resolution lens position 194 by performing a defocusing operation.

[0054] The determination of whether to shift 176 the high-resolution lens position 194 may be based on the confidence value 172. For example, the confidence value 172 may be correlated to a margin of error associated with the determination that the low-resolution lens position 192 will adequately capture the focus target 170. A smaller confidence value 172 may indicate there is a large margin of error, and as a result, the image controller 102 (e.g., the lens controller 114) may need to shift 176 the high-resolution lens position 194 by a relatively large offset 180 to ensure that the lens 109 is outside of the blurred image range for the focus target 170 in the high-resolution mode 138, as described with respect to Figure 2A.

[0055] To illustrate, based on the confidence value 172, the offset determination unit 160 may determine whether to shift 176 the high-resolution lens position 194. In response to determining to shift 176 the high-resolution lens position 194, the offset determination unit 160 may determine, based on the confidence value 172, the offset 180 to shift the high-resolution lens position 194 to ensure that the lens is outside of the blurred image range for the focus target 170.

[0056] In some implementations, the offset 180 may be determined based on additional parameters 164. For example, the parameter generation unit 152 may analyze the post-shutter autofocus frames 162 to determine a depth of field 164 A associated with the post-shutter autofocus frames 162, an optical spectrum 164B associated with the post-shutter autofocus frames 162, and / or a sensor pixel size 164C associated with the image sensor 106. In addition to the confidence value 172, the offset determination unit 160 may use the depth of field 164 A, the optical spectrum 164B, and / or the sensor pixel size 164C to determine the offset 180.

[0057] After the offset 180 is determined, the sensor mode controller 116 may be configured to switch the image sensor 106 from the low-resolution mode 136 to the high- resolution mode 138. The adjusted high-resolution lens determination unit 166 may apply the offset to the high-resolution lens position 194 to determine the adjusted high-resolution lens position 196. The lens controller 114 may be configured to move the lens to the adjusted high- resolution lens position 196. Once the lens 109 is in the adjusted high-resolution lens position 196, the high-resolution image generator 120 may capture the high-resolution image 190 such that the focus target 170 is relatively sharp.

[0058] The techniques described with respect to Figure 1 improve the autofocus for a high-resolution image capture in low light conditions while reducing power consumption at a device 100 and relaxing a memory footprint at the device 100. For example, in capturing the high-resolution image 190, power consumption and the memory footprint may be relaxed by operating the image sensor 106 in the low-resolution mode 136, instead of the high-resolutionmode 138, until detection of the shutter. Furthermore, the techniques described herein improve the sharpness of the focus target 170 in the high-resolution image 190 by performing the postshutter autofocus operation in the dark lighting condition. In particular, the post-shutter autofocus unit 122 determines whether (and how much) to shift 176 the mapped high-resolution lens position 194 of the lens 109 to ensure that the lens 109 is not within a blurred image region that would result in a blurry focus target 170 during the high-resolution image capture.III. Example Graphical Representations of Focus Target Sharpness

[0059] Figure 2A illustrates an example graphical representation 200A of shifting a high-resolution lens position out of blurred image range in low-light conditions using the postshutter autofocus operation when a low-resolution confidence focus is relatively high, in accordance with examples described herein. The graphical representation 200A depicts a horizontal axis 202 and a vertical axis 204. The horizontal axis 202 represents a lens position of the lens 109. The vertical axis 204 represents the sharpness of the focus target 170.

[0060] For each image stream 132, 134, the graphical representation 200A depicts a relationship between the lens position of the lens 109 and an image sharpness of the focus target 170. To illustrate, for the low-resolution image stream 132, the dotted plot depicts a relationship between the lens position of the lens 109 and an image sharpness of the focus target 170. For the high-resolution image stream 134, the solid plot depicts a relationship between the lens position of the lens 109 and an image sharpness of the focus target 170.

[0061] As depicted in the graphical representation 200A, for the low-resolution image stream 132, there is a parabolic relationship between the lens position of the lens 109 and the image sharpness focus target 170. For example, as the lens position moves along the horizontal axis towards the right direction, an image sharpness of the focus target 170 in the low- resolution image stream 132 increases until a peak low-resolution focus 292 is achieved. The peak low-resolution focus 292 may be achieved when the lens 109 is in the low-resolution lens position 192. If the lens position continues to move along the horizontal axis in the right direction after the lens position is in the low-resolution lens position 192, the image sharpness of the focus target 170 in the low-resolution image stream 132 decreases.

[0062] As depicted in the graphical representation 200A, for the high-resolution image stream 134, as the lens position of the lens 109 moves along the horizontal axis in the right direction, the image sharpness of the focus target 170 in the high-resolution image stream 134 may increase. However, as the lens position reaches the high-resolution lens position 194 (e.g., a lens position mapped from the low-resolution lens position 192), the image sharpness of the focus target 170 in the high-resolution image stream 134 drastically decreases (e.g., dips). Forexample, in Figure 2A, when the lens 109 is at the high-resolution lens position 194, the high- resolution focus 294 is relatively low (e.g., at a sharpness notch). In the scenario depicted in Figure 2A, the high-resolution focus 294 is less than the peak low-resolution focus 292.

[0063] The high-resolution focus 294 at the high-resolution lens position 194 is low due to a dip in image sharpness in the high-resolution mode 138 as the lens 109 approaches the low-resolution lens position 192. As depicted in Figure 2A, during the high-resolution mode 138, as the lens 109 moves closer to the low-resolution lens position 192, the image sharpness of the focus target 170 drastically dips to a sharpness notch 250. As the lens 109 moves past the low-resolution lens position 192 in the high-resolution mode 138, the image sharpness of the focus target 170 drastically increases and then begins to gradually decrease.

[0064] Thus, in the high-resolution mode 138, there is a blurred image range 210 (e.g., a lens position range indicated by the light gray shading) around the low-resolution lens position 192 that results in a drastic decrease in image sharpness of the focus target 170. Because the high-resolution lens position 194 falls within the blurred image range 210, the image sharpness of the focus target 170 at the high-resolution lens position 194 (e.g., the high- resolution focus 294) is relatively low.

[0065] By shifting the high-resolution lens position 194 by the offset 180 determined during the post-shutter autofocus operation, the position of the lens 109 moves from the high- resolution lens position 194 to the adjusted high-resolution lens position 196. In Figure 2A, the confidence value 172 of the low-resolution focus is relatively high. Thus, in Figure 2A, the lens 109 may be shifted out of the blurred image range 210 without having a dedicated range, such as the relaxed notch region 260 in Figure 2B, in which to shift.

[0066] Figure 2B illustrates an example graphical representation 200B of shifting a high-resolution lens position out of blurred image range in low-light conditions using the postshutter autofocus operation when a low-resolution confidence focus is relatively low, in accordance with examples described herein, in accordance with examples described herein.

[0067] As depicted in Figure 2B, a relaxed notch region 260 is depicted proximate to the blurred image range 210. When the low-resolution confidence focus is relatively low (e.g., a low confidence value 172), the offset 180 for shifting the position of the lens 109 is increased. As a result, the adjusted high-resolution lens position 196 has to be shifted outside of the relaxed notch region 260, as depicted in Figure 2B, to ensure the position of the lens 109 is out of the blurred image range 210.

[0068] Figure 2C illustrates an example graphical representation 200C of potentially shifting a high-resolution lens position out of blurred image range in low-light conditions usingthe post-shutter autofocus operation when a low-resolution confidence focus is low, in accordance with examples described herein, in accordance with examples described herein.

[0069] As depicted in Figure 2C, a notch region 262 is depicted proximate to the notch region 260. When the low-resolution confidence focus is low enough such that the offset 180 is outside the notch region 262, shifting the position of the lens 109 may be bypassed because the resulting sharpness, due to the shift, would be very low. However, as the low-resolution confidence focus increases such that the offset 180 is reduced (e.g., within the notch region 262), the position of the lens 109 may be shifted to move the lens outside of the blurred image range 210.

[0070] Thus, the graphical representations described with respect to Figures 2A, 2B, and 2C depict how the lens 109 is shifted based on the low-resolution confidence focus. In scenarios where the low-resolution confidence focus is relatively high, such as in Figure 2A, the high-resolution lens position 194 may be shifted to avoid the blurred image range 210. In scenarios where the low-resolution confidence focus is lower, such as in Figure 2B, a larger shift (e.g., outside of the notch region 260) of the high-resolution lens position 194 may be performed to ensure that the position of the lens 109 is outside the blurred image range 210. However, in scenarios where the low-resolution confidence is so low that the such that the shift of the high-resolution lens position 194 has to be outside of the notch region 262 to ensure that the position of the lens 109 is outside the blurred image range 210, shifting of the lens 109 may be bypassed because of a resulting poor image quality.IV. Example Process of Selectively Performing a Post-Shutter Autofocus Operation

[0071] Figure 3 illustrates an example of a process 300 of selectively performing a post-shutter autofocus operation, in accordance with examples described herein. The process 300 may be performed by the image controller 102 of Figure 1.

[0072] At process step 302, a high-resolution capture is triggered while a low- resolution output is streamed. For example, referring to Figure 1, the image controller 102 detects the shutter activation signal 130 while sensing the low-resolution image stream 132.

[0073] At decision step 304, the process 300 determines whether a post-shutter autofocus operation is needed. The determination as to whether the post-shutter autofocus operation is needed may be based on environmental lighting conditions. For example, if a bright lighting condition 140 exists, the image controller 102 may bypass performance of the post-shutter autofocus operation and may perform static mapping, at process step 306, between the low-resolution image stream 132 and the high-resolution image stream 134. However, ifthe dark lighting condition 142 exists, the image controller 102 may perform the post-shutter autofocus operation, at process step 308, to map the focus target 170 in the low-resolution image stream 132 to the focus target 170 in high-resolution image stream 134.

[0074] During the post-shutter autofocus operation, at process step 308, the image controller 102 may identify the focus target 170 and determine a confidence value 172 that the low-resolution lens position 192 adequately captures the focus target 170. Based on the focus target 170 and the confidence value 172, the image controller 102 may determine a margin of error, at process step 310. A high confidence value 172 may denote a small margin of error, and a low confidence value 172 may denote a high margin of error.

[0075] At decision step 312, the process 300 determines whether the margin of error is valid. If the margin of error is not valid (e.g., the margin of error is relatively high or is greater than a threshold), static mapping is performed, at process step 306. However, if the margin of error is valid, dynamic mapping is performed, at process step 314. As used herein, dynamic mapping may include performing static mapping (e.g., mapping the low-resolution lens position 192 to the high-resolution lens position 194) and adjusting the high-resolution lens position 194 by the offset to ensure the lens 109 is not within the blurred image range 210.

[0076] Figure 4 illustrates an example of a process 400 for performing a post-shutter autofocus operation, in accordance with examples described herein. The process 400 may be performed by the image controller 102 of Figure 1.

[0077] According to the process 400, one or more low-resolution frames 402 may be sensed while the image sensor 106 is operating in the low-resolution mode 136. As depicted in Figure 4, a low-resolution frame 402A and a low-resolution frame 402B are sensed by the image sensor 106. The low-resolution frames 402 may be part of the low-resolution image stream 132.

[0078] In Figure 4, the shutter activation signal 130 is detected after the low-resolution frame 402B is sensed. In response to detection of the shutter activation signal 130, one or more post-shutter autofocus frames 162 are sensed by the image sensor 106. As depicted in Figure 4, the post-shutter autofocus frame 162 A and the post-shutter autofocus frame 162B are sensed by the image sensor 106. However, in other implementations, additional post-shutter autofocus frames 162 may be sensed by the image sensor 106. The post-shutter autofocus frames 162 may also be low-resolution frames. For example, the image sensor 106 may sense the postshutter autofocus frames 162 while operating in the low-resolution mode 136.

[0079] A post-shutter autofocus algorithm 450 may be implemented using one or more of the post-shutter autofocus frames 162. For example, as described with respect to Figure 1,the post-shutter autofocus unit 122 may use the post-shutter autofocus frames 162 to determine (i) the low-resolution lens position 192 for the focus target 170 and (ii) the confidence value 172 associated with the low-resolution lens position 192. The post-shutter autofocus unit 122 may map the low-resolution lens position 192 to the corresponding high-resolution lens position 194. Based on the confidence value 172, the post-shutter autofocus unit 122 may determine (i) whether to shift 176 the high-resolution lens position 194 to improve the high- resolution image capture and (ii) the offset 180 for shifting the high-resolution lens position 194 to improve the high-resolution image capture.

[0080] As depicted in Figure 4, after performance of the post-shutter autofocus algorithm 450 (e.g., determining whether, and how much, to shift the high-resolution lens position 194), the image sensor 106 may switch to the high-resolution mode 138, implement the determined shift 176, and capture the high-resolution image 190.V. Example Machine Learning Process

[0081] Figure 5 shows a diagram 500 illustrating a training phase 502 and an inference phase 504 of trained machine learning model(s) 532, in accordance with example embodiments. According to some examples, the trained machine learning model(s) 532 can correspond to the machine learning network(s) 165. Some machine learning techniques involve training one or more machine learning algorithms on an input set of training data to recognize patterns in the training data and provide output inferences and / or predictions about (patterns in the) training data. The resulting trained machine learning algorithm can be termed as a trained machine learning model. For example, Figure 5 shows the training phase 502 where machine learning algorithm(s) 520 are being trained on training data 510 to become trained machine learning model(s) 532. Then, during the inference phase 504, the trained machine learning model(s) 532 can receive input data 530 and one or more inference / prediction requests 540 (perhaps as part of the input data 530) and responsively provide as an output one or more inferences and / or prediction(s) 550.

[0082] As such, the trained machine learning model(s) 532 can include one or more models of machine learning algorithm(s) 520. The machine learning algorithm(s) 520 may include, but are not limited to: an artificial neural network (e.g., a herein-described convolutional neural networks, a recurrent neural network, a Bayesian network, a hidden Markov model, a Markov decision process, a logistic regression function, a support vector machine, a suitable statistical machine learning algorithm, and / or a heuristic machine learning system). The machine learning algorithm(s) 520 may be supervised or unsupervised, and may implement any suitable combination of online and offline learning.

[0083] In some examples, the machine learning algorithm(s) 520 and / or the trained machine learning model(s) 532 can be accelerated using on-device coprocessors, such as graphic processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), and / or application specific integrated circuits (ASICs). Such on-device coprocessors can be used to speed up the machine learning algorithm(s) 520 and / or the trained machine learning model(s) 532. In some examples, the trained machine learning model(s) 532 can be trained, resided and executed to provide inferences on a particular computing device, and / or otherwise can make inferences for the particular computing device.

[0084] During the training phase 502, the machine learning algorithm(s) 520 can be trained by providing at least the training data 510 as training input using unsupervised, supervised, semi-supervised, and / or reinforcement learning techniques. Unsupervised learning involves providing a portion (or all) of the training data 510 to the machine learning algorithm(s) 520 and the machine learning algorithm(s) 520 determining one or more output inferences based on the provided portion (or all) of the training data 510. Supervised learning involves providing a portion of the training data 510 to the machine learning algorithm(s) 520, with the machine learning algorithm(s) 520 determining one or more output inferences based on the provided portion of the training data 510, and the output inference(s) are either accepted or corrected based on correct results associated with the training data 510. In some examples, supervised learning of the machine learning algorithm(s) 520 can be governed by a set of rules and / or a set of labels for the training input, and the set of rules and / or set of labels may be used to correct inferences of the machine learning algorithm(s) 520.

[0085] Semi-supervised learning involves having correct results for part, but not all, of the training data 510. During semi-supervised learning, supervised learning is used for a portion of the training data 510 having correct results, and unsupervised learning is used for a portion of the training data 510 not having correct results. Reinforcement learning involves the machine learning algorithm(s) 520 receiving a reward signal regarding a prior inference, where the reward signal can be a numerical value. During reinforcement learning, the machine learning algorithm(s) 520 can output an inference and receive a reward signal in response, where the machine learning algorithm(s) 520 are configured to try to maximize the numerical value of the reward signal. In some examples, reinforcement learning also utilizes a value function that provides a numerical value representing an expected total of the numerical values provided by the reward signal over time. In some examples, the machine learning algorithm(s)520 and / or the trained machine learning model(s) 532 can be trained using other machine learning techniques, including but not limited to, incremental learning and curriculum learning.

[0086] In some examples, the machine learning algorithm(s) 520 and / or the trained machine learning model(s) 532 can use transfer learning techniques. For example, transfer learning techniques can involve the trained machine learning model(s) 532 being pre-trained on one set of data and additionally trained using the training data 510. More particularly, the machine learning algorithm(s) 520 can be pre-trained on data from one or more computing devices and a resulting trained machine learning model provided to a particular computing device, where the particular computing device is intended to execute the trained machine learning model during the inference phase 504. Then, during the training phase 502, the pretrained machine learning model can be additionally trained using the training data 510, where the training data 510 can be derived from kernel and non-kernel data of the particular computing device. This further training of the machine learning algorithm(s) 520 and / or the pre-trained machine learning model using the training data 510 of the particular computing device’s data can be performed using either supervised or unsupervised learning. Once the machine learning algorithm(s) 520 and / or the pre-trained machine learning model has been trained on at least the training data 510, the training phase 502 can be completed. The trained resulting machine learning model can be utilized as at least one of the trained machine learning model(s) 532.

[0087] In particular, once the training phase 502 has been completed, the trained machine learning model(s) 532 can be provided to a computing device, if not already on the computing device. The inference phase 504 can begin after training the machine learning model(s) 532 are provided to the particular computing device.

[0088] During the inference phase 504, the trained machine learning model(s) 532 can receive the input data 530 and generate and output one or more corresponding inferences and / or prediction(s) 550 about the input data 530. As such, the input data 530 can be used as an input to the trained machine learning model(s) 532 for providing corresponding inference(s) and / or prediction(s) 550 to kernel components and non-kernel components. For example, the trained machine learning model(s) 532 can generate inference(s) and / or prediction(s) 550 in response to one or more inference / prediction requests 540. In some examples, the trained machine learning model(s) 532 can be executed by a portion of other software. For example, the trained machine learning model(s) 532 can be executed by an inference or prediction daemon to be readily available to provide inferences and / or predictions upon request. The input data 530 can include data from the particular computing device executing the trained machine learningmodel(s) 532 and / or input data from one or more computing devices other than the particular computing device.

[0089] If the trained machine learning model 532 corresponds to the machine learning network(s) 165, the input data 530 can include the post-shutter autofocus frames 162, an indication of the focus target 170, and data indicative of the lighting condition. Other types of input data are possible as well. Inference(s) and / or prediction(s) 550 can include other output data produced by the trained machine learning model(s) 532 operating on the input data 530 (and the training data 510). In some examples, the trained machine learning model(s) 532 can use output inference(s) and / or prediction(s) 550 as input feedback 560. The trained machine learning model(s) 532 can also rely on past inferences as inputs for generating new inferences.

[0090] Convolutional neural networks and / or deep neural networks used herein can be an example of the machine learning algorithm(s) 520. After training, the trained version of a convolutional neural network can be an example of the trained machine learning model(s) 532.VI. Additional Example Operations

[0091] Figure 6 illustrates a flow chart of a method 600 related to a new technology. The method 600 may be carried out by the device 100 among other possibilities. The embodiments of Figure 6 may be simplified by the removal of any one or more of the features shown therein. Further, these embodiments may be combined with features, aspects, and / or implementations of any of the previous figures or otherwise described herein.

[0092] The method 600 includes sensing, by an image controller, a low-resolution image stream captured by an image sensor operating in a low-resolution mode, at block 602. For example, referring to Figure 1, the image controller 102 senses the low-resolution image stream 132 captured by the image sensor 106 operating in the low-resolution mode 136.

[0093] The method 600 also includes detecting, by the image controller, a shutter activation signal while sensing the low-resolution image stream, at block 604. For example, referring to Figure 1, the image controller 102 detects the shutter activation signal 130 while sensing the low-resolution image stream 132.

[0094] The method 600 also includes performing a post-shutter autofocus operation at least in response to detecting the shutter activation signal. Performing the post-shutter autofocus operation includes analyzing one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position, at block 606. For example, referring to Figure 1, the post-shutter autofocus unit 122 performs the post-shutter autofocus operation. To illustrate, the post-shutter autofocus unit 122 analyzes the post-shutter autofocus frames 162 of the low-resolution image stream 132 to determine the low-resolution lens position 192 and the confidence value 172 associated with the low-resolution lens position 192.

[0095] Performing the post-shutter autofocus operation also includes mapping the low- resolution lens position to a high-resolution lens position, at block 608. For example, referring to Figure 1, the post-shutter autofocus unit 122 maps the low-resolution lens position 192 to the high-resolution lens position 194.

[0096] Performing the post-shutter autofocus operation also includes determining, based at least on the confidence value, an offset to shift the high-resolution lens position, at block 610. For example, referring to Figure 1, the post-shutter autofocus unit 122 determines, based at least on the confidence value 172, the offset 180 to shift the high-resolution lens position 194.

[0097] The method 600 also includes switching the image sensor from the low- resolution mode to a high-resolution mode, at block 612. For example, referring to Figure 1, the image controller switches the image sensor 106 from the low-resolution mode 136 to the high-resolution mode 138.

[0098] The method 600 also includes adjusting the high-resolution lens position by the offset to capture a high-resolution image while the image sensor is operating in the high- resolution mode, at block 614. For example, referring to Figure 1, the image controller adjusts the high-resolution lens position 194 by the offset 180 to capture the high-resolution image 190 while the image sensor 106 is operating in the high-resolution mode 138.

[0099] According to one implementation, prior to performing the post-shutter autofocus operation, the method 600 may include determining whether a lighting condition proximate to the image sensor corresponds to a bright lighting condition or a dark lighting condition. The post-shutter autofocus operation may be performed in response to a determination that the lighting condition corresponds to the dark lighting condition.

[0100] According to one implementation of the method 600, performing the post-shutter autofocus operation includes determining, based on the confidence value, whether to shift the high-resolution lens position and determining the offset in response to a determination to shift the high-resolution lens position.

[0101] According to one implementation of the method 600, adjusting the high- resolution lens position by the offset moves a lens out of a blurred image region. An image sharpness of high-resolution images is reduced when the lens is within the blurred image region.

[0102] According to one implementation of the method 600, adjusting the high- resolution lens position includes performing a defocusing operation.

[0103] According to one implementation of the method 600, the offset is determined based on at least one other parameter. The at least one other parameter may include a depth of field associated with the one or more frames, an optical spectrum associated with the one or more frames, or a sensor pixel size associated with the image sensor.

[0104] According to one implementation, the image sensor captures twelve (12) megapixel images while operating in the low-resolution mode. According to one implementation, the image sensor captures fifty (50) megapixel images while operating in the high-resolution mode.

[0105] The method 600 of Figure 6 may be used to ensure that the position of the lens 109 does not fall within the blurred image range 210 in the high-resolution mode 138. In particular, the post-shutter autofocus operation may be used to shift 176 the position of the lens 109, from the high-resolution lens position 194, out of the blurred image range 210 to ensure a relatively high image focus target sharpness value for the high-resolution image 190 captured in the high-resolution mode 138.VII. Conclusion

[0106] The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from its scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those described herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims.

[0107] The above detailed description describes various features and operations of the disclosed systems, devices, and methods with reference to the accompanying figures. In the figures, similar symbols typically identify similar components, unless context dictates otherwise. The example embodiments described herein and in the figures are not meant to be limiting. Other embodiments can be utilized, and other changes can be made, without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations.

[0108] With respect to any or all of the message flow diagrams, scenarios, and flow charts in the figures and as discussed herein, each step, block, and / or communication can represent a processing of information and / or a transmission of information in accordance with example embodiments. Alternative embodiments are included within the scope of these example embodiments. In these alternative embodiments, for example, operations described as steps, blocks, transmissions, communications, requests, responses, and / or messages can be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved. Further, more or fewer blocks and / or operations can be used with any of the message flow diagrams, scenarios, and flow charts discussed herein, and these message flow diagrams, scenarios, and flow charts can be combined with one another, in part or in whole.

[0109] A step or block that represents a processing of information may correspond to circuitry that can be configured to perform the specific logical functions of a herein-described method or technique. Alternatively or additionally, a block that represents a processing of information may correspond to a module, a segment, or a portion of program code (including related data). The program code may include one or more instructions executable by a processor for implementing specific logical operations or actions in the method or technique. The program code and / or related data may be stored on any type of computer readable medium such as a storage device including random access memory (RAM), a disk drive, a solid state drive, or another storage medium.

[0110] The computer readable medium may also include non-transitory computer readable media such as computer readable media that store data for short periods of time like register memory, processor cache, and RAM. The computer readable media may also include non-transitory computer readable media that store program code and / or data for longer periods of time. Thus, the computer readable media may include secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, solid state drives, compact-disc read only memory (CD-ROM), for example. The computer readable media may also be any other volatile or non-volatile storage systems. A computer readable medium may be considered a computer readable storage medium, for example, or a tangible storage device.

[0111] Moreover, a step or block that represents one or more information transmissions may correspond to information transmissions between software and / or hardware modules in the same physical device. However, other information transmissions may be between software modules and / or hardware modules in different physical devices.

[0112] The particular arrangements shown in the figures should not be viewed as limiting. It should be understood that other embodiments can include more or less of each element shown in a given figure. Further, some of the illustrated elements can be combined or omitted. Yet further, an example embodiment can include elements that are not illustrated in the figures.

[0113] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for the purpose of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.

Claims

CLAIMSWhat is claimed is:

1. A method of capturing a high-resolution image, the method comprising: sensing, by an image controller, a low-resolution image stream captured by an image sensor operating in a low-resolution mode; detecting, by the image controller, a shutter activation signal while sensing the low- resolution image stream; performing a post-shutter autofocus operation at least in response to detecting the shutter activation signal, wherein performing the post-shutter autofocus operation comprises: analyzing one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position; mapping the low-resolution lens position to a high-resolution lens position; and determining, based at least on the confidence value, an offset to shift the high- resolution lens position; switching the image sensor from the low-resolution mode to a high-resolution mode; and adjusting the high-resolution lens position by the offset to capture the high-resolution image while the image sensor is operating in the high-resolution mode.

2. The method of claim 1, further comprising, prior to performing the post-shutter autofocus operation: determining whether a lighting condition proximate to the image sensor corresponds to a bright lighting condition or a dark lighting condition, wherein the postshutter autofocus operation is performed in response to a determination that the lighting condition corresponds to the dark lighting condition.

3. The method of claim 1, wherein performing the post-shutter autofocus operation further comprises: determining, based on the confidence value, whether to shift the high-resolution lens position; anddetermining the offset in response to a determination to shift the high-resolution lens position.

4. The method of claim 1, wherein adjusting the high-resolution lens position by the offset moves a lens out of a blurred image region, wherein an image sharpness of high- resolution images is reduced when the lens is within the blurred image region.

5. The method of claim 1, wherein adjusting the high-resolution lens position comprises performing a defocusing operation.

6. The method of claim 1, wherein the offset is determined based on at least one other parameter.

7. The method of claim 6, wherein the at least one other parameter comprises a depth of field associated with the one or more frames.

8. The method of claim 6, wherein the at least one other parameter comprises a sensor pixel size associated with the image sensor.

9. The method of claim 6, wherein the at least one other parameter comprises an optical spectrum associated with the one or more frames.

10. The method of claim 1, wherein the image sensor captures low-resolution images while operating in the low-resolution mode.

11. The method of claim 1, wherein the image sensor captures high-resolution images while operating in the high-resolution mode.

12. A device comprising: a memory; and an image sensor; and an image controller coupled to the image sensor and to the memory, the image controller configured to: sense a low-resolution image stream captured by the image sensor operating in a low-resolution mode;detect a shutter activation signal while sensing the low-resolution image stream; perform a post-shutter autofocus operation at least in response to detecting the shutter activation signal, wherein, to perform the post-shutter autofocus operation, the image controller is configured to: analyze one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position; map the low-resolution lens position to a high-resolution lens position; and determine, based at least on the confidence value, an offset to shift the high-resolution lens position; switch the image sensor from the low-resolution mode to a high-resolution mode; and adjust the high-resolution lens position by the offset to capture a high- resolution image while the image sensor is operating in the high- resolution mode.

13. The device of claim 12, wherein, prior to performing the post-shutter autofocus operation, the image controller is configured to: determine whether a lighting condition proximate to the image sensor corresponds to a bright lighting condition or a dark lighting condition, wherein the postshutter autofocus operation is performed in response to a determination that the lighting condition corresponds to the dark lighting condition.

14. The device of claim 12, wherein, to perform the post-shutter autofocus operation, the image controller is further configured to: determine, based on the confidence value, whether to shift the high-resolution lens position; and determine the offset in response to a determination to shift the high-resolution lens position.

15. The device of claim 12, wherein adjusting the high-resolution lens position by the offset moves a lens out of a margin of error, wherein an image sharpness of high- resolution images is reduced when the lens is within the margin of error.

16. The device of claim 12, wherein, to adjust the high-resolution lens position, the image controller is configured to perform a defocusing operation.

17. The device of claim 12, wherein the offset is determined based on at least one other parameter, and wherein the at least one other parameter comprises a depth of field associated with the one or more frames, a sensor pixel size associated with the image sensor, or an optical spectrum associated with the one or more frames.

18. A non-transitory computer-readable medium comprising instructions that, when executed by an image controller of a device, cause the image controller to perform operations comprising: sensing a low-resolution image stream captured by an image sensor operating in a low-resolution mode; detecting a shutter activation signal while sensing the low-resolution image stream; performing a post-shutter autofocus operation at least in response to detecting the shutter activation signal, wherein performing the post-shutter autofocus operation comprises: analyzing one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position; mapping the low-resolution lens position to a high-resolution lens position; and determining, based at least on the confidence value, an offset to shift the high- resolution lens position; switching the image sensor from the low-resolution mode to a high-resolution mode; and adjusting the high-resolution lens position by the offset to capture a high-resolution image while the image sensor is operating in the high-resolution mode.

19. The non-transitory computer-readable medium of claim 18, wherein, prior to performing the post-shutter autofocus operation, the operations comprise: determining whether a lighting condition proximate to the image sensor corresponds to a bright lighting condition or a dark lighting condition, wherein the postshutter autofocus operation is performed in response to a determination that the lighting condition corresponds to the dark lighting condition.

20. The non-transitory computer-readable medium of claim 18, wherein performing the post-shutter autofocus operation further comprises: determining, based on the confidence value, whether to shift the high-resolution lens position; and determining the offset in response to a determination to shift the high-resolution lens position.

21. A computer program product comprising a computer hardware storage device having stored therein computer-executable program code for adjusting a facial region of interest, the computer-executable program code, when executed by a computer, causes the computer to: sense a low-resolution image stream captured by an image sensor operating in a low- resolution mode; detect a shutter activation signal while sensing the low-resolution image stream; perform a post-shutter autofocus operation at least in response to detecting the shutter activation signal, wherein performing the post-shutter autofocus operation comprises: analyzing one or more frames of the low-resolution image stream to determine a low-resolution lens position for a focus target and a confidence value associated with the low-resolution lens position; mapping the low-resolution lens position to a high-resolution lens position; and determining, based at least on the confidence value, an offset to shift the high- resolution lens position; switch the image sensor from the low-resolution mode to a high-resolution mode; and adjust the high-resolution lens position by the offset to capture a high-resolution image while the image sensor is operating in the high-resolution mode.

22. The computer program product of claim 21, wherein, prior to performing the post-shutter autofocus operation, the computer-executable program code, when executed by the computer, causes the computer to: determine whether a lighting condition proximate to the image sensor corresponds to a bright lighting condition or a dark lighting condition, wherein the postshutter autofocus operation is performed in response to a determination that the lighting condition corresponds to the dark lighting condition.

23. The computer program product of claim 21, wherein performing the post-shutter autofocus operation further comprises: determining, based on the confidence value, whether to shift the high-resolution lens position; and determining the offset in response to a determination to shift the high-resolution lens position.

Images