Dual Exposure Control in a Camera System

Abstract

Apparatuses and methods related to applying an illumination model to an image of an object are provided. A neural network can be trained to apply an illumination model to an input image. The training of the neural network can utilize confidence learning, which is based on a light prediction and a prediction confidence value associated with the illumination of the input image. A computing device can receive an input image of an object and data regarding a specific illumination model to be applied to the input image. The computing device can determine an output image of the object by applying the specific illumination model to the input image of the object using the trained neural network.

Description

Background Art

[0001] Many modern computing devices, including mobile phones, personal computers, and tablet computers, include image capture devices such as still cameras and / or video cameras. The image capture devices are capable of capturing images, such as images including people, animals, landscapes, and / or objects.

[0002] Some image capture devices and / or computing devices include a graphical display (e.g., a phone screen or an electronic viewfinder (EVF) on a mirrorless camera or a point-and-shoot camera) that serves as a viewfinder for the camera. Such camera devices typically display a preview of the image that will be captured if / when the user clicks (or taps) the shutter button. The preview is typically a video or image stream that is generated in real time based on what is sensed by the image sensor of the camera and may also be referred to as a live view preview image. Summary of the Invention

[0003] In one aspect, a computer-implemented method is provided. The method includes: (i) displaying, on a display device, a graphical interface for image capture, the graphical interface including: (a) a live view preview image based on an image data stream from an image sensor of an image capture device, and (b) a first control feature and a second control feature; (ii) receiving, via the graphical interface, first input data corresponding to a first interaction with the first control feature and responsively adjusting a first exposure setting based on the first interaction; (iii) applying the first exposure setting in real time to at least one highlighted region in the live view preview image; (iv) receiving, via the graphical interface, second input data corresponding to a second interaction with the second control feature and responsively adjusting a second exposure setting based on the second interaction; (v) applying the first exposure setting in real time to at least one highlighted region in the live view preview image; and (vi) subsequently receiving third input data corresponding to an image capture command and responsively operating the image capture device to capture an image according to both the first exposure setting and the second exposure setting.

[0004] In another aspect, a computing device is provided. The computing device includes one or more processors and one or more computer-readable media. Computer-executable instructions are stored on the one or more computer-readable media, and when executed by the one or more processors, the computer-executable instructions cause the computing device to perform functions. The functions include: (i) displaying on a display device a graphical user interface for image capture, the graphical user interface including: (a) a live view preview image based on an image data stream from an image sensor of an image capture device, and (b) a first control feature and a second control feature; (ii) receiving, via the graphical user interface, first input data corresponding to a first interaction with the first control feature, and responsive to the first interaction, adjusting a first exposure setting; (iii) applying the first exposure setting in real time to at least one highlighted region in the live view preview image; (iv) receiving, via the graphical user interface, second input data corresponding to a second interaction with the second control feature, and responsive to the second interaction, adjusting a second exposure setting; (v) applying the second exposure setting in real time to at least one highlighted region in the live view preview image; and (vi) subsequently receiving third input data corresponding to an image capture command, and responsive to the third input data, operating the image capture device to capture an image according to both the first exposure setting and the second exposure setting.

[0005] In another aspect, an article of manufacture is provided. The article of manufacture includes one or more computer-readable media having computer-readable instructions stored thereon, and when executed by one or more processors of a computing device, the computer-readable instructions cause the computing device to perform functions. The functions include: (i) displaying on a display device a graphical user interface for image capture, the graphical user interface including: (a) a live view preview image based on an image data stream from an image sensor of an image capture device, and (b) a first control feature and a second control feature; (ii) receiving, via the graphical user interface, first input data corresponding to a first interaction with the first control feature, and responsive to the first interaction, adjusting a first exposure setting; (iii) applying the first exposure setting in real time to at least one highlighted region in the live view preview image; (iv) receiving, via the graphical user interface, second input data corresponding to a second interaction with the second control feature, and responsive to the second interaction, adjusting a second exposure setting; (v) applying the second exposure setting in real time to at least one highlighted region in the live view preview image; and (vi) subsequently receiving third input data corresponding to an image capture command, and responsive to the third input data, operating the image capture device to capture an image according to both the first exposure setting and the second exposure setting.

[0006] The foregoing summary is illustrative only and is not intended to be limiting in any way. In addition to the above-described illustrative aspects, embodiments, and features, other aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 Disclosed is a computer-implemented method for providing an image capture interface.

[0008] Figures 2A to 2C Shown is an example of a single-slider live view interface for shadow and highlight control.

[0009] Figures 3A to 3C Shown is another example of a single-slider live view interface for shadow and highlight control.

[0010] Figure 4A Depicted is a convolutional neural network (CNN) architecture according to an example embodiment.

[0011] Figure 4B Depicted is a convolution according to an example embodiment.

[0012] Figure 5 Is a block diagram of an example computing device according to an example embodiment.

[0013] Figure 6 Depicted is a network of a computing cluster arranged as a cloud-based server system according to an example embodiment. DETAILED DESCRIPTION

[0014] This application describes methods and systems that facilitate providing control of a camera via an image capture interface. In particular, the methods and systems are capable of providing pre-image capture control of highlight levels and shadow levels independently of each other via a live view preview image.

[0015] Some camera applications, such as those on mobile devices, allow a user to adjust the global exposure used by the camera to capture an image before the image is captured. This global change in exposure can be reflected in the live view image that is displayed to preview the image that will be captured if the user presses the shutter button or otherwise initiates the image capture process. However, in many cases, the global adjustment of exposure is not sufficient, and the user may desire more localized, scene-specific, and / or artistic control.

[0016] Accordingly, illustrative graphical interfaces provide real-time control of both shadow and highlight independently of each other in a live view camera interface before the image is captured. In some implementations, adjustment of shadow and highlight via the live view interface can be achieved by separately adjusting the highlight total exposure time (TET) and the shadow TET, where the highlight TET and the shadow TET can affect different regions of the live view preview image and are used for image capture and / or image processing.

[0017] In this document, "Total Exposure Time" or "TET" can be the product of an exposure time (in some standard unit, such as milliseconds) and a gain multiplier. The exposure time can be set as part of the automatic exposure process of a camera device. Additionally, multiple TETs can be used to generate an output image or video, where different and / or additional gain multipliers are applied. For example, in an illustrative embodiment, a "highlight" or "short" TET can be the product of an exposure time and a first gain multiplier, while a "shadow" or "long" TET can be the product of the exposure time and a second gain multiplier (greater than the first gain multiplier). In some cases, a "long" TET can be achieved by applying additional digital gain to the short TET, such that long and short TETs are provided even when image capture uses only a single analog exposure time.

[0018] In some embodiments, a live view camera interface can include a single slider feature that allows for separate real-time adjustment of both the shadow level and the highlight level in a live view preview stream (such that the preview image has the same shadow and highlight levels as the image that will be captured subsequently via the shutter button on the live view interface). In other embodiments, the live view camera interface can include two slider features that allow for separate real-time adjustment of both the shadow level and the highlight level in the live view preview stream. Without departing from the scope of the present invention, other types of interface features can also provide separate real-time adjustment of the shadow level and the highlight exposure level.

[0019] In this document, a "live view preview image" should be understood as an image or sequence of images (e.g., video) that is generated and displayed based on an image data stream from an image sensor of an image capture device. For example, the image data can be generated by an image sensor of a camera (or a portion, subset, or sampling of the pixels on the image sensor). This image data represents the field of view (FOV) of the camera and thus indicates the image that will be captured in the event that the user presses the shutter button of the camera or otherwise initiates image capture. To assist the user in deciding how to position the camera for image capture, a camera device or other computing device can generate and display a live view preview image based on the image data stream from the image sensor. The live view preview image can be a real-time image feed (e.g., video) such that the user is notified in real time of the camera's FOV.

[0020] According to the illustrative methods and systems, a computing device can process an image data stream from a sensor to generate a live view preview image that more accurately represents the image or video that would actually be captured if / when the user initiates an image capture. Additionally, the computing device can provide an image capture interface that incorporates the live view preview image and includes control features for image processing. This arrangement enables adjustment of some image settings or attributes before the image is captured, which were previously only adjustable after the image was captured (e.g., via post-processing in an image editing application).

[0021] As used herein, the “shadow” or “shadow region” of an image (or sequence of images) should be understood to include the darkest pixel or region(s) in the image (or on the sequence of images). In practice, a pixel or region(s) in an image frame having a luminance level below a predetermined threshold can be identified as a shadow region. Of course, other methods for detecting darker regions eligible as shadow regions are also feasible.

[0022] As used herein, the “highlight” or “highlight region” of an image (or sequence of images) can include the brightest pixel or region(s) in the image (or on the sequence of images). In practice, a pixel or region in an image frame having a luminance level above a predetermined threshold can be identified as a highlight region. Note that different thresholds can be utilized to classify shadow and highlight regions (such that there can be regions that are not classified as shadow or highlight). Alternatively, the highlight and shadow thresholds can be the same (such that every region in the image is classified as shadow or highlight). Additionally, the shadow and / or highlight threshold(s) can also be adaptive and vary based on, for example, the characteristics of the scene being captured. In some cases, the shadow and / or highlight threshold(s) can vary spatially across the image frame. Other methods for detecting bright regions eligible as highlight regions are also feasible.

[0023] The embodiments described herein may allow for pre-image capture adjustment of the shadow level and highlight level to be performed separately from each other. Additionally, these adjustments may be applied in real-time to a live view preview such that the live view preview provides a more accurate representation of the image or video that will be captured in the event that the user presses the shutter button or initiates video capture. Further, adjusting the short TET and long TET (for shadow level and highlight level, respectively) prior to image capture can result in superior results compared to making such adjustments during post-processing after the image has been captured. In particular, example embodiments may adjust the capture TET (e.g., by adjusting the shutter speed, aperture, and / or image sensitivity) in order to adjust the short TET and long TET prior to image capture. Since adjusting the exposure time and analog gain (e.g., shutter speed, aperture, and / or image sensitivity) prior to image capture generally results in better image quality (e.g., less noise, fewer quantization artifacts, less clipping) than making the corresponding adjustment using digital gain during post-processing, cameras utilizing the methods and systems described herein may provide better image quality than they otherwise would.

[0024] I. Method for Providing an Image Capture Interface

[0025] Figure 1 is a computer-implemented method 100 for providing an image capture interface according to an example embodiment. The method may be implemented by an image capture device such as a digital camera or a mobile phone having one or more camera systems. Method 100 may also be implemented by a computing device in communication with the image capture device, such as by a mobile phone or other display device for controlling a stand-alone camera device (e.g., a DSLR camera or an action camera).

[0026] At block 102, method 100 involves displaying a graphical interface for image capture that includes (a) a live view preview image, and (b) a first control feature and a second control feature. At block 104, the computing device receives first input data corresponding to a first interaction with the first control feature via the graphical interface. In response, as shown at block 106, the computing device adjusts a first exposure setting based on the first interaction. The first control feature is capable of providing control over the exposure in a highlight region of an image frame of the camera. Thus, as shown at block 108, the computing device also applies the adjustment to the first exposure setting in real-time to at least one highlight region in the live view preview image.

[0027] Continuing with method 100, as shown at block 110, the computing device receives second input data corresponding to a second interaction with a second control feature via a graphical interface. In response, as shown at block 112, the computing device adjusts a second exposure setting based on the second interaction. The second control feature can provide control over the exposure in a dark region of the image frame of the camera. Thus, as shown at block 114, the computing device also applies the second exposure setting in real time to at least one dark region in the live view preview image.

[0028] Subsequently, as shown at block 116, the computing device receives third input data corresponding to an image capture command and responsively operates the image capture device to capture an image using both the first exposure setting and the second exposure setting. Note that either or both of the first exposure setting and the second exposure setting can be adjusted multiple times prior to the image capture at block 116 (e.g., at blocks 106 and 108 and at blocks 110 and 112, respectively).

[0029] In some embodiments of method 100, the first control feature and the second control feature can be part of a single exposure control feature in the image capture interface. For example, the first control feature and the second control feature can be graphical elements (e.g., circles or another shape) that are movable along the same slider feature. Specific examples of this arrangement are described later herein with reference to Figures 2A to 2C In other cases, the first control feature and the second control feature can be separate exposure control features in the image capture interface. Specific examples of this arrangement are described later herein with reference to Figures 3A to 3C In embodiments of method 100, other arrangements of the first control feature and the second control feature are also possible.

[0030] In some embodiments, the camera device or a control device communicatively coupled thereto can include physical controls that are mapped to control the first control feature and the second control feature (or possibly directly control the first exposure setting and the second exposure setting without any feedback from the graphical interface). For example, a DSLR or another type of camera can include a live view interface, as well as mechanical control features such as (a) control knob(s), (a) joystick(s), (a) button(s), and / or (a) slider(s). These mechanical control features can allow for real-time control of the first exposure setting and the second exposure setting such that changes to these settings are accurately represented in the live view interface.

[0031] II. Illustrative Graphical Interface

[0032] Figures 2A to 2CAn example of a single-slider live view interface 200 for shadow and highlight control is shown. Importantly, the live view interface 200 is capable of providing real-time pre-image capture control and visualization of shadow exposure levels and highlight exposure levels independently of each other in the live view preview 201. The live view interface 200 includes a slider 202 having a first control feature 204 and a second control feature 206. In most scenarios, the first control feature 204 and the second control feature 206 are independently movable on the slider 202 and control the highlight level and the shadow level, respectively. (For ease of explanation, the first control feature and the second control feature may also be referred to as the highlight control and the shadow control, respectively. However, it is contemplated that the configuration shown may be applied to other types of pre-image capture adjustments.)

[0033] In Figures 2A to 2C it, the live view interface 200 shows the change of the live view preview image 201 over time. Moving the highlight and / or shadow controls on the slider 202 will show the resulting adjustment to the camera exposure settings in real time in the live view preview image. Additionally, in Figures 2A to 2C each of them, the live view preview image 201 indicates the image or video that will be captured if the user taps the image capture button 208 at that time. Since the live view preview image 201 shows an accurate preview of the exposure settings that will be used for image capture, the live view interface 200 can be referred to as a what-you-see-is-what-you-get (WYSIWYG) viewfinder for the camera device.

[0034] In Figure 2A it, the shadow and highlight controls are positioned at approximately 1 / 3 and 2 / 3 of the height of the slider 202. In some embodiments, this particular arrangement may be the default arrangement, which is mapped to the auto exposure settings. Such auto exposure settings can be determined by the real-time image processing pipeline of the camera device (and are reflected in the Figure 2A shown live view preview image).

[0035] For example, in the absence of further input from the user, the auto - exposure process can determine a first exposure setting that controls the exposure settings for image capture. In an example embodiment, the first exposure setting can indicate the total exposure time (TET) for the entire image frame (e.g., global exposure, if the user makes no other adjustments). This exposure time can be achieved by setting the amount of analog gain (e.g., the sensitivity of the image sensor) and digital gain (e.g., the gain applied to the image data generated by the image sensor). In an implementation, the TET used to capture the image (e.g., "capture TET") can also be used as the exposure level for the highlight regions in the image and can thus also be referred to as the "highlight TET". (In practice, the shadow TET will typically be higher than the capture TET or the highlight TET so that shadow details are more visible). A longer TET can be determined for the shadow regions, which can be achieved by applying additional gain to the highlight TET. Thus, the highlight TET can also be characterized as a "short" TET, and the shadow TET can be characterized as a "long" TET.

[0036] Referring again to the image capture interface 200, the interface can also allow the user to adjust the exposure of the shadow regions separately from the highlight regions prior to image capture. Thus, the first exposure setting can effectively provide a mechanism for adjusting the exposure of the image highlights without adjusting the exposure level of the shadow regions. Thus, the first control feature 204 can allow the highlight exposure level to be controlled separately from the shadow exposure level. Correspondingly, the second exposure setting can indicate a second TET for the shadows (e.g., a local exposure setting). In this way, the second control feature 206 can allow adjustment of the second TET, which is achieved by applying additional gain (e.g., digital gain) to the shadow regions in the image frame. Thus, in most scenarios, adjusting the second TET will effectively adjust the exposure of the shadows in the live - view preview image (and the image captured via the shutter button 208) with little impact on the exposure of the highlights in the same image.

[0037] In some cases, separate control of the shadow level and the highlight level can be achieved by providing control over short exposure times and long exposure times (e.g., short TET and long TET). In a direct implementation, a short TET can be applied in the highlight region, and a long TET can be applied in the shadow region. In other implementations, the long / shadow TET can actually be a grid or matrix of gain values (or TET values) that specify the gain on a per-pixel or per-tile basis (where pixels or tiles are grouped within an image frame). In such an implementation, a pyramid-based multi-exposure technique or an approximation technique that utilizes machine learning (e.g., a convolutional neural network), such as Google's HDRnet, can be used to determine the shadow TET. Such techniques can help preserve small edge magnitudes (details) throughout the image that would otherwise be lost due to compression in the highlight region when applying gain to darker pixels using a simple global tone mapping.

[0038] Other processes for separately adjusting the shadow exposure level from the highlight are feasible. For example, a camera device can be operable to rapidly capture multiple images of a scene using different exposure settings (e.g., in a manner typical for HDR imaging on a mobile phone camera). When this method is combined with an illustrative image capture interface (such as Figures 2A to 2C the image capture interface shown in), the interface can provide separate control over the short exposure time and the long exposure time for the image sequence. Additionally, with this configuration, data from the long exposure image(s) of the shadow region and the short exposure image(s) of the highlight (or any pixels that may not be classified as being in the shadow region) can be used to construct a live view preview image 201.

[0039] As another example, a camera can include a stacked image sensor, such as a zig-zag sensor. Such a stacked sensor can have long exposure lines and short exposure lines that are diagonally interleaved through a pixel array of the sensor (e.g., in a zig-zag pattern), where the pixels in the "zig" lines capture short exposures and the pixels in the "zag" lines provide longer exposures simultaneously. A zig-zag sensor can also be implemented without a stacked image sensor, as long as it varies the exposure time of different pixels on the sensor. When an illustrative image capture interface (such as Figures 2A to 2C shown in) is paired with the stacked sensor or the zig-zag sensor, the interface can provide separate control over the short exposure and the long exposure. Additionally, with this configuration, data from the long exposure pixels of the shadow region and the short exposure pixels of the highlight (or any pixels that may not be classified as being in the shadow region) can be used to construct a live view preview image 201.

[0040] In some cases, the interface can effectively provide separate pre-image capture control of highlight and shadow exposure levels by allowing separate control of global exposure and range compression (e.g., brightness and contrast respectively). For example, a first control feature can allow adjustment of contrast, while a second control feature can allow adjustment of brightness. In such an implementation, moving the slider of the first control feature up or down can result in adjustment of the long TET and / or short TET to reduce or increase the difference between the long TET and short TET, thereby reducing or increasing contrast respectively. Also, moving the second control feature on the slider can adjust the long TET and short TET in the same direction; for example, by increasing or decreasing the long TET and short TET to increase or decrease the overall brightness of the image or image sequence displayed in the live view interface.

[0041] Returning to the function of the live view interface 200, when the user moves the shadow control 206 up on the slider 202, the brightness of the shadow increases and vice versa. For example, in Figure 2B compared to the position of the shadow control 206 in Figure 2A , the user has moved the shadow control 206 up. As a result, compared to the live view preview image shown in Figure 2A , the face and body of the object in the live view preview image shown in Figure 2B are brighter. Additionally, since the amount of upward movement of the highlight control 204 from Figures 2A to 2B is relatively small compared to the upward movement of the shadow control 206 from Figures 2A to 2B , the change in exposure of the highlight area of the live view preview image between Figure 2A and Figure 2B is much smaller compared to the change in brightness of the shadow area (e.g., the face and body of the object) between Figure 2A and Figure 2B .

[0042] Similarly, when the user moves the shadow control 206 down on the slider 202, this will reduce the brightness of the shadow. For example, in Figure 2C compared to the positions of the shadow control 204 in both Figure 2A and 2B , the user has moved the shadow control 206 down. As a result, compared to the live view preview images shown in Figure 2A and Figure 2B , the face and body of the object in the live view preview image shown in Figure 2C are darker. Additionally, since the amount of downward movement of the shadow control 206 from Figures 2B to 2C is greater than the amount of downward movement of the highlight control 204 from Figures 2B to 2C , the change in brightness of the shadow area (e.g., the face and body of the object) from Figures 2B to 2CCompared to the greater reduction, the exposure of the highlighted area of the live view preview image 201 is reduced less from Figures 2B to 2C than the reduction of

[0043] On the other hand, in order to maintain a certain image quality, the movement of the first control feature 204 and the second control feature 206 on the slider 202 can be partially dependent on each other. In particular, there may be a certain amount of range compression in the image that is considered acceptable, feasible, and / or practical. Thus, the first control feature 204 and the second control feature 206 can move independently of each other on the slider 202 as long as they do not change the highlight exposure and / or the shadow exposure in a way that causes the range compression in the live view preview image 200 to exceed a threshold.

[0044] More specifically, the amount of range compression in the image can be numerically characterized by a "relative shadow gain", which is a comparative measure of the shadow exposure (e.g., long TET) and the highlight (e.g., short TET). For example, the relative shadow gain can be calculated as the ratio of the long TET (taking into account the analog exposure plus any digital gain applied to the shadow area) to the short TET (taking into account the analog exposure plus any digital gain applied to the highlight area); or, in other words, the long TET divided by the short TET. Thus, a maximum value can be defined for the relative shadow gain, which corresponds to an unacceptable amount of range compression. The maximum relative shadow gain can be set such that the range compression does not raise the shadow to the extent of losing an unacceptable amount of detail and / or introducing unacceptable noise. Additionally, a minimum relative shadow gain can be set such that the range compression does not raise the shadow to the extent of underexposing or "squeezing" the shadow to the extent of losing an unacceptable amount of detail.

[0045] To keep the amount of range compression at an acceptable level, there may be situations where the computing device automatically adjusts the highlight exposure level to compensate for a user-specified adjustment to the shadow exposure level, and vice versa. In particular, the difference between the highlight exposure level (linked to the highlight control 204) and the shadow exposure level (linked to the shadow control 206) can be visually represented in the interface 200 by the distance between the shadow control 206 and the highlight control 204 on the slider 202. Thus, the distance between the shadow control 206 and the highlight control 204 can also indicate the relative shadow gain. More specifically, when the shadow control 206 and the highlight control 204 move closer together on the slider 202, this increases the exposure of the shadow (and decreases the exposure of the highlight) compared to the highlight, and thus increases the amount of digital gain applied to the shadow. As a result, as the shadow control 206 and the highlight control 204 move closer together on the slider 202, the range compression of the live view preview image (and subsequently captured images) increases, and vice versa.

[0046] In a scene where a maximum relative shadow gain is specified, the camera application can automatically move the shadow control 206 and / or the highlight control 204 so that they are further apart, in order to keep the relative shadow gain of the live view preview image 201 (e.g., shadow TET divided by highlight TET) below the maximum relative shadow gain.

[0047] For example, consider Figure 2B , where the highlight control 204 is higher on the slider 202 than in Figure 2A . Although the user can manually move the highlight control 204 to the position shown in Figure 2B , in response to the user moving the shadow control 206 upward from its position in Figure 2A to its position in Figure 2B , the camera application may also have automatically moved the highlight control 204 upward. Specifically, in Figure 2B , the distance between the highlight control 204 and the shadow control 206 on the slider 202 can be the minimum separation distance, which corresponds to the maximum allowable amount of range compression (e.g., maximum relative shadow gain).

[0048] Conversely, some implementations of the image capture interface 200 can be configured to maintain a minimum relative shadow gain. For example, a minimum relative shadow gain can be defined such that it corresponds to no range compression (e.g., where the ratio of shadow TET to highlight TET is equal to 1.0). Thus, the minimum relative shadow gain can limit the distance between the highlight control 204 and the shadow control 206 so that the relative shadow gain does not drop below the minimum level. As a result, the camera application can automatically move the shadow control 206 and / or the highlight control 204 so that they are closer together, in order to keep the relative shadow gain of the live view preview 201 at or above the minimum relative shadow gain.

[0049] Regarding the underlying image processing, when the user increases the shadow level to an extent that causes the relative shadow gain to exceed the maximum threshold, the application can set the short TET to be equal to the long TET divided by the maximum relative shadow gain. This implementation helps prevent an undesired amount of range compression while still adjusting the shadow brightness to the extent specified by the user input. Alternatively, when the shadow gain exceeds the maximum threshold, the application can preserve the highlight by not boosting the shadow as much as specified by the user input (or by limiting the amount of increase in the brightness of the shadow area). To this end, when the shadow gain exceeds the maximum threshold, the application can set the long TET to be equal to the product of the short TET and the maximum relative shadow gain.

[0050] On the other hand, the first control feature 204 and the second control feature 206 can maintain image quality, and the movement of the first control feature 204 and the second control feature 206 does not partially depend on each other. For example, the movement of the control features can be mapped to the control of the short TET and / or the long TET in a non-linear manner, such that there is no arrangement of control features with relative shading gain outside the acceptable range. As another example, the application can specify a "dead zone" on the slider, where the user can move the slider, but doing so has no effect. Other examples are also possible.

[0051] In other embodiments of the live view camera interface, two separate slider features can be provided, which allow separate control of the highlight exposure and the shadow exposure. Each slider can include a single control feature. Overall, the two sliders can provide similar functionality as described in the single-slider arrangement.

[0052] Figures 3A to 3C An example of a live view interface 300 with separate controls for shadow control and highlight control is shown. The live view interface 300 includes a first slider 302a having a first control feature 304 and a second slider 302b having a second control feature 306. The first control feature 304 and the second control feature 306 can provide control of the highlight exposure setting (e.g., short TET) and the shadow exposure setting (e.g., long TET) of the live view preview image 301. In most scenarios, the first control feature 304 and the second control feature 306 are independently movable on their respective sliders 302a and 302b, such that the interface 300 allows separate control of the highlight and shadow exposure settings. More generally, except that the first control feature 304 and the second control feature 306 are arranged on separate sliders, the first control feature 304 and the second control feature 306 can act in the same or similar manner as described above with reference to the control features 204 and 206.

[0053] In addition, in the same manner that the distance between the first control feature 204 and the second control feature 206 on the slider 202 corresponds to the relative darkness gain of the live view preview image 201, the vertical distance between the first control feature 304 and the second control feature 306 corresponds to the relative darkness gain of the live view preview image 301. Thus, in some embodiments, the image capture application can be operable to: (i) automatically move the darkness control 306 and / or the highlight control 304 such that they are further apart in the vertical direction in order to keep the relative darkness gain (e.g., darkness TET divided by highlight TET) of the live view preview image 301 below a maximum relative darkness gain, and / or (ii) automatically move the darkness control 306 and / or the highlight control 304 such that they are closer together in the vertical direction in order to keep the relative darkness gain of the live view preview image 301 at or above a minimum relative darkness gain.

[0054] Without departing from the scope of the present invention, other types of interface features can also provide separate real-time adjustment of the darkness and highlight exposure levels. A single knob interface can be provided that has an internally rotatable element and an externally rotatable element that rotate about the same center point and allows separate control of the highlight exposure level and the darkness exposure level. Alternatively, two separate knobs can be provided for separate control of the highlight exposure level and the darkness exposure level. As another example, multiple movable control elements (e.g., dots) can be provided in a single X-Y pad control element that allows separate control of the highlight exposure level and the darkness exposure level. Other examples are possible.

[0055] In another aspect, in addition to or as an alternative to the highlight and darkness exposure levels, the live view interface is capable of implementing the control features described herein for other pairs of related image processing functions. For example, the first control feature and the second control feature disclosed herein can additionally or alternatively provide real-time pre-image capture control of black level and white level, global exposure and contrast, and other possible pairs.

[0056] In another aspect, the live view interface is capable of providing a mechanism for a user to specify control points in the live view preview image. A control point is a specific location in the image that specifies where a localized effect or process is to be applied. Generally, an effect is applied in a local area around the control point and the effect is gradually reduced to blend the affected area of the image with the surrounding area. In such an embodiment, the control features described herein can be used to provide, for example, separate highlight and darkness exposure control for a specific control point or group of control points.

[0057] As described above, the two control features of the example interface can change the live view preview by separately changing the short TET and the long TET of the live view preview (with exceptions that may be due to the limitations of the relative dark gain as described above). Return reference Figures 2A to 2C , moving the control feature 204 upward along the slider generally increases the short TET of the live view preview, and vice versa. Similarly, moving the control feature 206 upward along the slider generally increases the long TET of the live view preview, and vice versa.

[0058] The two control features can also be mapped to other types of image adjustments, and / or can control the short TET and / or the long TET in different ways. For example, the first control feature and the second control feature (such as Figures 2A to 3C those shown in ) can provide separate control over (a) brightness or exposure value compensation (EVC) and (b) relative dark gain. This can be achieved by mapping the first control feature (e.g., control feature 204 or control feature 304) to both the short TET and the long TET (so it affects the brightness of the entire live view preview) and mapping the second control feature (e.g., control feature 206 or control feature 306) only to the long TET. In this case, moving the control feature 204 upward along the slider will generally increase the brightness or EVC of the live view preview (by increasing both the short TET and the long TET), and vice versa. And, by increasing the long TET of the live view preview, moving the control feature 206 upward along the slider will only change the combined exposure of the dark areas, and vice versa. In such an implementation, the image capture application can also automatically adjust one or both control features in a similar manner as described above to keep the relative dark gain below a threshold.

[0059] As another example, the first control feature and the second control feature (such as Figures 2A to 3C those shown in ) can provide separate control over (a) brightness or EVC and (b) contrast. This can be achieved by mapping the first control feature (e.g., control feature 204 or control feature 304) to both the short TET and the long TET (so it affects the brightness of the entire live view preview), and mapping the second control feature (e.g., control feature 206 or control feature 306) to adjust the short TET and the long TET in opposite directions.

[0060] Both knobs affect both TETs. Brightness moves them in the same direction; Contrast moves them in opposite directions. We decided not to pursue this because users generally want to adjust the shadow levels in isolation. In this case, moving the control feature 204 up along the slider will generally increase the brightness of the live view preview or the EVC (by increasing both the short TET and the long TET), and vice versa. Also, moving the control feature 206 up along the slider will adjust the contrast in the live view preview by increasing the short TET and decreasing the long TET simultaneously, and vice versa for moving down. In such an implementation, the image capture application can also automatically adjust one or both control features in a manner similar to the above to keep the relative shadow gain below a threshold.

[0061] Other mappings of the control features to different types of image adjustment processes are also possible. Additionally, note that, without departing from the scope of the present invention, in such an embodiment and in any other embodiment described herein, the effect of the up / down movement of the control features can be reversed.

[0062] III. Real-time adjustment of the live view preview image

[0063] As described above, the live view preview image can be a video feed generated by real-time processing of data from an image sensor. This can be achieved in various ways. In some cases, the initial settings of the live view preview image, such as the exposure level (e.g., short TET and / or long TET), can be determined by applying a trained neural network (e.g., a convolutional neural network or "CNN") to frames from the image sensor.

[0064] In such an embodiment, the CNN can provide a grid of image processing operators used for local tone mapping. This local tone mapping grid can indicate different gain levels for different regions in the image frame. In this case, the control features for adjusting the shadow level and / or the highlight level can be used to adjust the values of the coefficients for the gain applied at each point in the grid, and the adjusted tone mapping grid can be applied to generate the live view preview image.

[0065] Alternatively, the imaging application can use the data from the image sensor to generate two low-resolution synthetic exposures for the short TET and the long TET. Then the trained CNN can be applied to these synthetic exposures to blend portions from both and output a tone mapping grid that differently exposes the shadow regions and the highlight regions to output the live view preview image. In this case, the control features for adjusting the shadow level and / or the highlight level can be used to adjust the way these two low-resolution synthetic exposures are blended, such that the adjustment is used to update the live view preview image in real time.

[0066] Figure 4A Shows a CNN 400 according to an example. The CNN is designed to take advantage of the inherent structure found in most images. In particular, the nodes in the CNN are only connected to a small number of nodes in the previous layer. This CNN architecture can be considered three-dimensional, where the nodes are arranged in blocks having width, height, and depth. For example, the aforementioned 32×32 pixel block with 3 color channels can be arranged into an input layer having a width of 32 nodes, a height of 32 nodes, and a depth of 3 nodes.

[0067] More specifically, in the CNN 400, the initial input values 402 represented as pixels X1…X m are provided to the input layer 404. As described above, the input layer 404 can have three dimensions based on the width, height, and number of color channels of pixels X1…X m The input layer 404 provides values into one or more sets of feature extraction layers, each set containing instances of a convolutional layer 406, a rectified linear unit function (RELU) layer 408, and a pooling layer 410. The output of the pooling layer 410 is provided to one or more classification layers 412. The final output value 414 can be arranged in a feature vector representing a concise characterization of the initial input value 402.

[0068] The convolutional layer 406 can transform its input values by sliding one or more filters around a three-dimensional space of these input values. The filters are represented by biases applied to the nodes and weights of the connections therebetween, and typically have a width and height smaller than the input values. The result of each filter can be a two-dimensional block of output values (referred to as a feature map or grid), where the width and height can have the same size as the width and height of the input values, or one or more of these dimensions can have different sizes. The combination of the outputs of each filter produces a layer of feature maps in the depth dimension, where each layer represents the output of one of the filters. Such a feature map or grid can provide local tone mapping for the output image.

[0069] Applying a filter can involve computing the dot product sum between the entries in the filter and a two-dimensional depth slice of the input values. An example of this situation is shown in Figure 4B . Matrix 420 represents the input to the convolutional layer and can thus be, for example, image data. The convolution operation superimposes the filter 422 on the matrix 420 to determine the output 424. For example, when the filter 422 is located at the upper left corner of the matrix 420 and the dot product sum of each entry is computed, the result is 4. This is placed at the upper left corner of the output 424.

[0070] Back to Figure 4A, During training, the CNN learns filters such that these filters can eventually identify some types of features at specific locations in the input values. As an example, the convolutional layer 406 can include filters that can eventually detect edges and / or colors in image patches from which the initial input values 402 are derived. A hyperparameter called the receptive field determines the number of connections between each node in the convolutional layer 406 and the input layer 404. This allows each node to focus on a subset of the input values.

[0071] The RELU layer 408 applies an activation function to the output provided by the convolutional layer 406. The RELU function can be a simple threshold function defined as f(x) = max(0, x). Thus, the output is 0 when x is negative and x when x is non - negative. A smooth, differentiable approximation of the RELU function is the softplus function. It is defined as f(x) = log(1 + e x ). Nevertheless, other functions can be used in this layer.

[0072] The pooling layer 410 reduces the spatial size of the data by downsampling each two - dimensional depth slice of the output from the RELU layer 408. One possible method is to apply a 2×2 filter with a stride of 2 to each 2×2 block of the depth slice. This will reduce the width and height of each depth slice by 1 / 2, thus reducing the overall size of the data by 75%.

[0073] The classification layer 412 computes the final output value 414 in the form of a feature vector. As an example, in a CNN trained as an image classifier, each entry in the feature vector can encode the probability that the image patch contains an item of a specific class (e.g., face, cat, beach, tree, etc.).

[0074] In some embodiments, there are multiple sets of feature extraction layers. Thus, an instance of the pooling layer 410 can provide output to an instance of the convolutional layer 406. Additionally, for each instance of the pooling layer 410, there can be multiple instances of the convolutional layer 406 and the RELU layer 408.

[0075] The CNN 400 represents a general structure that can be used in image processing. The convolutional layer 406 and the classification layer 412 apply weights and biases similar to the layers in the ANN 300, and these weights and biases can be updated during backpropagation so that the CNN 400 can learn. On the other hand, the RELU layer 408 and the pooling layer 410 typically apply fixed operations and thus may not be able to learn.

[0076] It should be understood that a CNN can include a different number of layers than shown in the examples herein, and each of these layers can include a different number of nodes. Thus, the CNN 400 is for illustrative purposes only and should not be considered as limiting the structure of a CNN.

[0077] In addition, it should be understood that other methods for generating and updating the live view preview are feasible without departing from the scope of the present invention.

[0078] IV. Exemplary Computing Device

[0079] Figure 5 is a block diagram of an exemplary computing device 500 according to an exemplary embodiment. In particular, Figure 5 the illustrated computing device 500 can be configured to provide various imaging functions described herein, such as image capture and / or providing a live view interface (e.g., viewfinder function). The computing device 500 can take various forms, including but not limited to mobile phones, standalone cameras (e.g., DSLR, point-and-shoot cameras, camcorders, or movie cameras), tablet computers, laptop computers, desktop computers, server systems, cloud-based computing devices, wearable computing devices, or network-connected household appliances or consumer electronic devices, etc.

[0080] The computing device 500 can include a user interface module 501, a network communication module 502, one or more processors 503, a data storage device 504, one or more cameras 518, one or more sensors 520, and a power system 522, all of which can be linked together via a system bus, network, or other connection mechanism 505.

[0081] The user interface module 501 is operable to send data to and / or receive data from external user input / output devices. For example, the user interface module 501 can be configured to send data to and / or receive data from user input devices such as touchscreens, computer mice, keyboards, keypads, touchpads, trackballs, joysticks, voice recognition modules, and / or other similar devices. The user interface module 501 can also be configured to provide output to user display devices, such as one or more cathode ray tubes (CRTs), liquid crystal displays, light-emitting diodes (LEDs), displays using digital light processing (DLP) technology, printers, light bulbs, and / or other similar devices known now or developed later. The user interface module 501 can also be configured to generate audible output using devices such as speakers, speaker jacks, audio output ports, audio output devices, headphones, and / or other similar devices. The user interface module 501 can also be configured with one or more haptic devices that can generate haptic output, such as vibrations and / or other outputs detectable by touch and / or physical contact with the computing device 500. In some examples, the user interface module 501 can be used to provide a graphical user interface (GUI) for utilizing the computing device 500.

[0082] The network communication module 502 can include one or more devices providing one or more wireless interfaces 507 and / or one or more wired interfaces 508 configurable to communicate via a network. The wireless interface(s) 507 can include one or more wireless transmitters, receivers, and / or transceivers, such as Bluetooth TM transceivers, transceivers, Wi-Fi TM transceivers, WiMAX TM transceivers and / or other similar types of wireless transceivers configurable to communicate via a wireless network. The wired interface(s) 508 can include one or more wired transmitters, receivers, and / or transceivers, such as Ethernet transceivers, universal serial bus (USB) transceivers, or similar transceivers configurable to communicate via twisted pair, coaxial cable, fiber optic link, or similar physical connections to a wired network.

[0083] In some examples, the network communication module 502 can be configured to provide reliable, secure, and / or authenticated communication. For each communication described herein, information can be provided to facilitate reliable communication (e.g., guaranteed message delivery), possibly as part of a message header and / or footer (e.g., packet / message sequencing information, encapsulation headers and / or footers, size / time information, and transmission verification information such as cyclic redundancy check (CRC) and / or parity values). One or more cryptographic protocols and / or algorithms (such as, but not limited to, Data Encryption Standard (DES), Advanced Encryption Standard (AES), Rivest-Shamir-Adelman (RSA) algorithm, Diffie-Hellman algorithm, secure socket protocols (such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS)), and / or Digital Signature Algorithm (DSA)) can be used to secure (e.g., encode or encrypt) and / or decrypt / decode the communication. Other cryptographic protocols and / or algorithms can also be used, or in addition to those listed herein, other cryptographic protocols and / or algorithms can be used to protect (and then decrypt / decode) the communication.

[0084] The one or more processors 503 can include one or more general-purpose processors and / or one or more special-purpose processors (e.g., digital signal processors, tensor processing units (TPU), graphics processing units (GPU), application specific integrated circuits, etc.). The one or more processors 503 can be configured to execute computer-readable instructions 506 contained in the data storage device 504 and / or other instructions as described herein.

[0085] The data storage device 504 can include one or more non-transitory computer-readable storage media that can be read and / or accessed by at least one of the one or more processors 503. The one or more computer-readable storage media can include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage, which can be integrated, in whole or in part, with at least one of the one or more processors 503. In some examples, the data storage device 504 can be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage unit), while in other examples, the data storage device 504 can be implemented using two or more physical devices.

[0086] The data storage device 504 can include computer-readable instructions 506 and possibly additional data. In some examples, the data storage device 504 can include storage required to execute at least a portion of the methods, scenarios, and techniques described herein and / or at least a portion of the functionality of the devices and networks described herein. In some examples, the data storage device 504 can include storage for a trained neural network model 512 (e.g., a model of a trained convolutional neural network such as a convolutional neural network). In particular, in these examples, the computer-readable instructions 506 can include instructions that, when executed by the processor 503, cause the computing device 500 to provide some or all of the functionality of the trained neural network model 512.

[0087] In some examples, the computing device 500 can include one or more cameras 518. The camera(s) 518 can include one or more image capture devices, such as a static camera and / or a video camera, which are configured to capture light and record the captured light in one or more images; that is, the camera(s) 518 can generate an image(s) of the captured light. The one or more images can be one or more static images and / or one or more images used in a video stream. The camera(s) 518 can capture light and / or electromagnetic radiation emitted as visible light, infrared radiation, ultraviolet light, and / or as light at one or more other frequencies.

[0088] In some examples, computing device 500 can include one or more sensors 520. The sensors 520 can be configured to measure conditions within computing device 500 and / or conditions in the environment of computing device 500 and provide data regarding such conditions. For example, sensors 520 can include one or more of the following: (i) sensors for obtaining data regarding computing device 500, such as but not limited to a thermometer for measuring the temperature of computing device 500, a battery sensor for measuring the power of one or more batteries of power system 522, and / or other sensors for measuring the conditions of computing device 500; (ii) identification sensors for identifying other objects and / or devices, such as but not limited to radio frequency identification (RFID) readers, proximity sensors, one-dimensional barcode readers, two-dimensional barcode (e.g., Quick Response (QR) code) readers, and laser trackers, where the identification sensors can be configured to read identifiers, such as RFID tags, barcodes, QR codes, and / or other devices and / or objects configured to be read and provide at least identification information; (iii) sensors for measuring the position and / or movement of computing device 500, such as but not limited to tilt sensors, gyroscopes, accelerometers, Doppler sensors, GPS devices, sonar sensors, radar devices, laser displacement sensors, and compasses; (iv) environmental sensors for obtaining data indicative of the environment of computing device 500, such as but not limited to infrared sensors, optical sensors, light sensors, biosensors, capacitance sensors, touch sensors, temperature sensors, wireless sensors, radio sensors, motion sensors, microphones, sound sensors, ultrasonic sensors, and / or smoke sensors; and / or (v) force sensors for measuring one or more forces acting around computing device 500 (e.g., inertial forces and / or gravity), such as but not limited to one or more sensors for measuring one or more of the following: force in one or more dimensions, torque, ground force, friction, and / or a zero moment point (ZMP) sensor for identifying the ZMP and / or the position of the ZMP. Many other examples of sensors 520 are also feasible.

[0089] The power system 522 can include one or more batteries 524 and / or one or more external power interfaces 526 for providing power to the computing device 500. When electrically coupled to the computing device 500, each of the one or more batteries 524 can serve as a source for storing power for the computing device 500. One or more batteries 524 of the power system 522 can be configured to be portable. Some or all of the one or more batteries 524 can be easily removed from the computing device 500. In other examples, some or all of the one or more batteries 524 can be inside the computing device 500 and thus may not be easily removable from the computing device 500. Some or all of the one or more batteries 524 can be rechargeable. For example, a rechargeable battery can be recharged via a wired connection between the battery and another power source, such as by one or more power sources that are external to the computing device 500 and connected to the computing device 500 via one or more external power interfaces. In other examples, some or all of the one or more batteries 524 can be non-rechargeable batteries.

[0090] One or more external power interfaces 526 of the power system 522 can include one or more wired power interfaces, such as a USB cable and / or a power cord, which enable a wired power connection to one or more power sources external to the computing device 500. One or more external power interfaces 526 can include one or more wireless power interfaces, such as a Qi wireless charger, which enables a radio power connection to one or more external power sources, such as via the Qi wireless charger. Once an electrical power connection to an external power source is established using one or more external power interfaces 526, the computing device 500 can draw power from the external power source through the established electrical power connection. In some examples, the power system 522 can include associated sensors, such as battery sensors associated with one or more batteries or other types of electrical power sensors.

[0091] V. Cloud-based Server

[0092] Figure 6 A network 606 of computing clusters 609a, 609b, 609c arranged as a cloud-based server system according to an example embodiment is depicted. The computing clusters 609a, 609b, 609c can be cloud-based devices that store program logic and / or data for cloud-based applications and / or services; for example, performing at least one function of a convolutional neural network, confidence learning, predicting target images, predicting an original illumination model, a convolutional neural network, confidence learning, and / or method 300 and / or at least one function related to a convolutional neural network, confidence learning, predicting an original illumination model, a convolutional neural network, confidence learning, and / or method 300.

[0093] In some embodiments, the computing clusters 609a, 609b, 609c can be a single computing device residing in a single computing center. In other embodiments, the computing clusters 609a, 609b, 609c can include multiple computing devices in a single computing center, or even multiple computing devices in multiple computing centers located at different geographical locations. For example, Figure 6 Each of the computing clusters 609a, 609b, and 609c residing at different physical locations is depicted.

[0094] In some embodiments, the data and services at the computing clusters 609a, 609b, 609c can be encoded as computer-readable information stored in a non-transitory tangible computer-readable medium (or computer-readable storage medium) and accessible by other computing devices. In some embodiments, the computing clusters 609a, 609b, 609c can be stored on a single disk drive or other tangible storage medium, or can be implemented on multiple disk drives or other tangible storage mediums located at one or more different geographical locations.

[0095] Figure 6 A cloud-based server system according to an example embodiment is depicted. In Figure 6 this, the functions of the convolutional neural network, confidence learning, and / or the computing devices can be distributed among the computing clusters 609a, 609b, 609c. The computing cluster 609a can include one or more computing devices 600a, a cluster storage array 610a, and a cluster router 611a connected via a local cluster network 612a. Similarly, the computing cluster 609b can include one or more computing devices 600b, a cluster storage array 610b, and a cluster router 611b connected via a local cluster network 612b. Likewise, the computing cluster 609c can include one or more computing devices 600c, a cluster storage array 610c, and a cluster router 611c connected via a local cluster network 612c.

[0096] In some embodiments, each of the computing clusters 609a, 609b, and 609c can have an equal number of computing devices, an equal number of cluster storage arrays, and an equal number of cluster routers. However, in other embodiments, each computing cluster can have a different number of computing devices, a different number of cluster storage arrays, and a different number of cluster routers. The number of computing devices, cluster storage arrays, and cluster routers in each computing cluster can depend on one or more computing tasks assigned to each computing cluster.

[0097] In the computing cluster 609a, for example, the computing device 600a can be configured to execute convolutional neural networks, confidence learning, and / or various computing tasks of the computing device. In one embodiment, the convolutional neural networks, confidence learning, and / or various functions of the computing device can be distributed among one or more of the computing devices 600a, 600b, 600c. The computing devices 600b and 600c in the respective computing clusters 609b and 609c can be configured similarly to the computing device 600a in the computing cluster 609a. On the other hand, in some embodiments, the computing devices 600A, 600B, and 600C can be configured to perform different functions.

[0098] In some embodiments, the computing tasks and stored data associated with the convolutional neural networks, confidence learning, and / or the computing device can be distributed among the computing devices 600a, 600b, 600c at least in part based on the processing requirements of the convolutional neural networks, confidence learning, and / or the computing device, the processing capabilities of the computing devices 600a, 600b, 600c, the latency of the network links between the computing devices within each computing cluster and between the computing clusters themselves, and / or other factors that can contribute to the cost, speed, fault tolerance, resilience, efficiency, and / or other design goals of the overall system architecture.

[0099] The cluster storage arrays 610a, 610b, 610c of the computing clusters 609a, 609b, 609c can be data storage arrays including a disk array controller configured to manage read and write access to a group of hard disk drives. The disk array controller, alone or in combination with its respective computing devices, can also be configured to manage backup or redundant copies of the data stored in the cluster storage array to prevent disk drive or other cluster storage array failures and / or network failures that prevent one or more computing devices from accessing one or more cluster storage arrays.

[0100] Similar to the way in which the functions of the convolutional neural networks, confidence learning, and / or the computing device can be distributed across the computing devices 600a, 600b, 600c of the computing clusters 609a, 609b, 609c, the various active parts and / or backup parts of these components can be distributed across the cluster storage arrays 610a, 610b, 610c. For example, some cluster storage arrays can be configured to store a portion of the data of the convolutional neural networks, confidence learning, and / or the computing device, while other cluster storage arrays can store other portions of the data of the convolutional neural networks, confidence learning, and / or the computing device. Additionally, some cluster storage arrays can be configured to store backup versions of the data stored in other cluster storage arrays.

[0101] The cluster routers 611a, 611b, 611c in the compute clusters 609a, 609b, 609c can include networking devices configured to provide internal and external communication for the compute clusters. For example, the cluster router 611a in the compute cluster 609a can include one or more Internet switching and routing devices configured to (i) provide local area network communication between the compute devices 600a and the cluster storage array 610a via the local cluster network 612a, and (ii) provide wide area network communication between the compute cluster 609a and the compute clusters 609b and 609c via the wide area network link 613a to the network 606. The cluster routers 611b and 611c can include network devices similar to the cluster router 611a, and the cluster routers 611b and 611c can perform networking functions similar to the networking functions performed by the cluster router 611a for the compute cluster 609a for the compute clusters 609b and 609b.

[0102] In some embodiments, the configuration of the cluster routers 611a, 611b, 611c can be based at least in part on the data communication requirements of the compute devices and the cluster storage arrays, the data communication capabilities of the network devices in the cluster routers 611a, 611b, 611c, the latency and throughput of the local cluster networks 612a, 612b, 612c, the latency, throughput, and cost of the wide area network links 613a, 613b, 613c, and / or other factors that can contribute to adjusting the cost, speed, fault tolerance, resiliency, efficiency, and / or other design criteria of the system architecture.

[0103] VII. Conclusion

[0104] The present disclosure is not limited to the specific embodiments described in this application, which are intended to be illustrative of various aspects. Many modifications and variations can be made without departing from its spirit and scope, which will be apparent to those skilled in the art. From the foregoing description, functional equivalent methods and apparatuses within the scope of the present disclosure will be apparent to those skilled in the art other than those enumerated herein. These modifications and variations are intended to fall within the scope of the appended claims.

[0105] The foregoing detailed description has described various features and functions of the disclosed systems, apparatuses, and methods with reference to the accompanying drawings. In the drawings, like reference numerals generally identify like components unless the context dictates otherwise. The illustrative embodiments described in the detailed description, the drawings, and the claims are not meant to be limiting. Other embodiments can be utilized and other changes can be made without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure as generally described herein and illustrated in the drawings can be arranged, substituted, combined, separated, and designed in a variety of different configurations, all of which are explicitly contemplated herein.

[0106] Regarding any or all of the ladder diagrams, scenarios, and flowcharts as discussed herein in the figures, each block and / or communication can represent the processing of information and / or the transmission of information according to an example embodiment. Alternative embodiments are included within the scope of these example embodiments. In these alternative embodiments, for example, functions described as blocks, transmissions, communications, requests, responses, and / or messages can be performed in a different order than shown or discussed, including substantially simultaneously or in the reverse order, depending on the functions involved. Additionally, more or fewer blocks and / or functions can be used with any of the ladder diagrams, scenarios, and processes discussed herein, and these ladder diagrams, scenarios, and flowcharts can be combined partially or fully with each other. Figure 1 and these ladder diagrams, scenarios, and flowcharts can be used starting from, and these ladder diagrams, scenarios, and flowcharts can be combined partially or fully with each other.

[0107] Blocks representing information processing can correspond to circuits capable of being configured to perform specific logical functions of the methods or techniques described herein. Alternatively or additionally, blocks representing information processing can correspond to a module, segment, or portion of program code (including associated data). The program code can include one or more instructions executable by a processor to implement a specific logical function or action in a method or technique. The program code and / or associated data can be stored on any type of computer-readable medium, such as a storage device including a magnetic disk or hard drive or other storage medium.

[0108] The computer-readable medium can also include non-transitory computer-readable media, such as non-transitory computer-readable media that store data for a short period of time, such as register memory, processor cache, and random access memory (RAM). The computer-readable medium can also include non-transitory computer-readable media that store program code and / or data for a longer period of time, such as secondary or persistent long-term storage devices, such as read-only memory (ROM), optical disks or magnetic disks, compact disc read-only memory (CD-ROM). The computer-readable medium can also be any other volatile or non-volatile storage system. The computer-readable medium can be considered, for example, a computer-readable storage medium or a tangible storage device.

[0109] Furthermore, blocks representing one or more information transmissions can correspond to information transmissions between software and / or hardware modules within the same physical device. However, other information transmissions can occur between software modules and / or hardware modules in different physical devices.

[0110] Although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are provided for purposes of explanation and not limitation, and the true scope is indicated by the appended claims.

Claims

1. A method for dual-exposure control in a camera system, comprising: Display a graphical interface for image capture on a display device, the graphical interface including: (a) a live view preview image based on an image data stream from an image sensor of an image capture device, and (b) a first control feature and a second control feature; Receive first input data corresponding to a first interaction with the first control feature via the graphical interface, and adjust a first exposure setting responsively based on the first interaction; Apply the first exposure setting in real time to at least one highlight region in the live view preview image; Receive second input data corresponding to a second interaction with the second control feature via the graphical interface, and adjust a second exposure setting responsively based on the second interaction; Apply the second exposure setting in real time to at least one shadow region in the live view preview image; and Subsequently receive third input data corresponding to an image capture command, and operate the image capture device responsively to capture an image according to both the first exposure setting and the second exposure setting, wherein applying the first exposure setting includes adjusting a short total exposure time and a long total exposure time of the live view preview, and wherein the adjustment of the long total exposure time and the short total exposure time is performed in the same direction; and wherein applying the second exposure setting includes adjusting the long total exposure time of the live view preview.

2. The method according to claim 1, wherein, The first control feature and the second control feature are displayed as part of a single control feature in the graphical interface.

3. The method according to claim 2, wherein, The single control feature includes a slider, and wherein the first control feature and the second control feature are movable along the slider.

4. The method according to claim 3, wherein, The distance along the slider between the first control feature and the second control feature corresponds to the range compression of the live view preview image.

5. The method according to claim 2, wherein, The single control feature includes a knob interface, wherein the first control feature includes an internally rotatable element of the knob interface, wherein the second control feature includes an externally rotatable element of the knob interface, and wherein both the internally rotatable element and the externally rotatable element rotate around the center point of the knob interface.

6. The method according to claim 2, wherein, The single control feature includes an X-Y pad, and wherein the first control feature and the second control feature are movable within the X-Y pad.

7. The method according to claim 1, wherein, The first control feature includes a first slider, and the second control feature includes a second slider.

8. The method according to claim 7, wherein, The first slider is operable to adjust a highlight exposure setting used by the image capture device to generate the image data stream, and wherein the second slider is operable to adjust a shadow exposure setting used by the image capture device to generate the image data stream.

9. The method according to claim 1, wherein, The first exposure setting includes a highlight exposure setting used by the image capture device to generate the image data stream.

10. The method according to claim 1, wherein, The second exposure setting includes a shadow exposure setting.

11. The method according to claim 10, wherein, The shadow exposure setting affects the amount of gain to be applied to one or more shadow regions of the image data stream from the image capture device.

12. The method according to claim 1, further comprising: Apply the second exposure setting to the image data stream to generate the live view preview image.

13. The method according to claim 1, wherein, The live view image data provides a what-you-see-is-what-you-get (WYSIWYG) viewfinder for the image capture device.

14. The method according to claim 1, wherein, The graphical interface further includes an image capture feature for causing the image capture device to capture image data.

15. A computing device, comprising: One or more processors; And One or more computer-readable media having computer-readable instructions stored thereon that, when executed by the one or more processors, cause the mobile computing device to perform functions including the method according to any one of claims 1 to 14.

16. A computing device, comprising: An apparatus for performing the method according to any one of claims 1 to 14.

17. An article configured for dual-exposure control in a camera system, comprising one or more computer-readable media having computer-readable instructions stored thereon, the computer-readable instructions, when executed by one or more processors of a computing device, cause the computing device to perform functions including the method according to any one of claims 1 to 14.

18. The article according to claim 17, wherein, The one or more computer-readable media include one or more non-transitory computer-readable media.

19. A method for dual-exposure control in a camera system, comprising: Displaying on a display device a graphical user interface for image capture, the graphical user interface including: (a) a live view preview based on an image data stream from an image sensor of an image capture device, and (b) a first control feature and a second control feature; Receiving, via the graphical user interface, first input data corresponding to a first interaction with the first control feature and responsively adjusting a first image setting based on the first interaction; Applying the first image setting in real time to at least one highlighted region in the live view preview; Receiving, via the graphical user interface, second input data corresponding to a second interaction with the second control feature and responsively adjusting a second image setting based on the second interaction; Applying the second image setting in real time to at least one shaded region in the live view preview image; and Subsequently receiving third input data corresponding to an image capture command and responsively operating the image capture device to capture an image according to both the first image setting and the second image setting, wherein applying the first image setting includes adjusting a short total exposure time and a long total exposure time of the live view preview, and wherein the adjustment of the long total exposure time and the short total exposure time is performed in the same direction; and wherein applying the second image setting includes adjusting the long total exposure time of the live view preview.

20. The method according to claim 19, wherein, The first control feature provides adjustment of the highlighted region in the live view preview, and wherein the second control feature provides adjustment of the shaded region in the live view preview.

21. The method according to claim 19, wherein, The first control feature provides adjustment of the brightness of the live view preview, and wherein the second control feature provides shaded adjustment in the live view preview.

22. The method according to claim 19, wherein, The first control feature provides adjustment of the brightness of the live view preview, and wherein the second control feature provides adjustment of the contrast in the live view preview.

Images