User interface for camera focus
The system addresses focus ambiguity in camera systems by providing alternative views and manual controls to ensure precise focus adjustments, especially in regions with mixed depth levels, improving focus accuracy.
Patent Information
- Application Number
- JP2024525448
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-11
- Filing Date
- 2022-08-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-08-30
AI Technical Summary
Existing camera systems struggle to accurately determine the focus depth when image content within a user-selected region contains multiple depths, leading to ambiguity in focus settings, especially on smaller screens where user inputs cover large areas with varying depth levels.
The system determines whether image content within a user-selected region exists at different depths and outputs alternative versions, such as zoomed-in or blurred views, allowing users to make precise focus adjustments through subsequent selections, including manual controls and histograms for depth guidance.
Enhances focus accuracy by enabling users to clearly identify and select the appropriate depth of focus, even in regions with varying depth levels, without requiring additional user inputs when content is at a single depth.
Smart Images

Figure 2024543004000001_ABST
Abstract
Description
[Technical field]
[0001] This application claims priority to U.S. Patent Application No. 17 / 454,547, filed November 11, 2021, the entire contents of which are incorporated herein by reference.
[0002] The present disclosure relates to image capture or processing. [Background technology]
[0003] A camera device includes one or more cameras that capture frames (e.g., images). Examples of camera devices include standalone digital cameras or digital video camcorders, camera-equipped wireless communication device handsets such as mobile phones having one or more cameras, cellular or satellite radio telephones, camera-equipped personal digital assistants (PDAs), computing panels or tablets, gaming devices, computing devices that include a camera such as so-called "webcams," smart watches, devices with their own cameras, devices configured to control another device with a camera, or any device with digital imaging or video capabilities.
[0004] The camera device processes the captured frames and outputs the frames for display, hi some embodiments, the camera device controls exposure, focus, and white balance to capture high quality images. Summary of the Invention
[0005] In general, this disclosure describes techniques for controlling a camera focus level (e.g., a position of a camera lens to focus at a desired depth), including techniques for generating a user interface that a user interacts with to control the camera focus level. As described in more detail, in response to a user selection of a region to focus on within a preview image, one or more processors may determine whether image content exists at different depths within the region. If image content exists at different depths within the region, the one or more processors may output an alternative version of the image content within the region (e.g., a zoomed-in version) and receive a subsequent user selection within the alternative version. The one or more processors may control the camera focus level (e.g., a camera lens position) based on the subsequent user selection. However, if image content exists within the region at a single depth (e.g., no image content at different depths), the one or more processors may not output an alternative version of the image content within the region and may control a focus level for the camera based on the user selection (i.e., no subsequent user selection is required).
[0006] In some cases, a user selection of a region for controlling the focus level of the camera may cover a relatively large area. For example, a user may touch a screen presenting a preview image to define the region. On a relatively small screen, a touch on the screen may encompass a relatively large amount of image content (e.g., the area is proportional to the ratio between the size of the screen and the user-perceived size of the fingertip, which may be how the user touched the screen). In such cases, it may be undefined where exactly the focus of the camera should be, especially if image content exists at different depths. That is, the focus depth may be unknown because image content exists at different depths within the user-selected region. By zooming in on the image content of the region where image content at different depths exists, the user may be able to better select where the camera should focus because the image content of the region appears much larger on the screen. However, if image content exists at a single depth (e.g., no image content exists at different depths), the focus can be adjusted (e.g., by changing the lens position) to the depth of the image content within the user-selected region without having to zoom in on the image content of the region. In this manner, the exemplary techniques may improve the operation of controlling focus levels on a camera with practical applications to selectively provide alternative versions for better selection of focus depth when image content is present at different depths.
[0007] In one example, a device for image capture includes a memory configured to store a preview image and one or more processors coupled to the memory, where the one or more processors are configured to receive a user selection to focus on a region of the preview image stored in the memory, determine whether image content is present at different depths within the region, output an alternative version of the image content within the region based on a determination that image content is present at different depths within the region, receive a subsequent user selection within the alternative version, and control a focus level of a camera based on the subsequent user selection.
[0008] In one example, a method for image capture includes receiving a user selection to focus on a region of a preview image stored in a memory, determining that image content is present at different depths within the region, outputting an alternative version of the image content within the region based on the determination that image content is present at different depths within the region, receiving a subsequent user selection within the alternative version, and controlling a focus level of a camera based on the subsequent user selection.
[0009] In one example, a computer-readable storage medium storing instructions that, when executed, cause one or more processors to receive a user selection to focus on a region of a preview image stored in memory, determine that image content is present at different depths within the region, output an alternative version of the image content within the region based on the determination that image content is present at different depths within the region, receive a subsequent user selection within the alternative version, and control a focus level of a camera based on the subsequent user selection.
[0010] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a block diagram of a device configured to perform one or more of the embodiments of the techniques described in this disclosure. [Diagram 2] 2 is a block diagram illustrating the central processing unit (CPU) and system memory of the device of FIG. 1 in further detail. [Diagram 3] FIG. 1 is a conceptual diagram illustrating an example of a device for displaying image content and user selection of regions. [Figure 4A] FIG. 13 is a conceptual diagram showing focus settings at different depths. [Figure 4B] FIG. 13 is a conceptual diagram showing focus settings at different depths. [Diagram 5] FIG. 11 is a conceptual diagram illustrating an example of a user interface for a touch focus assist mode. [Figure 6] FIG. 13 is a conceptual diagram illustrating an example of a user interface for a manual focus assist mode. [Figure 7A] FIG. 13 is a conceptual diagram illustrating an example of a user interface to which a blur is applied. [Figure 7B] FIG. 13 is a conceptual diagram illustrating an example of a user interface to which a blur is applied. [Figure 8] 10 is a flowchart illustrating one or more example techniques of operation. [Figure 9] 10 is a flowchart illustrating one or more example techniques of operation. [Figure 10] 10 is a flowchart illustrating one or more example techniques of operation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Example techniques described in this disclosure relate to camera focus (e.g., the depth at which a camera should focus for image or video capture). In some camera focus techniques, a user selects an area to focus on. For example, during a preview mode, a screen displays the image content to be captured. The preview image may include the image content. The user may touch on the screen in an area of the preview image at which the camera should focus. However, in some implementations, there may be image content at different depths within the user-selected area (e.g., where the user touched). In such cases, it may be unclear at what depth the focus should be set (e.g., it may be unclear what the focus level is, and what the focus level should be adjusted to).
[0013] In one or more embodiments, in response to a user selection of a region of the preview image to focus on, the one or more processors may determine whether image content is present at multiple depths within the selected region. If image content is present at multiple depths, the one or more processors may implement a focus assist process to allow the user to more accurately select the focus depth and image content to focus on. However, if image content is present at only one depth, the one or more processors may control the focus level such that the focus is at the depth and region of the selected region.
[0014] When image content exists at multiple depths, in some implementations, the screen may present an alternative version of the image content. As an example, the alternative version of the image content may be a zoomed-in image content of a selected area, such that the selected area appears larger. In this alternative version of the image content, the user may again select an area to set the depth. For example, in the zoomed-in version of the image content, the user may again select an area to set the depth, giving the user a more precise way of selecting the focus depth.
[0015] As another example, after presenting an alternative version of the image content, a user may manually adjust the focus of a camera until the desired image content is in focus. In some examples, to manually adjust the focus level (e.g., the depth at which the camera should focus), the screen may display a histogram indicating the number of pixels that are farther or closer than the focus depth. The user may utilize the histogram to select whether they want to focus on image content that is closer to the focus or farther from the focus. However, the exemplary techniques do not require a histogram to be present.
[0016] Thus, the one or more processors may be configured to receive a user selection to focus on a region of the preview image and determine whether image content is present at different depths within the region. Based on there being image content at different depths within the region, the one or more processors may output an alternative version of the image content within the region (e.g., a zoomed-in version). In such an example, the one or more processors may receive a subsequent user selection within the alternative version and control (e.g., set or adjust) a focus level (e.g., lens position) for the camera based on the subsequent user selection. However, based on there being no image content at different depths within the region, the one or more processors may control a focus level for the camera based on the user selection (e.g., there may be no output of an alternative version of the image content or no receipt of a subsequent user selection). For example, to control the focus level, the one or more processors may adjust the position of the lens of the camera without receiving a subsequent user selection.
[0017] There may be various examples of subsequent user selection. As an example, the subsequent user selection may be a user selection of a sub-region within the region. For example, as the image content of the region is zoomed in, as an example, the user may better select a sub-region that defines the focus depth of the camera. As another example, the subsequent user selection may be a user selection to manually control the focus level (e.g., manually the focus position of the lens). For example, the user may manually adjust the focus level to a desired focus depth.
[0018] 1 is a block diagram of a device configured to perform one or more of the embodiments of the techniques described in this disclosure. Examples of camera device 10 include a standalone digital camera or digital video camcorder, a camera-equipped wireless communication device handset such as a mobile phone having one or more cameras, a cellular or satellite radiotelephone, camera-equipped personal digital assistants (PDAs), a computing panel or tablet, a watch, a gaming device, a computing device that includes a camera such as a so-called "webcam," or any device with digital imaging or video capabilities.
[0019] As shown in the embodiment of Figure 1, camera device 10 includes a camera 12 (e.g., having an image sensor and a lens), a camera processor 14 and memory 20 local to camera processor 14, a central processing unit (CPU) 16, a graphical processing unit (GPU) 18, a user interface 22, a memory controller 24 that provides access to system memory 30, and a display interface 26 that outputs signals that cause graphical data to be displayed on a display 28. Although the embodiment of Figure 1 shows camera device 10 including one camera 12, in some embodiments camera device 10 may include multiple cameras, such as for omnidirectional image or video capture.
[0020] Also, although various components are illustrated as separate components, in some embodiments, the components may be combined to form a system on a chip (SoC). As an example, the camera processor 14, the CPU 16, the GPU 18, and the display interface 26 may be formed on a common integrated circuit (IC) chip. In some embodiments, one or more of the camera processor 14, the CPU 16, the GPU 18, and the display interface 26 may be in separate IC chips. Additional examples of components that may be configured to perform the exemplary techniques include digital signal processors (DSPs), vector processors, or other hardware blocks used for neural network (NN) calculations. Various other permutations and combinations are possible, and the techniques should not be considered limited to the examples illustrated in FIG. 1.
[0021] 1 (whether formed on one device or different devices) may be formed as at least one of fixed function or programmable circuitry, such as in one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or other equivalent integrated or discrete logic circuitry. Examples of local memory 20 and system memory 30 include one or more volatile or non-volatile memory or storage devices, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data medium, or optical storage medium.
[0022] The various units shown in Figure 1 communicate with each other using a bus 32. The bus 32 can be any of a variety of bus structures, such as a third generation bus (e.g., a HyperTransport bus or an InfiniBand bus), a second generation bus (e.g., an Advanced Graphics Port bus, a Peripheral Component Interconnect (PCI) express bus, or an Advanced eXtentisible Interface (AXI) bus), or another type of bus or device interconnect. The particular configuration of buses and communication interfaces between the different components shown in Figure 1 is merely an example, and other configurations of camera devices and / or other image processing systems having the same or different components may be used to implement the techniques of this disclosure.
[0023] Camera processor 14 is configured to receive image frames (or simply "images") from camera 12 and process the images to generate output images for display. CPU 16, GPU 18, camera processor 14, or some other circuitry may be configured to process output images, including image content generated by camera processor 14, into images for display on display 28. In some embodiments, GPU 18 may be further configured to render graphics content on display 28.
[0024] In some embodiments, camera processor 14 may be configured as an image processing pipeline. For example, camera processor 14 may include a camera interface that interfaces between camera 12 and camera processor 14. Camera processor 14 may include additional circuitry for processing image content. Camera processor 14 outputs an image having the resulting image content (e.g., pixel values for each of the image pixels) to system memory 30 via memory controller 24.
[0025] CPU 16 may comprise a general-purpose or special-purpose processor that controls the operation of camera device 10. A user may provide input to camera device 10 to cause CPU 16 to execute one or more software applications. The software applications executing on CPU 16 may include, for example, a media player application, a video game application, a graphical user interface application, or another program. A user may provide input to camera device 10 via one or more input devices (not shown), such as a keyboard, a mouse, a microphone, a touchpad, or another input device, that are coupled to camera device 10 via a user interface 22.
[0026] One example of a software application is a camera application. CPU 16 executes the camera application, which in response causes CPU 16 to generate content that display 28 outputs. GPU 18 may be configured to process the content generated by CPU 16 for rendering on display 28. For example, display 28 may output information such as light intensity, whether a flash is enabled, and other such information. A user of camera device 10 may interface with display 28 to configure the manner in which an image is generated (e.g., with or without a flash, focus settings, exposure settings, and other parameters).
[0027] As an example, after executing a camera application, the camera device 10 may be considered to be in a preview mode. In the preview mode, the camera 12 outputs image content to the camera processor 14, which performs camera processing and outputs image content to the system memory 30, which the display interface 26 retrieves and outputs to the display 28. In the preview mode, the user may view, via the display 28, image content that is captured when the user operates a button (actual or on the display) to take a picture. As another example, rather than taking a still image (e.g., a photograph), the user may record video content (e.g., a series of images). During recording, the user may be able to view the image content being captured on the display 28.
[0028] In this disclosure, a preview image may be referred to as an image generated in a preview mode. For example, in a preview mode, an image output by camera 12 and stored (e.g., in local memory 20 or system memory 30) for processing by camera processor 14, or an image generated and stored (e.g., in local memory 20 or system memory 30) by camera processor 14 may be referred to as a preview image. In general, a preview image may be an image generated in a preview mode prior to capture and long-term storage of the image.
[0029] During preview mode or recording, camera device 10 (e.g., via CPU 16) may control how camera 12 captures an image (e.g., prior to capturing or storing an image). This disclosure describes the exemplary techniques as being performed by CPU 16. However, the exemplary techniques should not be considered limited to CPU 16 performing the exemplary techniques. For example, CPU 16 may be configured in combination with camera processor 14, GPU 18, DSP, vector processor, and / or display interface 26 to perform the exemplary techniques described in this disclosure. For example, one or more processors may be configured to perform the exemplary techniques described in this disclosure. Examples of one or more processors include camera processor 14, CPU 16, GPU 18, display interface 26, DSP, vector processor, or any combination of one or more of camera processor 14, CPU 16, GPU 18, display interface 26, DSP, or vector processor.
[0030] CPU 16 may be configured to control exposure and / or focus to capture visually pleasing images. For example, CPU 16 may be configured to generate signals to control the exposure, focus, and white balance of camera 12, as some non-limiting examples. CPU 16 may be configured to control exposure, focus, and white balance based on a preview image received from camera processor 14 in a preview mode or during recording. In this manner, for still images, exposure, focus, and white balance are adjusted when the user engages in taking a picture (e.g., parameters for exposure, focus, and possibly white balance are determined prior to image capture such that exposure, focus, and white balance may be corrected during image capture). For recording, exposure, focus, and white balance may be updated periodically during recording.
[0031] This disclosure describes exemplary techniques for controlling the focus level of camera 12, including examples of user interfaces that a user may interact with to control the focus level of camera 12 (e.g., adjust where the lens of camera 12 should be positioned to focus at a desired depth). For focus, CPU 16 may display a preview image on display 28. The user may then select an area of camera 12 to focus on, and in response, CPU 16 may control the focus depth to that of the area selected by the user (e.g., adjust the lens of camera 12 to focus at the desired depth).
[0032] However, there may be various instances where controlling the focus depth to be the depth of the region selected by the user is undefined. Within the region selected by the user, there may be image content at different depths, and it may be unclear which image content the camera 12 should focus on. As an example, the display 28 may be relatively small (such as the display of a so-called smart watch). The user may select an area on the preview image displayed on the display 28 by touch, for example, on a touch screen. Because the size of the display 28 is relatively small, the touch from the user may cover a relatively large area of the image content of the preview image. Within this relatively large region, the image content may be at different depths, and the focus depth may be unclear.
[0033] In one or more embodiments described in this disclosure, CPU 16 may be configured to determine whether image content exists at different depths in a region of the preview image. Based on a determination that image content exists at different depths in the region, CPU 16 may output an alternative version of the image content in the region. As an example, CPU 16 may output a zoomed-in version of the image content in the region.
[0034] For ease of illustration, CPU 16 is described as outputting a zoomed-in version of the image content in the region. However, the exemplary techniques are not so limited. In some embodiments, CPU 16 may output a blurred version of the image content in the region, where the areas that are in focus are not blurred and the areas that are out of focus are blurred. There may be various other ways of outputting alternative versions of the image content.
[0035] CPU 16 may then receive a subsequent user selection within the alternative version and may adjust the focus level (e.g., focus depth) of camera 12 based on the subsequent user selection. However, if there is image content at a single depth in the region (e.g., no image content at different depths), CPU 16 may not output an alternative version of the image content in the region. In such an embodiment, CPU 16 may control (e.g., set or adjust) the focus level (e.g., focus depth) of camera 12 based on the original user selection (e.g., no subsequent user selection is required).
[0036] In other words, the condition may be whether image content exists at different depths within the region. In response to the condition being met (e.g., image content exists at different depths within the region), CPU 16 may output an alternative version of the image content within the region. In response to the condition not being met (e.g., image content does not exist at different depths within the region), CPU 16 may not output an alternative version of the image content within the region.
[0037] In some embodiments, the alternate version of the region may encompass the entire area of display 28 reserved for displaying image content. For example, a zoomed-in version (e.g., one example of an alternate version) may be the image content of the region displayed as a larger object. In this case, image content outside of the region may not be visible on display 28. There may be various methods for generating a zoomed-in version of a region, such as upsampling, and the exemplary techniques are not limited to any zooming technique.
[0038] There may be various ways in which CPU 16 may determine whether image content exists at different depths within a region. As an example, CPU 16 may determine the relative depth of an object in a preview image based on the parallax between the two images. For example, a sensor in camera 12 (or possibly some other sensor) may capture two images of the image content for purposes of determining the relative depth. The images captured to determine the relative depth do not necessarily need to be high resolution images, although it may be possible for the photographs to be high resolution. That is, the images captured by the sensor to determine the relative depth may have the same resolution as the preview image, although the exemplary technique is not so limited.
[0039] The relative disparity of the image content in the two images may indicate the depth of an object. Using the relative disparity, CPU 16 may generate a depth map that indicates the relative depth of the object in the first image. For example, the depth map may indicate how far an object is from camera 12, and based on the depth of the object, CPU 16 may determine whether there is image content at different depths within the region.
[0040] As another example, CPU 16 may generate a defocus map, which is another example of a depth map, sometimes referred to as a PDAF (phase detection autofocus) pixel-based depth map or focus map. The defocus map may be similar to the depth map described above, however, the defocus map indicates the location of objects relative to the current focus depth of camera 12 (i.e., relative to the current focus distance). For example, the defocus map may indicate whether a pixel in the preview image is for an object closer to camera 12 from the current focus depth (e.g., the current focus) or for an object farther from the current focus depth. In some examples, the PDAF pixel-based depth map may be generated using a machine learning algorithm.
[0041] As another example, the camera 12 or camera device 10 may include a time-of-flight (ToF) laser and sensor. The CPU 16 may determine the amount of time it takes for light to travel from the ToF laser to the object and reflect back to the ToF sensor to determine the depth of the object and generate a depth map based on the depth of the object. Based on the depth map, the CPU 16 may determine whether image content is present at different depths within the region.
[0042] The above describes some exemplary techniques that CPU 16 may utilize to determine whether image content exists at different depths within a region. However, the exemplary techniques of this disclosure should not be considered limited to such examples. As an additional technique, CPU 16 may utilize stereo depth or structured light to determine whether image content exists at different depths within a region.
[0043] After CPU 16 outputs an alternative version of the image content in a region (e.g., after display 28 displays the alternative version of the image content), CPU 16 may receive a subsequent user selection within the alternative version. There may be various examples of subsequent user selection within the alternative version.
[0044] As an example, if the alternative version is a zoomed-in version of the image content, a subsequent user selection within the zoomed-in version of the image content within the region may be a user selection that selects a sub-region within the region. For example, the zoomed-in version of the image content within the region may be a larger-sized version of the image content. Within this larger-sized version of the image content displayed on display 28, a user may make a touch user selection (e.g., a tap) of an area that is a sub-region within the region.
[0045] As another example, a subsequent user selection within an alternate version of the image content within the region may be a user selection to manually control the focus level. For example, the user may swipe (e.g., right or left) on the display 28, move a slider on the display 28, and depress and drag a dial on the display 28, as some examples of manual user selections to control focus and manually adjust focus depth. In some examples, the user may manipulate a physical controller, such as a dial, slider, or rotate a button or "crown" for manual user selection to manually control focus.
[0046] In some embodiments, CPU 16 may output a histogram that is displayed on display 28 and indicates the amount of image content that is closer or further from the focus point during subsequent user selection. For example, when a user is swiping to manually adjust a focus level, the histogram may indicate how many samples are closer or further from the focus point that the user is manually adjusting.
[0047] CPU 16 may control the focus level of camera 12 based on a subsequent user selection. For example, CPU 16 may control the focus depth of camera 12. As one example, CPU 16 may control the focus depth of camera 12 based on a sub-region selected as part of a subsequent user selection. As another example, CPU 16 may control the focus depth of camera 12 based on a manually adjusted focus level.
[0048] Camera 12 may then capture an image at the focus level of camera 12 (e.g., at a lens position set by CPU 16). For example, a user may interact with camera device 10 to cause camera device 10 to capture an image (e.g., an image for long-term storage), and camera 12 may capture the image at the focus level.
[0049] In some embodiments, camera 12 may capture one image at a focus level. In some embodiments, camera 12 may capture multiple images at a focus level, and a user may select one or more images to retain. For example, system memory 30 may store multiple images, and a user may view them offline (e.g., at some later time) to determine which images to retain and / or which images to perform post-processing (e.g., manipulation).
[0050] Memory controller 24 facilitates the transfer of data to and from system memory 30. For example, memory controller 24 may receive memory read and write commands and service such commands with respect to memory 30 in order to provide memory services to components within camera device 10. Memory controller 24 is communicatively coupled to system memory 30. Although memory controller 24 is shown in the embodiment of camera device 10 of FIG. 1 as being a processing circuit that is separate from both CPU 16 and system memory 30, in other embodiments, some or all of the functionality of memory controller 24 may be implemented on one or both of CPU 16 and system memory 30.
[0051] System memory 30 may store program modules and / or instructions and / or data that are accessible by camera processor 14, CPU 16, and GPU 18. For example, system memory 30 may store user applications (e.g., instructions for a camera application), obtained frames from camera processor 14, etc. System memory 30 may additionally store information for use by and / or generated by other components of camera device 10. For example, system memory 30 may serve as device memory for camera processor 14.
[0052] Figure 2 is a block diagram illustrating in further detail the central processing unit (CPU) and system memory of the device of Figure 1. As shown in Figure 2, the camera 12 includes a lens 34 and a sensor 36. The CPU 16 may be configured to execute a focus application 38.
[0053] In one or more embodiments, CPU 16 may be configured to generate a depth map 40 that indicates the depth of samples in the preview image. CPU 16 may store depth map 40 in system memory 30. CPU 16 may generate depth map 40 utilizing a variety of techniques, such as based on disparity of pixels in images with the same image content, based on relative distance to focus, or based on a ToF sensor, as some non-limiting examples.
[0054] In one or more embodiments, system memory 30 may store object code for a focus application 38 that is retrieved and executed by CPU 16. In one or more embodiments, rather than being a software application running on CPU 16, CPU 16 may include dedicated hardware for performing the operations of controlling the focus level of camera 12.
[0055] In one or more embodiments, light enters through lens 34 and is received by sensor 36. Focus application 38 may modify the position of lens 34 within camera 12 to adjust the focus of lens 34 (e.g., control the focus level by adjusting the position of lens 34). For example, by modifying the position of lens 34, focus application 38 may control the focus on sensor 36 and therefore which portions of image content are in focus. In some embodiments, it may be possible to move the position of lens 34 to control the focus. In this manner, the focus application may control the focus level of camera 12 (e.g., the position of lens 34) based on subsequent user selections.
[0056] To capture an image, a user may interact with camera device 10. As described above, prior to capturing an image, display 28 may display a preview image for image capture and / or video capture (e.g., a preview of the image to be captured). According to examples described in this disclosure, the user may determine where the focus of camera 12 should be set (e.g., where the position of lens 12 should be). System memory 30 may store the preview image.
[0057] For example, focus application 38 may receive a user selection to focus on an area of a preview image stored in system memory 30. One example of a user selection may be a touch (e.g., a tap) by a user on display 28 where the user wants camera 12 to focus (e.g., what the focus level of camera 12 should be). In this example, CPU 16 may determine the coordinates of where the user touched display 28 and determine that the area encompassed by where the user touched display 28 is the user selection area. The user selection technique being a "touch" or "tap" is by way of example and should not be considered limiting. The user may manually enter the coordinates, select the area from a menu, etc.
[0058] According to one or more embodiments described in this disclosure, the focus application 38 may determine whether image content exists at different depths within a region. For example, the focus application 38 may access a depth map 40. The focus application 38 may determine a region within the depth map 40 that corresponds to a user-selected region of the preview image. Based on the depths of samples within the region as indicated by the depth map 40, the focus application 38 may determine whether image content exists at different depths within the region.
[0059] Based on a determination that image content exists at different depths within the region, focus application 38 may output an alternate version of the image content within the region. In some examples, rather than or in addition to focus application 38 outputting an alternate version of the image content, GPU 18 may be configured to output an alternate version of the image content. As another example, CPU 16 may execute a zoom application or function, and focus application 38 may invoke a zoom application, which may output a zoomed-in version of the image content within the region (e.g., one example of an alternate version). Other methods of outputting an alternate version of the image content within the region are also possible. Display 28 may be configured to display an alternate version of the image content within the region.
[0060] The condition may be whether image content exists at different depths in the region. For example, in response to the condition that image content exists at different depths in the region being met, CPU 16 and / or GPU 18 may output an alternative version of the image content in the region. In some embodiments, CPU 16 and / or GPU 18 may output an alternative version of the image content in the region only if image content exists at different depths in the region. For example, CPU 16 and / or GPU 18 outputs an alternative version of the image content in the region only if the condition that image content exists at different depths in the region is met. As an example, if the condition is not met (i.e., image content does not exist at different depths in the region), CPU 16 and / or GPU 18 may not output an alternative version of the image content in the region.
[0061] The focus application 38 may receive a subsequent user selection within the alternative version. There may be various examples of subsequent user selection within the alternative version. As an example, the focus application 38 may receive a subsequent user selection within the alternative version of the image content within the region that selects a sub-region within the region. For example, a subsequent user selection within the alternative version of the image content within the region that selects a sub-region within the region may be a touch user selection that selects a sub-region within the region. As another example, the focus application 38 may receive a subsequent user selection within the alternative version of the image content to manually control focus. For example, a subsequent user selection within the alternative version of the image content to manually control focus may be a swipe user selection, a slider movement, a dial press and drag, as some examples. Examples of sub-region selection and manual control of focus are described with respect to FIGS. 5 and 6. In some examples, a user may manually control focus by manipulating a physical controller, such as a dial, slider, or rotating a button or "crown" for manual user selection.
[0062] The focus application 38 may control the focus level of the camera 12 based on subsequent user selections. For example, the focus application 38 may set the focus depth (e.g., set the focus) of the camera 12. As an example, the focus application 38 may control the position of the lens 34 and / or the sensor 36 to control the focus depth. The camera device 10 may be configured to capture an image at the focus of the camera 12. For example, after setting the focus, a user may interact with the camera device 10 to capture an image (e.g., a still image or video) with the focus of the camera 12 at the focus level.
[0063] 3 is a conceptual diagram illustrating an example of a device that displays image content and user selection of a region. For example, FIG. 3 illustrates camera device 10 as a clock, with display 28 being the clock face. In some implementations, the sensor may be on the clock face. In this implementation, a user may touch region 50 to control the focus level.
[0064] 4A and 4B are conceptual diagrams showing focus levels set at different depths. 4A and 4B show zoomed-in versions of region 50 of FIG. 3. However, in 4A and 4B, the focus depths are different (e.g., different focal points). For example, in 4A, the position of lens 34 is at position 175, which results in a focus depth at the depth of the screw shown in the image. In 4B, the position of lens 34 is at position 207, which results in a focus depth at the depth of the spring shown in the image.
[0065] 4A and 4B, relying solely on a touch by a user within region 50 (FIG. 3) to define focus depth may not be sufficient to determine the focus level of camera 12. For example, selection of region 50 alone may make it unclear whether camera 12 should focus on a screw and lens 34 should have a lens position of 175, or whether camera 12 should focus on a spring and lens 34 should have a lens position of 207.
[0066] Thus, in one or more embodiments, one or more processors (e.g., CPU 16) may be configured to determine whether image content exists at different depths in region 50. If image content exists at different depths in region 50, CPU 16 may cause display 28 to display an alternate version of the image content (e.g., a zoomed-in version). For example, the alternate version of the image content may be such that it is easier for a user to properly identify where to focus camera 12. However, if image content exists at a single depth in region 50 (e.g., no image content exists at different depths), CPU 16 may not cause display 28 to display the alternate version.
[0067] FIG. 5 is a conceptual diagram illustrating an example of a user interface for touch focus assist mode. FIG. 5 illustrates a zoomed-in version of region 50 of FIG. 3. The zoomed-in version of region 50 is an example of an alternative version of region 50. In touch focus assist mode, a user selects a sub-region 52 within region 50. As an example, a subsequent user selection within an alternative version (e.g., a zoomed-in version) of the image content within region 50 that selects sub-region 52 within region 50 may be a touch user selection that selects sub-region 52 within region 50. As shown in FIG. 5, the image content of region 50 is larger in the zoomed-in version of region 50, so the user can better specify that the focus should be on sub-region 52.
[0068] FIG. 6 is a conceptual diagram illustrating an example of a user interface for the manual focus assist mode. Similar to FIG. 5, FIG. 6 illustrates a zoomed-in version of the region 50 of FIG. 3, which is an example of an alternative version of the region 50. When in the manual focus assist mode, the display 28 may display an indication 54 indicating the position of the lens 34 and / or the sensor 36. The user may see where the focus of the camera 12 is based on which objects appear sharp and which objects appear blurry on the display 28. That is, the CPU 16 may be configured to apply an exaggerated blur effect to out-of-focus areas, simulating a shallow depth-of-field effect for the purpose of emphasizing which areas are accurately in focus.
[0069] A user may swipe left and right, move sliders, and hold and drag on the display to change the focus depth (e.g., focus) by way of one non-limiting example and to move the position of the lens 34 and / or sensor 36. As the user changes the position of the lens 34 and / or sensor 36, different objects on the display 28 may become blurred and previously blurred objects may come into focus and appear sharp. The CPU 16 may control the focus of the camera based on where the user stops swiping (e.g., where the user manually controls the focus).
[0070] In some embodiments, CPU 16 may be configured to output a histogram 56 that indicates the amount of image content closer to the focal point and farther from the focal point during a user selection. For example, as a user provides a user selection (e.g., a swipe user selection, a move user selection on a slider, hold and drag), the shape of histogram 56 may change to indicate how many samples are closer to the focal point (e.g., the left side of histogram 56) and how many samples are farther from the focal point (e.g., the right side of histogram 56). CPU 16 need not necessarily output histogram 56 in all embodiments.
[0071] In the example of FIG. 6, there is one histogram 56. However, the example technique is not so limited. In some examples, there may be two or more histograms. For example, a first histogram may display the relative depth of pixels in the full field of view (i.e., all pixels to be captured). A second histogram may display the relative depth of pixels currently in the view (e.g., if zoom-in is used to zoom in on a portion to select depth). The first and second histograms may be displayed overlaid or separately.
[0072] 7A and 7B are conceptual diagrams illustrating an example of a user interface with blurring. In the above examples, an example of an alternative version of the image content is a zoomed-in version of the image content. However, in some examples, rather than or in addition to outputting a zoomed-in version of the image content, CPU 16 may output a blurred image, which is another example of an alternative version of the image content.
[0073] For example, as shown in Figures 7A and 7B, after a user selection to focus on a region of a preview image, and after determining that there is image content at different depths within that region, CPU 16 may output a blurred version of the image content within that region. In the blurred version, areas that are currently out of focus may be rendered (e.g., by GPU 18) with a relatively heavy blur. The blurred version may be thought of as simulating a very shallow depth of field. In this way, it becomes clear which areas are in focus and which areas are out of focus.
[0074] As an example, the positions of the lens 34 and / or the sensor 36 may be different in Figs. 7A and 7B. In Fig. 7A, the pencil 58A appears heavily blurred to indicate that the focus level of the camera 12 is not at the depth of the pencil 58A. However, in Fig. 7B, the pencil 58B appears in focus and is not blurred to indicate that the focus of the camera 12 is at the depth of the pencil 58B. The user may manually adjust the focus of the camera 12, and in the example of Figs. 7A and 7B, the blurring may indicate the focus of the camera 12 to the user. When the focus of the camera 12 (e.g., the focus depth of the camera 12) is at a desired location, the user may stop changing the position of the lens 34 and / or the sensor 36, and the CPU 16 may control the focus level of the camera 12 based on the position of the lens 34 and / or the sensor 36.
[0075] 8 is a flow chart illustrating one or more exemplary techniques of operation. For ease of explanation, FIGS. 8-10 are described with respect to one or more processors configured to perform the exemplary techniques. Examples of the one or more processors include any one or combination of CPU 16, GPU 18, display interface 26, or any other processing components of camera device 10.
[0076] In the example of FIG. 8, one or more processors may receive a user selection to focus on a region of a preview image (e.g., region 50 of FIG. 3) stored in memory (e.g., system memory 30) (70). As an example, the user may touch or tap on display 28, and the area encompassed by the touch may correspond to the region on which camera 12 should focus. However, other techniques are possible, such as by entering region information based on coordinates, selecting from a menu, etc. In general, one or more processors receiving a user selection may refer to one or more processors receiving a signal generated by display 28 or some other unit indicating the region selected by the user.
[0077] The one or more processors may determine whether image content exists at different depths in the region (72). As an example, the one or more processors may access a depth map 40 from the system memory 30 and determine whether image content exists at different depths in the region based on the depth map 40. For example, as described above, the depth map 40 may indicate relative depths of samples in a preview image. If the depth map 40 indicates that there are samples with different depths in the image content in the region, the one or more processors may determine that image content exists at different depths. If the depth map 40 indicates that there are no samples with different depths in the image content in the region, the one or more processors may determine that no image content exists at different depths (e.g., image content exists at a single depth).
[0078] Based on a determination that image content is present at different depths within the region, the one or more processors may output an alternative version of the image content in the region (74). As an example, in response to a condition that image content is present at different depths within the region being satisfied, the one or more processors may output an alternative version of the image content in the region. In some cases, the one or more processors may output an alternative version of the image content in the region only when image content is present at different depths within the region.
[0079] As one example, to output an alternative version of the image content in the region, the one or more processors may output a zoomed-in version of the image content in the region. As another example, to output an alternative version of the image content in the region, the one or more processors may blur portions of the region that are out of focus.
[0080] The one or more processors may receive a subsequent user selection within the alternative version (76). As above, the one or more processors receiving the subsequent user selection may reference a signal generated by the display 28 or some other unit indicating the subsequent user selection. The one or more processors may control a focus level of the camera 12 based on the subsequent user selection (78). For example, the one or more processors may set a focus depth of the camera 12. The camera device 10 may be configured to capture an image at a focus level of the camera 12.
[0081] As an example, to receive a subsequent user selection, the one or more processors may receive a subsequent user selection within the alternative version of the image content within the region that selects a sub-region within the region. In such an example, to control the focus level, the one or more processors may control the focus level based on the subsequent user selection of the sub-region. For example, the subsequent user selection within the alternative version of the image content within the region that selects a sub-region within the region may be a touch user selection that selects the sub-region within the region.
[0082] As another example, to receive a subsequent user selection, the one or more processors may receive a subsequent user selection in the alternative version of the image content to manually control the focus level. In such an example, to control the focus level, the one or more processors may control the focus level based on the user's manual control of the focus level. For example, the subsequent user selection in the alternative version of the image content to manually control the focus level may be a user selection to manually adjust the focus depth. In some examples, the one or more processors may be configured to output a histogram (e.g., histogram 56) indicating the amount of image content closer to the focus and farther from the focus during the subsequent user selection.
[0083] 9 is a flow chart illustrating one or more exemplary techniques of operation. FIG. 9 illustrates an example where image content exists at a single depth (e.g., no image content exists at different depths). For example, in FIG. 8, assume that the user selection is a first user selection, the region is a first region, the focus level is a first focus level, and the preview image is a first preview image.
[0084] In some examples, the one or more processors may receive a second user selection to focus on a second region of the second preview image (80). In this example, the one or more processors may determine whether image content is present at different depths in the second region (82). For example, the one or more processors may perform a technique similar to the technique described in FIG. 8 to determine whether image content is present at different depths in the second region.
[0085] Based on a determination that there is no image content at different depths in the second region, the one or more processors may control a second focus level of the camera 12 based on a second user selection (84). For example, because there is no image content at different depths, there may be no confusion about where to focus the camera 12. Thus, rather than displaying an alternative version of the image content, the one or more processors may be able to control the focus level of the camera 12 based on a user selection (e.g., no subsequent user selection is required).
[0086] Figure 10 is a flow chart illustrating one or more exemplary techniques of operation. In the example of Figure 10, display 28 may receive a user touch at a region (90). Display 28 may send a signal to one or more processors that identifies the area encompassed by the region.
[0087] The one or more processors may determine whether image content is present at different depths within the region (92). If the one or more processors determine that image content is not present at different depths ("NO" at 92), the one or more processors may control a focus level of camera 12 based on the region touched by the user (94).
[0088] If the one or more processors determine that image content exists at different depths ("YES" at 92), the one or more processors may output an alternative version of the image content in the region (96). Examples of alternative versions include a zoomed-in version and a blurred version, although other examples are possible.
[0089] The one or more processors may determine (98) whether a user is pressing, holding, or hovering over display 28. For example, display 28 may generate a signal that is received by the one or more processors indicating whether a user is pressing, holding, or hovering over display 28.
[0090] If the one or more processors determine that the user is pressing, holding, or hovering over display 28 ("YES" at 98), the one or more processors may control a focus level of camera 12 based on a touch focus assist mode. An example of a touch focus assist mode is described above with respect to FIG.
[0091] If the one or more processors determine that the user is not pressing, holding, or hovering over the display 28 ("NO" at 98), the one or more processors may determine whether the user is swiping left or right on the display 28 (102). If the one or more processors determine that the user is swiping left or right on the display 28 ("YES" at 102), the one or more processors may control a focus level of the camera 12 based on a manual focus assist mode (104). Examples of manual focus assist modes include those shown in FIGS. 6, 7A, and 7B.
[0092] If the one or more processors determine that the user is not swiping left or right on display 28, the one or more processors may terminate control of the focus level (106). For example, if the user does not provide input on how to control the focus level, the operation of controlling focus may time out.
[0093] In the example of FIG. 10, the example operations are not necessarily performed in the order shown. For example, in some examples, the one or more processors may first determine whether there is a user selection on the display 28 (e.g., a swipe, a slider movement, a hold and drag of a dial, etc.) to manually control the focus level (102) and control the focus level based on whether there is a user selection. In such examples, if there is no user selection on the display 28 to manually control the focus level, the one or more processors may determine whether there is a press, hold, or hover on the display 28 for selection of a sub-region to control the focus level (98). In some examples, the user may manually control the focus by manipulating a physical controller, such as a dial, slider, or rotating a button or "crown" for manual user selection.
[0094] Described below is one or more example techniques according to this disclosure.
[0095] Example 1. A device for image capture comprising: a memory configured to store a preview image; and one or more processors coupled to the memory, wherein the one or more processors are configured to receive a user selection to focus on a region of the preview image stored in the memory; determine whether image content is present at different depths within the region; based on a determination that image content is present at different depths within the region, output an alternative version of the image content within the region; receive a subsequent user selection within the alternative version; and control a camera focus level based on the subsequent user selection.
[0096] Example 2. The device of example 1, wherein outputting an alternative version of the image content in the region includes outputting a zoomed-in version of the image content in the region.
[0097] Example 3. The device of any of Examples 1 and 2, wherein controlling the focus level of the camera includes controlling a lens position of a lens of the camera.
[0098] Example 4. A device as described in any of Examples 1-4, wherein receiving a subsequent user selection includes receiving a subsequent user selection within an alternative version of the image content within the region, selecting a sub-region within the region, and controlling the focus level includes controlling the focus level based on the subsequent user selection of the sub-region.
[0099] Example 5. The device of example 4, wherein a subsequent user selection within the alternative version of the image content within the region that selects a sub-region within the region includes a touch user selection that selects the sub-region within the region.
[0100] Example 6. A device described in any of Examples 1-5, wherein receiving a subsequent user selection includes receiving a subsequent user selection within an alternative version of the image content to manually control the focus level, and controlling focus includes controlling focus based on the user's manual control of the focus level.
[0101] Example 7. The device of Example 6, wherein the one or more processors are configured to output a histogram indicating an amount of image content closer to the focus and another amount of image content further from the focus during subsequent user selection.
[0102] Example 8. The device of Examples 6 and 7, wherein a subsequent user selection within the alternative version of the image content to manually control the focus level includes a user selection of a swipe, slide, or drag and hold to manually adjust the focus depth.
[0103] Example 9. The device of Examples 1-8, wherein the user selection is a first user selection, the region is a first region, the focus level is a first focus level, and the preview image is a first preview image, and the one or more processors are configured to receive a second user selection to focus on a second region of the second preview image, determine whether image content is present at a different depth in the second region, and based on a determination that image content is not present at a different depth in the second region, control a second focus level of the camera based on the second user selection.
[0104] Example 10. A device described in any of Examples 1-9, wherein outputting an alternative version of the image content in the region based on a determination that image content exists at a different depth within the region includes outputting an alternative version of the image content in the region in response to a condition that image content exists at a different depth within the region being satisfied.
[0105] Example 11. A device as described in any of Examples 1-10, wherein outputting an alternative version of the image content in the region includes blurring portions of the region that are out of focus.
[0106] Example 12. A device according to any of Examples 1-11, wherein the device is configured to capture images at a camera focus level.
[0107] Example 13. A device described in any of Examples 1-12, wherein determining whether image content is present at different depths within the region includes determining whether image content is present at different depths based on one or more of a phase detection autofocus (PDAF) based depth map, a defocus map, time of flight (ToF), stereo depth, or structured light.
[0108] Example 14. A method for image capture comprising: receiving a user selection to focus on a region of a preview image stored in memory; determining that image content is present at a different depth within the region; outputting an alternative version of the image content within the region based on the determination that image content is present at a different depth within the region; receiving a subsequent user selection within the alternative version; and controlling a focus level of a camera based on the subsequent user selection.
[0109] Example 15. The method of example 14, wherein outputting the alternative version of the image content includes outputting a zoomed-in version of the image content.
[0110] Example 16. The method of any of Examples 14 and 15, wherein controlling the focus level of the camera includes controlling a lens position of the camera lens.
[0111] Example 17. A method as described in any of Examples 14-16, wherein receiving a subsequent user selection includes receiving a subsequent user selection within an alternative version of the image content within the region, selecting a sub-region within the region, and controlling the focus level includes controlling the focus level based on the subsequent user selection of the sub-region.
[0112] Example 18. The method of example 17, wherein a subsequent user selection within an alternative version of the image content within the region that selects a sub-region within the region includes a touch user selection that selects a sub-region within the region.
[0113] Example 19. A method as described in any of Examples 14-18, wherein receiving a subsequent user selection includes receiving a subsequent user selection within an alternative version of the image content to manually control the focus level, and controlling focus includes controlling focus based on manual control of the focus level by the user.
[0114] Example 20. The method of example 19, further comprising outputting a histogram indicating an amount of image content closer to the focus point and another amount of image content further from the focus point during subsequent user selection.
[0115] Example 21. A method as described in any of Examples 19 and 20, wherein a subsequent user selection within an alternative version of the image content to manually control the focus level includes a user selection of a swipe, slide, or drag and hold to manually adjust the focus depth.
[0116] Example 22. A method according to any of Examples 14-21, wherein the user selection is a first user selection, the region is a first region, the focus level is a first focus level, and the preview image is a first preview image, and the method further includes receiving a second user selection to focus on a second region of the second preview image, determining that image content is present at a different depth in the second region, and based on a determination that image content is not present at a different depth in the second region, controlling a second focus level of the camera based on the second user selection.
[0117] Example 23. A method as described in any of Examples 14-22, wherein outputting an alternative version of the image content in the region based on a determination that image content exists at a different depth within the region includes outputting an alternative version of the image content in the region in response to a condition that image content exists at a different depth within the region being satisfied.
[0118] Example 24. The method of any of Examples 14-23, wherein outputting an alternative version of the image content in the region includes blurring portions of the region that are out of focus.
[0119] Example 25. The method of any of Examples 14-24, further comprising capturing an image at a camera focus level.
[0120] Example 26. A method as described in any of Examples 14-25, wherein determining that image content exists at different depths within the region includes determining whether image content exists at different depths based on one or more of a phase detection autofocus (PDAF) based depth map, a defocus map, time of flight (ToF), stereo depth, or structured light.
[0121] Example 27. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to receive a user selection to focus on a region of a preview image stored in memory, determine that image content is present at different depths within the region, output an alternative version of the image content within the region based on the determination that image content is present at different depths within the region, receive a subsequent user selection within the alternative version, and control a focus level of a camera based on the subsequent user selection.
[0122] Example 28. The computer-readable storage medium of Example 27, wherein the instructions for causing the one or more processors to control the focus level of the camera comprise instructions for causing the one or more processors to control a lens position of a lens of the camera.
[0123] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium. In this manner, a computer-readable medium may generally correspond to a tangible computer-readable storage medium that is non-transitory. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.
[0124] By way of example and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be understood that computer-readable storage media and data storage media do not include carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. Disk and disc as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.
[0125] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the above structures, or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.
[0126] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but they do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or may be provided by a collection of interoperable hardware units including one or more processors as described above in conjunction with suitable software and / or firmware.
[0127] Various examples have been described. These and other examples are within the scope of the following claims.
Claims
1. 1. A device for image capture, comprising: a memory configured to store a preview image; and one or more processors coupled to the memory, the one or more processors comprising: receiving a user selection to focus on an area of the preview image stored in the memory; determining whether image content exists at different depths within the region; outputting an alternative version of the image content in the region based on the determination that image content exists at a different depth within the region; receiving a subsequent user selection within the alternative version; The device is configured to control a focus level of a camera based on the subsequent user selection.
2. The device of claim 1 , wherein outputting the alternative version of the image content in the region comprises outputting a zoomed-in version of the image content in the region.
3. The device of claim 1 , wherein controlling the focus level of the camera comprises controlling a lens position of a lens of the camera.
4. receiving the subsequent user selection includes receiving the subsequent user selection within the alternative version of the image content within the region selecting a sub-region within the region; The device of claim 1 , wherein controlling the focus level comprises controlling the focus level based on the subsequent user selection of the sub-region.
5. The device of claim 4 , wherein the subsequent user selection in the alternative version of the image content within the region selecting the sub-region within the region comprises a touch user selection selecting the sub-region within the region.
6. receiving the subsequent user selection includes receiving the subsequent user selection within the alternative version of the image content to manually control the focus level; The device of claim 1 , wherein controlling the focus comprises controlling the focus based on manual control of the focus level by a user.
7. 7. The device of claim 6, wherein the one or more processors are configured to output a histogram indicating an amount of the image content that is closer to a focal point and another amount of the image content that is further from a focal point during the subsequent user selection.
8. 7. The device of claim 6, wherein the subsequent user selection within the alternative version of the image content to manually control the focus level includes a user selection of a swipe, slide, or drag and hold to manually adjust focus depth.
9. the user selection is a first user selection, the region is a first region, the focus level is a first focus level, and the preview image is a first preview image, and the one or more processors: receiving a second user selection to focus on a second region of the second preview image; determining whether image content exists at different depths in the second region; 2. The device of claim 1, configured to control a second focus level for the camera based on the second user selection based on the determination that no image content is present at different depths in the second region.
10. 2. The device of claim 1, wherein outputting the alternative version of the image content in the region based on the determination that image content is present at a different depth within the region comprises outputting the alternative version of the image content in the region in response to a condition that image content is present at a different depth within the region being satisfied.
11. The device of claim 1 , wherein outputting the alternative version of the image content in the region comprises blurring portions of the region that are out of focus.
12. The device of claim 1 , wherein the device is configured to capture an image at the focus level of the camera.
13. 2. The device of claim 1, wherein determining whether image content is present at different depths within the region includes determining whether image content is present at different depths based on one or more of a phase detection autofocus (PDAF) based depth map, a defocus map, time of flight (ToF), stereo depth, or structured light.
14. 1. A method for image capture, comprising: receiving a user selection to focus on an area of the preview image stored in the memory; determining that image content exists at different depths within the region; outputting an alternative version of the image content in the region based on the determination that image content exists at a different depth within the region; receiving a subsequent user selection within the alternative version; controlling a focus level of the camera based on said subsequent user selection; A method comprising:
15. The method of claim 14 , wherein outputting the alternative version of the image content comprises outputting a zoomed-in version of the image content.
16. The method of claim 14 , wherein controlling the focus level of the camera comprises controlling a lens position of a lens of the camera.
17. receiving the subsequent user selection includes receiving the subsequent user selection within the alternative version of the image content within the region selecting a sub-region within the region; The method of claim 14 , wherein controlling the focus level comprises controlling the focus level based on the subsequent user selection of the sub-region.
18. The method of claim 17 , wherein the subsequent user selection within the alternative version of the image content within the region selecting the sub-region within the region comprises a touch user selection selecting the sub-region within the region.
19. receiving the subsequent user selection includes receiving the subsequent user selection within the alternative version of the image content to manually control the focus level; The method of claim 14 , wherein controlling the focus comprises controlling the focus based on manual control of the focus level by a user.
20. 20. The method of claim 19, further comprising outputting a histogram indicating an amount of the image content that is closer to a focal point and another amount of the image content that is further from a focal point during the subsequent user selection.
21. 20. The method of claim 19, wherein the subsequent user selection within the alternative version of the image content to manually control the focus level comprises a user selection of a swipe, slide, or drag and hold to manually adjust focus depth.
22. 11. A method, the user selection being a first user selection, the region being a first region, the focus level being a first focus level, and the preview image being a first preview image, the method comprising: receiving a second user selection to focus on a second region of the second preview image; determining that image content exists at different depths in the second region; and controlling a second focus level of the camera based on the second user selection based on the determination that no image content is present at different depths in the second region.
23. 15. The method of claim 14, wherein outputting the alternative version of the image content in the region based on the determination that image content is present at different depths in the region comprises outputting the alternative version of the image content in the region in response to a condition that image content is present at different depths in the region being satisfied.
24. The method of claim 14 , wherein outputting the alternative version of the image content in the region comprises blurring portions of the region that are out of focus.
25. The method of claim 14 , further comprising capturing an image at the focus level of the camera.
26. 15. The method of claim 14, wherein determining that image content is present at different depths within the region comprises determining whether image content is present at different depths based on one or more of a Phase Detection Autofocus (PDAF) based depth map, a defocus map, time of flight (ToF), stereo depth, or structured light.
27. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to: receiving a user selection to focus on an area of the preview image stored in the memory; determining that image content exists at different depths within the region; outputting an alternative version of the image content in the region based on the determination that image content exists at a different depth in the region; receiving a subsequent user selection within the alternative version; A computer-readable storage medium for controlling a focus level of a camera based on the subsequent user selection.
28. 30. The computer-readable storage medium of claim 27, wherein the instructions for causing the one or more processors to control the focus level of the camera comprise instructions for causing the one or more processors to control a lens position of a lens of the camera.
Citation Information
Patent Citations
Camera
JP2010192999A
Image display device, imaging apparatus including the same, image display method, program, storage medium storing the same
JP2013142725A
Imaging apparatus and control method of the same
JP2014150466A
Imaging apparatus, control method and program thereof, and storage medium
JP2014228629A
Photographing apparatus and imaging method
JP2015059998A
