Field of view correction technology for shutterless camera systems

Abstract

An exemplary embodiment relates to a field of view correction technique for a shutterless camera system. A mobile device displaying an initial preview of a scene being captured by an image capture device of a computing device can determine a zoom operation configured to cause the imaging capture device to focus on a target. The imaging capture device is configured to change a focal length when performing the zoom operation. While the image capture device performs the zoom operation, the computing device then maps the focal length used by the imaging capture device to a virtual focal length so that the field of view of the scene remains constant across image frames displayed by the display screen between the initial preview of the scene and the zoomed preview of the scene focused on the target.

Description

[Background technology]

[0001] Many modern computing devices, such as mobile phones, personal computers, and tablets, include image capture devices (e.g., still cameras and / or video cameras). Image capture devices can capture images that may depict a variety of scenes, including scenes containing people, animals, landscapes, and / or objects. Some image capture devices are configured with telephoto capabilities. Summary of the Invention

[0002]

[0003] Exemplary embodiments presented herein relate to field of view (FOV) correction techniques for shutterless camera systems. To reduce undesirable visual artifacts that can occur during autofocus sweeps, mobile devices or other types of computing devices may apply FOV correction techniques that use camera parameter interpolation to keep the field of view constant across image frames displayed by the device. If the camera used by the mobile device is shutterless and sequentially reads out rows (or columns) of image frames, the mobile device may analyze the actual focal length and optical center for each row (or column) when applying the FOV correction technique to accommodate the different exposure intervals associated with the sequential readout.

[0003] To that end, in a first exemplary embodiment, a computer-implemented method is provided. The method includes displaying, on a display screen of a computing device, an initial preview of a scene being captured by an image capture device of the computing device, the image capture device operating at an initial focal length when capturing the initial preview of the scene. The method also includes the computing device determining a zoom operation configured to cause the imaging capture device to focus on a target, the imaging capture device being configured to change the focal length when performing the zoom operation. The method further includes mapping the focal length used by the imaging capture device to a virtual focal length while the image capture device performs the zoom operation, such that a field of view of the scene remains constant across image frames displayed by the display screen between the initial preview of the scene and the zoom preview of the scene focused on the target, and displaying, on the display screen of the computing device, the zoom preview of the scene focused on the target.

[0004] In a second exemplary embodiment, a mobile device is provided. The mobile device includes a display screen, an image capture device, one or more processors, and data storage. The data storage has stored therein computer-executable instructions that, when executed by the one or more processors, cause the mobile device to perform operations. The operations include displaying, on the display screen, an initial preview of a scene being captured by an image capture device of the computing device, the image capture device operating at an initial focal length when capturing the initial preview of the scene. The operations also include determining a zoom operation configured to focus the image capture device on a target, the image capture device configured to change the focal length when performing the zoom operation. The operations further include mapping the focal length used by the image capture device to a virtual focal length while the image capture device performs the zoom operation, such that a field of view of the scene remains constant across image frames displayed by the display screen between the initial preview of the scene and the zoomed preview of the scene focused on the target. The operations also include displaying, on the display screen, the zoomed preview of the scene focused on the target.

[0005] In a third exemplary embodiment, a non-transitory computer-readable medium includes program instructions executable by one or more processors to cause the one or more processors to perform operations. The operations include displaying, on a display screen, an initial preview of a scene being captured by an image capture device of a computing device, the image capture device operating at an initial focal length when capturing the initial preview of the scene. The operations also include determining a zoom operation configured to cause the image capture device to focus on a target, the image capture device configured to change the focal length when performing the zoom operation. The operations further include mapping the focal length used by the image capture device to a virtual focal length while the image capture device performs the zoom operation, such that a field of view of the scene remains constant across image frames displayed by the display screen between the initial preview of the scene and the zoomed preview of the scene focused on the target. The operations also include displaying, on the display screen, the zoomed preview of the scene focused on the target.

[0006] In the fourth exemplary embodiment, the system may include various means for performing each of the operations of the above exemplary embodiments.

[0007] These and other embodiments, aspects, advantages, and alternatives will become apparent to those skilled in the art upon reading the following detailed description, with reference to the accompanying drawings as appropriate. Moreover, it should be understood that this summary and other descriptions and illustrations provided herein are intended to illustrate embodiments by way of example only, and thus, that numerous variations are possible. For example, structural elements and process steps may be rearranged, combined, distributed, eliminated, or otherwise modified while remaining within the scope of the claimed embodiments. [Brief explanation of the drawings]

[0008] [Figure 1A] 1A-1D illustrate front and side views of a digital camera device according to one or more exemplary embodiments. [Figure 1B] 1 illustrates a rear view of a digital camera device according to one or more exemplary embodiments. [Figure 2] 1 illustrates a block diagram of a computing system with image capture capabilities, in accordance with one or more exemplary embodiments. [Figure 3] 1 illustrates a simplified representation of an image capture component capturing an image of a person, according to one or more exemplary embodiments. [Figure 4] 1 illustrates an image capture device that implements autofocus (AF) techniques, according to one or more exemplary embodiments. [Figure 5] FIG. 1 is a block diagram of a mobile device configured to perform the disclosed FOV correction techniques, according to one or more exemplary embodiments. [Figure 6] 1 illustrates a comparison of an actual camera view and a virtual camera view modified by an FOV correction technique, in accordance with one or more exemplary embodiments. [Figure 7] 1 is a flowchart of a method for applying FOV correction to image frames being captured by a camera system, according to one or more example embodiments. [Figure 8A] 10 illustrates a focal length representation determined based on an average focal length for an exposure interval, according to one or more example embodiments. [Figure 8B] 10 illustrates a focal length representation determined based on a focal length at the center of an exposure interval, according to one or more example embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] Exemplary methods, devices, and systems are described herein. It should be understood that the words "example" and "exemplary" are used herein to mean "serving as an example, instance, or illustration." Any embodiment or feature described herein as an "example" or "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or features. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein.

[0010] Accordingly, the exemplary embodiments described herein are not intended to be limiting. The aspects of the present disclosure as generally described herein and shown in the figures can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein. Furthermore, unless the context suggests otherwise, the features illustrated in each of the figures may be used in combination with one another. Thus, the figures should generally be viewed as component aspects of one or more overall embodiments, with the understanding that not all of the illustrated features are required for each embodiment.

[0011] Depending on the context, a "camera" may refer to an individual image capture device or a device that includes one or more image capture components. Generally, an image capture device may include an aperture, a lens, a recording surface, and a shutter, as described below. The terms "image" and "payload image" may be used herein to describe a final image of a scene that a user of the camera records and can later view. The terms "image frame" and "frame" may be used herein to refer to a temporarily stored representation of a scene that is displayed for preview purposes or that is captured and analyzed to determine one or more qualities of a scene prior to capturing an image (e.g., to determine what types of objects are present in a given scene, areas of interest within a given scene, appropriate exposure time, ambient light intensity, motion blur resistance, etc.).

[0012] Furthermore, in some implementations, the image processing steps described herein may be performed by a camera device, while in other implementations, the image processing steps may be performed by a computing device that communicates with (and possibly controls) one or more camera devices.

[0013] I. Overview Autofocus (AF) is a feature that allows digital cameras, smartphones, and other types of camera devices to automatically sharpen an image and focus on a particular spot or subject with little to no input from the user. There are various ways that a camera can perform AF, including passive AF techniques (e.g., contrast-detection AF (CDAF) and phase-detection AF (PDAF)), and active or hybrid techniques (e.g., laser AF).

[0014] Some AF techniques involve automatically adjusting the distance between the camera lens and the image sensor until the camera operates at a focal length that focuses on a specific spot or subject. For example, the camera may sweep the lens between various positions relative to the image sensor until the camera's software determines that the target is in focus. Rapid changes in focal length during AF can cause the camera's FOV to change abruptly, resulting in the camera displaying image frames with breathing artifacts that can adversely affect a user's experience when using the camera to capture images of a scene. Specifically, when the camera performs AF, the image previews displayed by the camera may appear to be captured from different perspectives due to the rapid changes in focal length caused by the AF sweep.

[0015] Exemplary embodiments relate to FOV correction techniques that can be implemented by mobile devices and other computing systems to reduce breathing artifacts that can occur when a camera rapidly adjusts its focal length to focus on a target or aspect within a scene. For example, when a camera on a mobile device begins an AF sweep to focus on a target within a scene, the mobile device can execute software that warps the camera's real focal length to a virtual focal length, thereby enabling the mobile device to display an image preview of the scene in which the FOV remains constant, even though the camera's actual FOV changes during the AF sweep. By generating an image preview in which the FOV appears constant as the camera performs AF, undesirable visual artifacts that are aesthetically pleasing to the user can be eliminated, and the camera can display an image preview in which the FOV appears constant.

[0016] Additionally, the disclosed FOV correction techniques can be used in shutterless cameras in which rows (or columns) of an image are read out sequentially, rather than all at once. For example, if a camera uses a rolling shutter, rows of the image sensor may be read out sequentially, rather than the entire image sensor being read out simultaneously. In such an example, a mobile device may use a row-by-row camera parameterization when performing the disclosed techniques to accommodate different actual focal lengths across scan lines of the image sensor. In this manner, the FOV correction techniques can be implemented in a manner that factors in the sequential readout of scan lines.

[0017] To further explain, an example method may be performed using a camera (e.g., a camera system that is a component of a mobile device such as a mobile phone or a DSLR camera) and may include operating the camera at an initial focal length such that the camera initially displays a preview of the scene on a display screen. To focus on a target located within the scene, the computing device may determine and perform a zoom operation that causes the camera to focus on the target. For example, after detecting a target in the scene automatically or based on user input, the camera may perform an AF sweep until it transitions to a focal length that allows it to closely focus on the target. In some examples, a target may enter the scene while the camera is already capturing a preview of the scene, which may trigger the AF technique.

[0018] For mobile devices and other types of camera devices, AF and other zoom operations may involve physically adjusting the distance between the image sensor and the lens. As a result, these adjustments in the focal length between the image sensor and the lens can cause image frames displayed by the camera to have significantly different FOVs when viewed by a user. To reduce undesirable effects associated with changing FOVs across image frames, a computing device can implement the disclosed FOV correction techniques, which may include mapping the varying real focal lengths used by the camera across image frames to a fixed virtual focal length. By determining and mapping the real focal lengths determined for successive image frames to a fixed virtual focal length, the computing device may display image frames depicting a scene in which the FOV remains constant and appears stable as the camera performs a zoom operation (e.g., an AF sweep) to focus on a target. The computing device can then display, and potentially capture, a previous zoomed image of the scene focused on the target on the display screen, with an overall smooth appearance appearing from the same fixed virtual view as the original depiction of the scene.

[0019] The disclosed FOV correction techniques may include using a fixed virtual focal length determined based on a previously generated calibration model for the camera. For example, the camera's intrinsic and extrinsic parameters may be measured and mapped to some predefined VCM sample points (or optical image stabilization (OIS)-VCM sample points). The mapping may then be stored as part of the camera's calibration model. In some cases, the camera's calibration model is generated during the manufacturing process of the mobile device associated with the camera.

[0020] When performing the disclosed FOV correction techniques, a computing device can acquire frame-based data for each image frame, such as VCM and / or OIS data along with a timestamp, while the camera system performs an AF sweep. Because each image frame represents a unit of data processing, the frame-based data can be used to determine camera geometric data as the camera adjusts the focal length during the AF sweep. The computing device may use a camera-internal interpolation and calibration model to derive the camera's actual focal length (and principal point) for each image frame. If an image frame includes multiple VCM samples with different timestamps, the computing device can use the camera-internal interpolation and calibration model to estimate camera-internal parameters based on the different timestamps.

[0021] After deriving the real focal length(s) of the image frames based on camera internal interpolation, the computing device may then warp the real focal length(s) to a fixed virtual focal length. This virtual focal length allows the image frames to appear to have a field of view that may be constant relative to preceding and subsequent image frames that are also modified for display using FOV correction techniques. In this manner, successive image frames can be displayed with an FOV that appears constant and stable, even though the image frames are actually captured by the camera when the camera is operating at different real focal lengths.

[0022] In some examples, the warping transform used by the computing device is a homography transform that enables the computing device to output a preview image of the scene that appears to be from the same viewpoint with a constant FOV, even as the camera changes its focal length in real time to focus on a target (i.e., performs an AF sweep). In addition to warping the real focal length, in some examples, the computing device may warp the principal point(s) derived for the image frames during camera-internal interpolation to a virtual principal point. As such, the computing device may iteratively apply the warping transform across scanlines to multiple image frames that occur between the initial preview of the scene and the zoomed preview of the scene as the image capture device performs a zoom operation.

[0023] In examples involving shutterless camera systems, a computing system associated with the camera system may perform camera parameter interpolation to determine the actual focal length and principal point based on geometric data (e.g., time-stamped VCM and / or OIS samples) for each scan line of an image frame. In particular, the camera may use a rolling shutter, so the actual focal length may vary from scan line to scan line. By deriving and warping the actual focal length for each row, the FOV correction technique can account for variations caused by sequential readout of scan lines.

[0024] In some examples, the computing device may determine the actual focal length based on the average focal length of the exposure interval for each scan line in the image frame. For example, in the case of a shutterless camera, the computing system may determine the focal length using sampled VCM data for each row of the image frame and determine a focal length representation that can be mapped to a virtual focal length based on the average focal length during the exposure time. In this manner, the computing device can perform per-scan line adaptation by calculating the average VCM readout for each scan line caused by the rolling shutter, and then compensate for potential delays between the VCM and the scan line. In other examples, the actual focal length for a scan line in the image frame may be determined in other ways. For example, the focal length representation for the scan line may be based on the focal length(s) at the center of the exposure interval.

[0025] In some examples, the computing system may perform a row-by-row homography and use backward meshing to refine the output image being displayed. In general, warping behavior can be described as either forward or backward. In forward form, warping can take a source location and output a destination location to which it is warped. In backward form, warping can take a destination location and output a source location from which the destination location occurs. Thus, in some examples, a computing device can use backward form warping when rendering a display when the final pixel location is known and the computing device is trying to determine where the pixel is located in the source image.

[0026] Additionally, the computing system may use one or more meshes. A mesh is a discretized representation of the warping and may consist of warped values ​​at the vertices of a grid. To query the warping from the grid, interpolation may be applied by a warping engine used by the computing system. A mesh may be used to represent the warping behavior and may be sampled on the discretized grid. The FOV-compensated warping may be determined as a function of focal length. By factoring in the rolling shuttering effect, the computing device may interpolate the focal length representation for each mesh row using previously derived camera internal samples. The computing system may then generate an FOV-compensated inverse mesh warp. The mesh may be consumed by the warping engine to obtain the FOV-compensated effect.

[0027] Additionally, the computing system may limit the FOV-compensated inverse mesh according to the zoom level. In some examples, scaling included in the FOV-compensation technique may truncate a portion of the view, thereby modifying the actual FOV of the camera. Thus, the computing system may be configured to apply the FOV-compensation technique to a portion of the zoom section rather than the entire zoom section. For example, the computing system may be able to turn off the FOV-compensation technique when capturing full-resolution images using the image capture device. In this manner, the zoom level may be used to limit the application of the FOV-compensated inverse mesh.

[0028] In some examples, the computing system may also combine the FOV-corrected inverse mesh with warping meshes from other processing techniques. For example, the computing system may perform multiple warping techniques during operation that can further refine the image output by the computing system.

[0029] The following description and accompanying drawings illustrate features of various exemplary embodiments. The embodiments provided are by way of example and not by way of limitation. Accordingly, dimensions of the drawings are not necessarily to scale.

[0030] II. Exemplary Systems As cameras become more prevalent, they may be used as standalone hardware devices or integrated into other types of devices. For example, still and video cameras are now commonly included in wireless computing devices (e.g., smartphones and tablets), laptop computers, wearable computing devices, video game interfaces, home automation devices, and automobiles and other types of vehicles. The image capture component of a camera may include one or more apertures through which light passes, one or more recording surfaces for capturing images represented by the light, and one or more lenses positioned in front of each aperture to focus at least a portion of the image onto the recording surface(s). The apertures may be fixed size or adjustable.

[0031] In an analog camera, the recording surface may be photographic film. In a digital camera, the recording surface may include an electronic image sensor (e.g., a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor) for transferring and / or storing the captured image in a data storage unit (e.g., memory). The image sensor may include an array of photosites configured to capture incident light through an aperture. When an exposure to capture an image occurs, each photosite may collect photons from the incident light and store the photons as an electrical signal. Once the exposure is complete, the camera may close each photosite and proceed to measure each photosite's electrical signal.

[0032] In that case, the signal of the image sensor's photosite array can be quantified as a digital value with a precision that can be determined by the bit depth. Bit depth can be used to quantify the number of unique colors available in the image's color palette, in terms of the number of "bits," or 0s and 1s, used to specify each color. This does not mean that the image necessarily uses all of these colors; instead, it means that the image can specify colors with that level of precision. For example, in the case of a grayscale image, bit depth can quantify the number of unique shades available. Thus, an image with a higher bit depth can encode more shades or colors, since there are more combinations of 0s and 1s available.

[0033] To capture a scene in a color image, a color filter array (CFA) placed near the image sensor can allow only one color of light to enter each photosite. For example, a digital camera may include a CFA (e.g., a Bayer array) that allows the photosites of the image sensor to capture only one of the three primary colors (red, green, and blue (RGB)). Other potential CFAs may use other color systems, such as a cyan, magenta, yellow, and black (CMYK) array. As a result, the photosites can measure the colors of the scene for subsequent display in a color image.

[0034] In some examples, a camera may utilize a Bayer array consisting of alternating rows of red-green and green-blue filters. In a Bayer array, because the human eye is more sensitive to green light than both red and blue light, each primary color receives an unequal percentage of the total area of ​​the image sensor's photosite array. In particular, redundancy due to green pixels may produce images that appear less noisy and more detailed. Therefore, when constructing a color image of a scene, the camera can approximate the other two primary colors to obtain full color at every pixel. For example, the camera may perform a Bayer demosaicing or interpolation process to convert the array of primary colors into an image containing full color information at each pixel. The Bayer demosaicing or interpolation may depend on the image format, size, and compression technology used by the camera.

[0035] One or more shutters may be coupled to or near the lens or recording surface. Each shutter may be in either a closed position, blocking light from reaching the recording surface, or an open position, allowing light to reach the recording surface. The position of each shutter may be controlled by a shutter button. For example, the shutter may be in a closed position by default. When the shutter button is triggered (e.g., pressed), the shutter may change from the closed position to the open position for a period known as a shutter cycle. During the shutter cycle, an image may be captured at the recording surface. At the end of the shutter cycle, the shutter may change back to the closed position.

[0036] Alternatively, the shuttering process may be electronic. For example, before the electronic shutter of a CCD image sensor "opens," the sensor may be reset to remove any residual signal at its photosites. While the electronic shutter remains open, the photosites may accumulate charge. When or after the shutter is closed, these charges may be transferred to longer-term data storage. A combination of mechanical and electronic shuttering may also be possible. Regardless of type, the shutter or shutters may be activated and / or controlled by something other than a shutter button. For example, the shutter(s) may be activated by a soft key, a timer, or other trigger. As used herein, the term "image capture" may refer to any mechanical and / or electronic shuttering process that may result in one or more images being recorded, regardless of how the shuttering process is triggered or controlled.

[0037] The exposure of a captured image may be determined by a combination of the size of the aperture, the brightness of the light entering the aperture, and the length of the shutter cycle (also called the shutter length or exposure length). Additionally, exposure may be affected by applying digital and / or analog gain to the image. In some embodiments, the terms "exposure length," "exposure time," or "exposure time interval" may refer to the shutter length multiplied by the gain for a particular aperture size. Thus, these terms may be used somewhat interchangeably and, in some cases, should be interpreted as shutter length, exposure time, and / or any other metric that controls the amount of signal response resulting from light reaching the recording surface.

[0038] A still camera may capture one or more images each time image capture is triggered. A video camera may continuously capture images at a particular rate (e.g., 24 images or frames per second) as long as image capture remains triggered (e.g., while the shutter button is held down). Some digital still cameras may open the shutter when the camera device or application becomes active and remain in this position until the camera device or application becomes inactive. While the shutter is open, the camera device or application may capture a representation of the scene and display it on the viewfinder. When image capture is triggered, one or more separate digital images of the current scene may be captured.

[0039] Cameras may include software for controlling one or more camera functions and / or settings, such as aperture size, exposure time, gain, etc. Additionally, some cameras may include software for digitally processing images during or after capture of those images.

[0040] As noted above, a digital camera may be a standalone device or may be integrated with other devices. As an example, FIG. 1A shows the form factor of digital camera device 100 as seen from a front view 101A and a side view 101B. Additionally, FIG. 1B also shows the form factor of digital camera device 100 as seen from a back view 101C and another back view 101D. Digital camera device 100 may also be described as a mobile device and may have the form of a mobile phone, a tablet computer, or a wearable computing device. Other embodiments are possible.

[0041] 1A and 1B, digital camera device 100 may include various elements, such as a body 102, a front-facing camera 104, a multi-element display 106, a shutter button 108, and additional buttons 110. Front-facing camera 104 may be located on the side of body 102 that typically faces the user during operation, or on the same side as multi-element display 106.

[0042] 1B, digital camera device 100 further includes rear-facing camera 112, which is shown positioned on the opposite side of body 102 from front-facing camera 104. Additionally, rear views 101C and 101D shown in FIG. 1B represent two alternative placements for rear-facing camera 112, although other placements are possible. Also, referring to a camera as front or rear is arbitrary, and digital camera device 100 may include one or more cameras positioned on various sides of body 102.

[0043] Multi-element display 106 may represent a cathode ray tube (CRT) display, a light-emitting diode (LED) display, a liquid crystal (LCD) display, a plasma display, or any other type of display known in the art. In some embodiments, multi-element display 106 may display a digital representation of the current image being captured by front-facing camera 104 and / or rear-facing camera 112, or an image that may be captured by any one or more of these cameras, or a recently captured image. Thus, multi-element display 106 may function as a camera viewfinder. Multi-element display 106 may also support touchscreen and / or presence sensing capabilities, which may allow settings and / or configurations of any aspect of digital camera device 100 to be adjusted.

[0044] The front-facing camera 104 may include an image sensor and associated optical elements (e.g., a lens) and may provide zoom capabilities or may have a fixed focal length. In other embodiments, interchangeable lenses may be used with the front-facing camera 104. The front-facing camera 104 may have a variable mechanical aperture and a mechanical and / or electronic shutter. The front-facing camera 104 may also be configured to capture still images, video images, or both. The rear-facing camera 112 may be a similar type of image capture component and may include an aperture, a lens, a recording surface, and a shutter. Specifically, the rear-facing camera 112 may operate similarly to the front-facing camera 104.

[0045] Either or both of the front-facing camera 104 and the rear-facing camera 112 may include or be associated with an illumination component that provides a light field that illuminates the target object. For example, the illumination component may provide flash or constant illumination of the target object. The illumination component may also be configured to provide a light field that includes one or more of structured light, polarized light, and light with specific spectral content. Other types of light fields known and used to recover 3D models from objects are possible within the context of the embodiments herein.

[0046] Additionally, either or both of the front-facing camera 104 and / or the rear-facing camera 112 may include or be associated with an ambient light sensor that can continuously or over time determine the ambient brightness of a scene that the camera can capture. In some devices, the ambient light sensor can be used to adjust the brightness of a screen display associated with the camera (e.g., a viewfinder). If the determined ambient brightness is high, the screen brightness level can be increased to make the screen easier to see. If the determined ambient brightness is low, the screen brightness level can be decreased, also to make the screen easier to see, but potentially to conserve power. The ambient light sensor can also be used to determine an exposure time for image capture.

[0047] Digital camera device 100 may be configured to capture images of a target object using multi-element display 106 and either front-facing camera 104 or rear-facing camera 112. The captured images may be multiple still images or a video stream. Image capture may be triggered by actuating button 108, pressing a soft key on multi-element display 106, or some other mechanism. Depending on the implementation, images may be captured automatically at specific time intervals, for example, immediately upon pressing shutter button 108, upon suitable lighting conditions for the target object, upon moving digital camera device 100 a predetermined distance, or according to a predetermined capture schedule.

[0048] In some examples, one or both of the front-facing camera 104 and the rear-facing camera 112 are calibrated monocular cameras. A monocular camera may be an image capture component configured to capture 2D images. For example, a monocular camera may use a modified refractive telescope used to magnify images of distant objects by passing light through a series of lenses and prisms. Thus, monocular cameras and / or other types of cameras may have intrinsic matrices that can be used for the depth estimation techniques presented herein. The camera's intrinsic matrices are used to convert 3D camera coordinates to 2D homogeneous image coordinates.

[0049] As noted above, the functionality of digital camera device 100 may be integrated into computing devices such as wireless computing devices, mobile phones, tablet computers, wearable computing devices, robotic devices, laptop computers, vehicle cameras, etc. For illustrative purposes, FIG. 2 is a simplified block diagram showing some of the components of an exemplary computing system 200 that may include a camera component 224.

[0050] By way of example, and without limitation, computing system 200 may be a cellular mobile phone (e.g., a smartphone), a still camera, a video camera, a computer (such as a desktop, notebook, tablet, or handheld computer), a personal digital assistant (PDA), a home automation component, a digital video recorder (DVR), a digital television, a remote control, a wearable computing device, a robotic device, a vehicle, or some other type of device with at least some image capture and / or image processing functionality. It should be understood that computing system 200 may represent a physical camera device, such as a digital camera, a particular physical hardware platform on which a camera application runs in software, or other combination of hardware and software configured to perform camera functions.

[0051] 2, computing system 200 includes a communications interface 202, a user interface 204, a processor 206, data storage 208, and a camera component 224, all of which may be communicatively linked to each other by a system bus, network, or other connection mechanism 210. Computing system 200 may include other components not shown in FIG.

[0052] Communications interface 202 may enable computing system 200 to communicate with other devices, access networks, and / or transport networks using analog or digital modulation. Thus, communications interface 202 may facilitate circuit-switched and / or packet-switched communications, such as plain old telephone service (POTS) communications and / or Internet Protocol (IP) or other packetized communications. For example, communications interface 202 may include a chipset and antenna arranged for wireless communication with a wireless access network or access point. Communications interface 202 may also take the form of or include a wired interface, such as an Ethernet, Universal Serial Bus (USB), or High-Definition Multimedia Interface (HDMI) port. Communications interface 202 may also take the form of or include a Wi-Fi, Bluetooth, Global Positioning System (GPS), or wide-area wireless interface (e.g., WiMAX or 3GPP Long Term Evolution (LTE)). However, other forms of physical layer interfaces and other types of standard or proprietary communication protocols may be used via communication interface 202. Additionally, communication interface 202 may include multiple physical communication interfaces (e.g., a Wi-Fi interface, a BLUETOOTH interface, and a wide area wireless interface).

[0053] The user interface 204 may function to enable the computing system 200 to interact with a human or non-human user, such as by receiving input from the user and providing output to the user. Accordingly, the user interface 204 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensing panel, computer mouse, trackball, joystick, microphone, etc. The user interface 204 may also include one or more output components, such as one or more display screens that may be combined with a presence-sensing panel. The display screens may be based on CRT, LCD, and / or LED technology, or other technologies now known or later developed. The user interface 204 may also be configured to generate audible output(s) via speakers, speaker jacks, audio output ports, audio output devices, earphones, and / or other similar devices.

[0054] In some embodiments, user interface 204 may include a display that functions as a viewfinder for still camera and / or video camera functions supported by computing system 200. Additionally, user interface 204 may include one or more buttons, switches, knobs, and / or dials that facilitate configuring and focusing camera functions and capturing images (e.g., capturing photographs). Some or all of these buttons, switches, knobs, and / or dials may be capable of being implemented by a presence-sensing panel.

[0055] Processor 206 may include one or more general-purpose processors, such as microprocessors, and / or one or more special-purpose processors, such as digital signal processors (DSPs), graphics processing units (GPUs), floating-point units (FPUs), network processors, or application-specific integrated circuits (ASICs). In some cases, the special-purpose processors may be capable of image processing, image alignment, and image combining, among other possibilities. Data storage 208 may include one or more volatile and / or non-volatile storage components, such as magnetic, optical, flash, or organic storage, and may be wholly or partially integrated with processor 206. Data storage 208 may include removable and / or non-removable components.

[0056] Processor 206 may be capable of executing program instructions 218 (e.g., compiled or non-compiled program logic and / or machine code) stored in data storage 208 to perform various functions described herein. Thus, data storage 208 may include a non-transitory computer-readable medium having stored thereon program instructions that, when executed by computing system 200, cause computing system 200 to perform any of the methods, processes, or operations disclosed herein and / or in the accompanying drawings. Execution of program instructions 218 by processor 206 may result in processor 206 using data 212.

[0057] By way of example, program instructions 218 may include an operating system 222 (e.g., an operating system kernel, device driver(s), and / or other modules) and one or more application programs 220 (e.g., camera functionality, address book, email, web browsing, social networking, image applications, and / or game applications) installed on computing system 200. Similarly, data 212 may include operating system data 216 and application data 214. Operating system data 216 may be primarily accessible to operating system 222, while application data 214 may be primarily accessible to one or more of application programs 220. Application data 214 may be located in a file system that is visible or hidden to a user of computing system 200.

[0058] Application programs 220 may communicate with operating system 222 through one or more application programming interfaces (APIs). These APIs may facilitate, for example, application programs 220 to read and / or write application data 214, send or receive information via communications interface 202, receive and / or display information on user interface 204, etc.

[0059] In some terms, application program 220 may be referred to as an "app" for short. Additionally, application program 220 may be downloadable to computing system 200 through one or more online application stores or application markets. However, application programs may also be installed on computing system 200 in other ways, such as through a web browser or through a physical interface of computing system 200 (e.g., a USB port).

[0060] The camera component 224 may include, but is not limited to, an aperture, a shutter, a recording surface (e.g., photographic film and / or an image sensor), a lens, and / or a shutter button. Thus, the camera component 224 may be controlled at least in part by software executed by the processor 206. In some examples, the camera component 224 may include one or more image capture components, such as a monocular camera. While the camera component 224 is shown as part of the computing system 200, it may be physically separate in other embodiments. For example, the camera component 224 may capture and provide images via a wired or wireless connection to the computing system 200 for subsequent processing.

[0061] 3 is a simplified representation of an image capture component 300 capturing an image of a person 306. The image capture component 300 includes a recording surface 302 (image sensor) and a lens 304, and may include other components not shown. During image capture, light representing the person 306 and other elements of the scene (not shown) may pass through the lens 304, allowing the image capture component 300 to subsequently create an image of the person 306 on the recording surface 302. As a result, a display interface connected to the image capture component 300 may display a digital image of the person 306. In the embodiment shown in FIG. 3, the image of the person 306 appears upside down on the recording surface 302 due to the optical properties of the lens 304, and image processing techniques can invert the image for display.

[0062] In some camera configurations, the lens 304 may be adjustable. For example, the lens 304 may move left and right, thereby changing the camera's focal distance for image capture. The adjustment may be made by applying voltage to a motor (not shown in FIG. 3 ) that controls the position of the lens 304 relative to the recording surface 302, allowing the camera to focus on the person 306 over a range of distances. The distance between the lens 304 and the recording surface 302 at any given time may be referred to as the focal length, which may be measured in millimeters or other units. In turn, the distance between the lens 304 and its focal area may be referred to as the focal distance, which may similarly be measured in millimeters or other units.

[0063] FIG. 4 illustrates imaging hardware that performs a zoom operation. In an exemplary embodiment, a focal length 402 is shown as the distance between a lens 404 and an image sensor 406 of a camera 400. As the camera 400 performs an AF sweep or other zoom operation, the focal length 402 changes, thereby adjusting the camera's FOV 408. A motor or other technique can be used to move the image sensor 406 relative to the lens 404 to adjust the focal length 402. The mechanical system of the camera 400 shown in FIG. 4 is coupled with AF software that helps the camera 400 automatically detect where to focus in a scene.

[0064] The intrinsic matrix of the camera 400 may be expressed as:

number

[0065] 5 illustrates a mobile device 500 that may perform the FOV correction techniques disclosed herein. The mobile device 500 may take the form of a smartphone or other type of device that includes an image capture device 502 and associated components for capturing images. In some examples, the mobile device 500 may be implemented as the digital camera device 100 shown in FIGS. 1A-1B and / or may include components of the computing system 200 shown in FIG. 2.

[0066] In an exemplary embodiment, mobile device 500 includes an imaging capture device 502, a processor 504, a display screen 506, and data storage 508. The data storage may include a camera parameter interpolator 510, a row interior interpolator 512, and a calibration model 514. Data storage 508 may also store other data, such as instructions for performing the disclosed FOV correction techniques.

[0067] Mobile device 500 may implement the disclosed FOV compensation techniques to reduce undesirable visual artifacts when performing an AF sweep or other zoom-related operation that involves automatic adjustment of the distance between the image sensor and one or more lenses of image capture device 502. For example, when the image capture device is displaying a preview of a scene on a display screen, if a target moves within the FOV of the image capture device, processor 504 or other component may cause image capture device 502 to focus on the target. To keep the FOV of image frames displayed on display screen 506 constant when image capture device 502 performs AF, mobile device 500 may use a virtual focal length that enables the displayed image frames to have a constant FOV.

[0068] The mobile device 500 can use the frame metadata 516 to stabilize the FOV between successive frames by correcting the real focal length(s) of the image capture device 502 in each image frame by warping the image from the real focal length(s) to a fixed virtual focal length. In this way, the images are captured at substantially the same virtual focal length. The warp may be a homography transformation that warps the frames from the real focal length to the virtual focal length. The homography may allow the image frames to be shifted from one view to another view of the same scene. Thus, the warp transformation may be expressed as:

number

[0069] In the warp shown in Equation 2, K real (t) represents the camera intrinsic matrix at time (t), and f real (t) is the focal length at time (t), and the optical center at time (t) is o x (t) and o y (t) and f virtual represents the time-independent virtual focal length. As a result, K virtual (t) represents the camera intrinsic matrix with the focal length replaced by the virtual focal length. As shown, the warp is x (t),o y (t)] to the ratio f virtual / f real This is equivalent to scaling by (t).

[0070] When performing the disclosed FOV correction techniques, the mobile device 500 may obtain and use frame metadata 516 when the image capture device 502 performs a zoom operation (e.g., an AF sweep) to capture image frame data depicting a scene. The frame metadata 516 may include time-stamped VCM samples and / or time-stamped optical image stabilization (OIS) samples, which may be used by the camera parameter interpolator 510 to generate time-stamped camera internal data. The camera parameter interpolator 510 may use the calibration model 514 to output the real focal lengths and principal points of different image frames, which may warp the real focal lengths of the image frames to a virtual focal length, which can then be displayed by the display screen 506 as an image preview with a constant FOV.

[0071] The mobile device 500 can also apply the disclosed techniques when the image capture device 502 is shutterless. Specifically, when the image capture device 502 uses an electronic rolling shutter, each image row (or column) may be read out sequentially. The mobile device 500 can use the row internal interpolator 512 to generate f real (t) and optical center [o x (t),o y For example, f in row (i) of the image can be considered. real (t) and optical center [o x (t),o y (t)] are expressed as f real (i), o x (i), o y (i) By analyzing the focal length and optical center of the image rows, the mobile device 500 factors in, in some instances, rolling shutter skew time that occurs due to the method of reading out the image.

[0072] In some examples, the mobile device 500 is configured to perform a row-by-row homography and apply a backward mesh. For example, if the mobile device 500 is trying to maintain a constant optical center, the mobile device may use a forward mesh and a backward mesh as follows:

number

[0073] In equations 3 and 4, f v represents the virtual focal length, f(y) represents the real focal length, and p x,y is a vector (x,y) representing the input point position, and p oc is the vector representing the optical center (o x (i),o y(i)). The forward mesh shown in Equation 3 may be used by a computing system. Specifically, given a source location, Equation 3 may be used to output a destination location to which the pixel will be warped. The backward mesh shown in Equation 4 may be used by a computing system in some examples. Given a destination location, the backward mesh shown in Equation 4 may output the source location from which the destination location originated. For example, the computing system may use the backward mesh to render the display because it has data indicating where the final pixel should be displayed and is trying to know where it is on the source image.

[0074] In some examples, the mobile device 500 may use a dynamic setting to turn FOV compensation techniques on or off based on the zoom level. For example, a limit on warping may be expressed as:

number

number

[0075] Thus, the FOV-corrected inverse mesh can be combined with warping meshes from other processes. In some instances, meshes may be concatenated sequentially. For example, meshes from other processing techniques may provide functions such as lens distortion correction, stabilization, and facial distortion removal.

[0076] 6 illustrates a comparison of camera views with and without FOV rolling shutter correction. Specifically, comparison 600 shows actual camera view 602, which represents a display that mobile device 500 may output without applying FOV rolling shutter correction technique, and virtual camera view 604 after applying FOV rolling shutter correction technique 606. In comparison 600, the different outputs indicate scaling differences per scanline. As shown, a curved line 608 in actual camera view 602 may become a straight line 610 after applying FOV rolling shutter correction technique 606 as a representation of scaling differences per scanline.

[0077] III. Exemplary Methods 7 is a flowchart according to an example embodiment. The embodiment illustrated by FIG. 7 may be implemented by a computing system such as the digital camera device 100 shown in FIG. 1 or the mobile device 500 shown in FIG. 5. However, the embodiment may also be implemented by other types of devices or device subsystems, such as by a computing system located remotely from the camera. Furthermore, the embodiment may be combined with any aspect or feature disclosed in this specification or the accompanying drawings.

[0078] At block 702, the method 700 includes displaying, on a display screen of the computing device, an initial preview of a scene being captured by an image capture device of the computing device. The image capture device is operating at an initial focal length when capturing the initial preview of the scene. In some examples, the image capture device is a shutterless camera system.

[0079] At block 704, the method 700 includes the computing device determining a zoom operation configured to cause the image capture device to focus on the target. In some examples, the image capture device is configured to change a focal length when performing the zoom operation. For example, the computing device may cause the image capture device to perform an AF technique to focus on the target.

[0080] At block 706, method 700 includes mapping the focal length used by the image capture device to a virtual focal length so that the field of view of the scene remains constant across image frames displayed by the display screen between an initial preview of the scene and a zoom preview of the scene focused on the target while the image capture device performs a zoom operation. In some examples, the computing system may determine the virtual focal length based on the initial focal length. In other examples, the computing system may obtain a calibration model of the image capture device and determine the virtual focal length based on the calibration model of the image capture device. The computing system may then calculate a scaling ratio between the given focal length of the image frame and the virtual focal length and then apply the scaling ratio to map the focal length to the virtual focal length.

[0081] In some examples, the computing system may acquire frame-based data for each image frame while the image capture device performs a zoom operation and determine geometric data for the image capture device based on the frame-based data for each image frame. The frame-based data may include VCM data in some examples. In other examples, the frame-based data may further include OIS data. Accordingly, the computing system may then apply a warping transform configured to map the focal lengths determined for the image frames to a virtual focal length, where the focal length is determined for the image frames based on the geometric data corresponding to the image frames.

[0082] In some examples, mapping the focal lengths includes determining a real focal length used by the image capture device for the image frame based on VCM data corresponding to the image frame, and applying a warping transform that maps the determined real focal length for the image to a virtual focal length. Determining the real focal lengths may include determining a set of real focal lengths corresponding to scan lines in the image frame. For example, the real focal lengths for the scan lines in the image frame may be determined based on an average focal length of an exposure interval for the scan lines. In other examples, the real focal lengths for the scan lines in the image frame may be determined based on a given focal length at the center of an exposure interval for the scan lines. Thus, the computing system can then apply a warping transform to map each real focal length from the set of real focal lengths to a virtual focal length.

[0083] In some examples, the computing system may obtain frame-based data representing internal parameters corresponding to the image capture device. The frame-based data may include a timestamp. Thus, the computing system may then interpolate a focal length representation for each mesh row based on the frame-based data. In some examples, the computing system may generate an inverse mesh warp based on the focal length representation for each mesh row and apply the inverse mesh warp to a given image frame. The process may be performed iteratively.

[0084] In some examples, the computing system can detect a target within a scene based on one or more visual features of one or more image frames being captured by an image capture device, the one or more image frames following an initial preview of the scene, and the computing system can determine a zoom operation in response to detecting the target.

[0085] At block 708, the method 700 includes a display screen of the computing device displaying a zoom preview of the scene focused on the target. In some examples, the computing system may display image frames between the initial preview of the scene and the zoom preview while the image capture device performs a zoom operation. Applying a warping transformation may reduce one or more visual artifacts that occur when the image capture device performs a zoom operation.

[0086] In some examples, the computing system may generate, for each frame, a bundle adjustment to apply to one or more camera calibrations and one or more focus distances. Then, for a set of consecutive frames, the computing system may generate a revised bundle adjustment based on each bundle adjustment for the consecutive frames. The computing system may also detect one or more visual features in the initial preview and the zoomed preview, and then generate an image-based visual correspondence between the initial preview and the zoomed preview based on the one or more visual features.

[0087] In some examples, the computing system may determine an average focal length over an exposure interval of an image frame and apply a warping transformation to map the average focal length over the exposure interval of the image frame to a virtual focal length. As an example, FIG. 8A illustrates a focal length representation based on the average focal length over the exposure interval. The computing device may interpolate the focal length representation for each mesh row by determining the focal length representation based on the average focal length over the exposure interval.

[0088] 8A , a graph 800 plots focal length on a Y-axis 802 against time on an X-axis 804 using focal length samples 806. Exposure times 808 of image frames are shown with rows arranged against time representing an example of a rolling shutter. As further shown, a computing device may interpolate the focal length in row (n) 809 based on the average focal length in a region 810 extending over the exposure time 808.

[0089] In some examples, the computing system may determine a focal length representation for an image frame based on a median exposure level. For example, FIG. 8B shows a focal length representation determined based on a focal length at the center of an exposure interval. The computing device may interpolate a focal length representation for each mesh row by determining the focal length representation based on the median focal length in the exposure interval.

[0090] 8B , graph 820 is similar to graph 800, with focal length represented on the Y-axis 802 and time represented on the X-axis 804. Graph 820 further includes identical focal length samples 806, each having a time-dependent focal length. As shown in FIG. 8B , in some examples, the computing device may interpolate the focal length of row (n) 809 based on the focal length 822 determined for the median exposure interval 808. IV. Conclusion

[0091] The present disclosure should not be limited in terms of the specific embodiments described herein, which are intended as illustrations of various aspects. As will be apparent to those skilled in the art, many modifications and variations can be made without departing from its scope. Functionally equivalent methods and apparatuses within the scope of the present disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing description. Such modifications and variations are intended to fall within the scope of the appended claims.

[0092] The foregoing detailed description, with reference to the accompanying drawings, describes various features and functions of the disclosed systems, devices, and methods. The illustrative embodiments set forth in the specification and drawings are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that aspects of the present disclosure, as generally described herein and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are expressly contemplated herein.

[0093] With respect to any or all of the message flow diagrams, scenarios, and flowcharts in the figures and as described herein, each step, block, and / or communication may represent the processing of information and / or the transmission of information according to the exemplary embodiments. Alternative embodiments are included within the scope of these exemplary embodiments. In these alternative embodiments, for example, functions described as steps, blocks, transmissions, communications, requests, responses, and / or messages may be performed out of the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functionality involved. Furthermore, more or fewer blocks and / or functions may be used in any of the ladder diagrams, scenarios, and flowcharts discussed herein, and these ladder diagrams, scenarios, and flowcharts may be combined with each other, either in part or in whole.

[0094] Steps or blocks representing the processing of information may correspond to circuitry that can be configured to perform specific logical functions of the methods or techniques described herein. Alternatively or additionally, steps or blocks representing the processing of information may correspond to modules, segments, or portions of program code (including associated data). The program code may include one or more instructions executable by a processor to implement specific logical functions or operations in the method or technique. The program code and / or associated data may be stored on any type of computer-readable medium, such as a storage device, including a disk, hard drive, or other storage medium.

[0095] Computer-readable media may also include non-transitory computer-readable media, such as register memory, processor cache, and computer-readable media that store data for a short period of time, such as random access memory (RAM). Computer-readable media may also include non-transitory computer-readable media that store program code and / or data for a long period of time. Thus, computer-readable media may include, for example, secondary or persistent long-term storage, such as read-only memory (ROM), optical or magnetic disks, compact disk read-only memory (CD-ROM), etc. Computer-readable media may also be any other volatile or non-volatile storage system. Computer-readable media may be considered, for example, to be a computer-readable storage medium or a tangible storage device.

[0096] Additionally, steps or blocks representing one or more information transmissions may correspond to information transmissions between software and / or hardware modules within the same physical device, although other information transmissions may be between software and / or hardware modules in different physical devices.

[0097] The particular arrangement shown in the drawings should not be considered limiting. It should be understood that other embodiments may include more or less of each element shown in a given drawing. Furthermore, some of the illustrated elements may be combined or omitted. Furthermore, example embodiments may include elements that are not shown.

[0098] Furthermore, any recitation of elements, blocks, or steps in the specification or claims is for clarity purposes only, and therefore, such recitation should not be construed as requiring or implying that these elements, blocks, or steps be adhered to a particular arrangement or performed in a particular order.

[0099] While various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, the true scope being indicated by the following claims.

Claims

1. 1. A computer-implemented method comprising: a display screen of a computing device displaying an initial preview of a scene being captured by an image capture device of the computing device, the image capture device operating at an initial focal length when capturing the initial preview of the scene, the method further comprising: The method further includes determining a zoom operation configured to cause the imaging capture device to focus on a target, the imaging capture device being configured to change a focal length when performing the zoom operation. mapping a focal length used by the imaging capture device to a virtual focal length such that a field of view of the scene remains constant across image frames displayed by the display screen between the initial preview of the scene and a zoom preview of the scene focused on the target while the image capture device performs the zoom operation; the display screen displays the zoomed preview of the scene focused on the target; A method comprising:

2. acquiring frame-based data for each image frame while the image capture device performs the zoom operation; determining geometric data for the image capture device based on the frame-based data for each image frame; The method of claim 1 further comprising:

3. The method of claim 2 , wherein the frame-based data includes voice coil motor (VCM) data.

4. Mapping the focal length used by the image capture device to the virtual focal length comprises: determining an actual focal length used by the image capture device for an image frame based on the VCM data corresponding to the image frame; applying a warping transformation that maps the real focal length determined for the image frame to the virtual focal length; The method of claim 3, comprising:

5. Determining the actual focal length comprises:

5. The method of claim 4, comprising determining a set of actual focal lengths corresponding to scan lines in the image frame, the actual focal lengths for the scan lines in the image frame being determined based on an average focal length between exposure intervals of the scan lines.

6. Applying the warping transformation The method of claim 5 , comprising applying the warping transformation to map each real focal length from the set of real focal lengths to the virtual focal length.

7. Determining the actual focal length comprises:

5. The method of claim 4, further comprising determining a set of actual focal lengths corresponding to scan lines in the image frame, the actual focal lengths for the scan lines in the image frame being determined based on a given focal length at a center of an exposure interval of the scan line.

8. Applying the warping transformation The method of claim 7 , comprising applying the warping transformation to map each real focal length from the set of real focal lengths to the virtual focal length.

9. Determining the zoom operation configured to cause the image capture device to focus on the target includes: The method of claim 1 , comprising causing the image capture device to perform an autofocus (AF) technique.

10. acquiring frame-based data representative of internal parameters corresponding to the image capture device, the frame-based data including a timestamp, the method further comprising: The method of claim 1 , comprising interpolating a focal length representation for each mesh row based on the frame-based data.

11. Interpolating the focal length representation for each mesh row comprises: The method of claim 10 , comprising determining the focal length representation based on an average focal length over an exposure interval.

12. Interpolating the focal length representation for each mesh row comprises: The method of claim 10, comprising determining the focal length representation based on a central focal length in an exposure interval.

13. Mapping a focal length used by the image capture device to the virtual focal length comprises: generating an inverse mesh warp based on said focal length representation for each mesh row; applying said inverse mesh warp to a given image frame; The method of claim 10, comprising:

14. detecting the target within the scene based on one or more visual features of one or more image frames being captured by the image capture device, the one or more image frames following the initial preview of the scene; Determining the zoom operation includes: The method of claim 1 , comprising determining the zoom movement in response to detecting the target.

15. The method of claim 1 , further comprising determining the virtual focal length based on the initial focal length.

16. obtaining a calibration model of the image capture device; determining the virtual focal length based on the calibration model of the image capture device; The method of claim 1 further comprising:

17. Mapping a focal length used by the image capture device to the virtual focal length comprises: Calculating a scaling ratio between a given focal length of an image frame and the virtual focal length; applying said scaling ratio; 17. The method of claim 16, comprising:

18. A mobile device, A display screen; an image capture device; one or more processors; and a data storage storing computer-executable instructions that, when executed by the one or more processors, cause the mobile device to perform functions, the functions comprising: the display screen displaying an initial preview of a scene being captured by the image capture device, the image capture device operating at an initial focal length when capturing the initial preview of the scene, the function further comprising: determining a zoom operation configured to cause the imaging capture device to focus on a target, the imaging capture device being configured to change a focal length when performing the zoom operation, the function further comprising: mapping a focal length used by the imaging capture device to a virtual focal length such that a field of view of the scene remains constant across image frames displayed by the display screen between the initial preview of the scene and a zoom preview of the scene focused on the target while the image capture device performs the zoom operation; the display screen displays the zoomed preview of the scene focused on the target; Including mobile devices.

19. The mobile device of claim 18 , wherein the image capture device is a shutterless camera system.

20. 1. A non-transitory computer-readable medium comprising program instructions executable by one or more processors to cause the one or more processors to perform operations, the operations comprising: a display screen displaying an initial preview of a scene being captured by an image capture device, the image capture device operating at an initial focal length when capturing the initial preview of the scene, the operation further comprising: determining a zoom operation configured to cause the imaging capture device to focus on a target, the imaging capture device configured to change a focal length when performing the zoom operation, the operation further comprising: mapping a focal length used by the imaging capture device to a virtual focal length such that a field of view of the scene remains constant across image frames displayed by the display screen between the initial preview of the scene and a zoom preview of the scene focused on the target while the image capture device performs the zoom operation; the display screen displays the zoomed preview of the scene focused on the target; 1. A non-transitory computer-readable medium comprising:

Images