FIELD OF VIEW CORRECTION TECHNIQUES FOR ELECTRONIC SHUTTER CAMERA SYSTEMS

Abstract

Embodiments relate to field of view correction techniques for shutterless camera systems. A mobile device displaying a first preview of a subject captured by an image capture device of the computing device may perform a zoom operation configured to cause the image capture device to focus on a target. The image capture device is configured to change the focal length when the zoom operation is performed. Concurrent with the image capture device performing the zoom operation, the computing device may then map the focal lengths used by the image capture device to a virtual focal length such that the field of view of the subject remains consistent across the image frames of the first preview of the subject and the zoomed preview of the subject focused on the target displayed on the display screen, and display the zoomed preview of the subject focused on the target.

Claims

[1] A computer-implemented method comprising: Displaying a first preview of a subject captured by an image capture device of the computing device on a display screen of the computing device, wherein the image capture device operates at an initial focal length during the capture of the initial subject preview; Determining, by the computing device, a zoom operation configured to cause the image capture device to focus on a target, wherein the image capture device is configured to change the focal length when the zoom operation is performed; simultaneously with the zooming operation by the image capture device, mapping the focal lengths used by the image capture device to a virtual focal length such that the field of view of the subject remains consistent across the image frames of the first preview of the subject and the zoomed preview of the subject focusing on the target displayed on the display screen; and Display the zoomed, target-focused preview of the subject through the display screen. [2] The method of claim 1, further comprising: for each image frame, obtaining frame-based data while the image capture device performs the zooming operation; and Determining geometric data for the image capture device based on the frame-based data for each image frame. [3] The method of claim 2, wherein frame-based data also includes voice coil motor (VCM) data. [4] The method of claim 3, wherein mapping the focal lengths used by the image capture device to the virtual focal length comprises: Determining a real focal length used by the image capture device for an image frame based on the VCM data corresponding to the image frame; and Apply a distortion transformation that maps the real focal length specified for the image frame to the virtual focal length. [5] The method of claim 4, wherein determining the real focal length comprises: Determining a set of real focal lengths corresponding to image lines in the image frame, wherein a real focal length for an image line in the image frame is determined based on an average focal length of an exposure interval for the image line. [6] The method of claim 5, wherein applying the distortion transformation comprises: Apply the distortion transformation to map each real focal length from the set of real focal lengths to the virtual focal length. [7] The method of claim 4, wherein determining the real focal length comprises: Determining a set of real focal lengths corresponding to image lines in the image frame, wherein a real focal length for an image line in the image frame is determined based on a given focal length at halftime of an exposure interval for the image line. [8] The method of claim 7, wherein applying the distortion transformation comprises: Apply the distortion transformation to map each real focal length from the set of real focal lengths to the virtual focal length. [9] The method of claim 1, wherein determining the zoom operation configured to cause the image capture device to focus on the target comprises: Causing the image capture device to perform autofocus (AF) using appropriate technology. [10] The method of claim 1, further comprising: Obtaining frame-based data representing intrinsic parameters corresponding to the image capture device, wherein frame-based data includes timestamps; and Interpolate a focal length representation per grid row based on the frame-based data. [11] The method of claim 10, wherein interpolating the focal length representation per grid line comprises: Determine the focal length representation based on an average focal length of an exposure interval. [12] The method of claim 10, wherein interpolating the focal length representation per grid line comprises: Determine the focal length representation based on the focal length at halftime of an exposure interval. [13] The method of claim 10, wherein mapping the focal lengths used by the image capture device to the virtual focal length comprises: Generating a backward grating distortion based on the focal length representation per grating line; and Applying the backward mesh warp to a given image frame. [14] The method of claim 1, further comprising: Detecting the target in the subject based on one or more visual features in one or more image frames captured by the image capture device, wherein the image frame(s) follow the initial preview of the scene; and wherein determining the zoom operation comprises: Determining the zoom action in response to detecting the target. [15] The method of claim 1, further comprising: Determine the virtual focal length based on the initial focal length. [16] The method of claim 1, further comprising: Obtaining a calibration model for the image acquisition device; and Determine the virtual focal length based on the calibration model for the image capture device. [17] The method of claim 16, wherein mapping the focal lengths used by the image capture device to the virtual focal length comprises: Calculating a scaling ratio between a given focal length for an image frame and the virtual focal length; and Applying the scaling ratio. [18] A mobile device comprising: a display screen; an image capture device; one or more processors; and A data storage device, the data storage device having stored thereon computer-executable instructions that, when executed by the processor(s), cause the mobile device to perform functions including: Displaying a first preview of a subject captured by an image capture device of the computing device on a display screen, wherein the image capture device operates at an initial focal length during the capture of the initial subject preview; Determining a zoom operation configured to cause the image capture device to focus on a target, wherein the image capture device is configured to change the focal length when the zoom operation is performed; simultaneously with the zooming operation by the image capture device, mapping the focal lengths used by the image capture device to a virtual focal length such that the field of view of the subject remains consistent across the image frames of the first preview of the subject and the zoomed preview of the subject focusing on the target displayed on the display screen; and Display the zoomed, target-focused preview of the subject through the display screen. [19] The mobile device of claim 18, wherein the image capture device is a camera system with an electronic shutter (and as such, a “shutterless” camera system). [20] A non-transitory computer-readable medium comprising program instructions executable by one or more processors to cause the one or more processors to perform operations comprising: Displaying a first preview of a subject captured by an image capture device of the computing device on a display screen, wherein the image capture device operates at an initial focal length during the capture of the initial subject preview; Determining a zoom operation configured to cause the image capture device to focus on a target, wherein the image capture device is configured to change the focal length when the zoom operation is performed; simultaneously with the zooming operation by the image capture device, mapping the focal lengths used by the image capture device to a virtual focal length such that the field of view of the subject remains consistent across the image frames of the first preview of the subject and the zoomed preview of the subject focusing on the target displayed on the display screen; and Display the zoomed, target-focused preview of the subject through the display screen.

Images