Image stabilization control for variable optical zoom imaging systems

Abstract

This disclosure provides systems, methods, and devices for image signal processing that support image stabilization of a variable optical zoom (VOZ) system. In a first aspect, an image processing method includes receiving a first plurality of images from a VOZ system, the first plurality of images captured at a first lens position and stabilized to a target zoom level using a first image stabilization process. Movement information of the VOZ system may exceed a predetermined threshold. In response, a second lens position may be determined, and a second image frame may be captured at the second lens position and received from the VOZ system. The second image frame may then be stabilized to the target zoom level using the first image stabilization process or a different second image stabilization process. Other aspects and features are also claimed and described.

Description

Cross Reference to Related Applications

[0001] This application claims the benefit of Indian Patent Application No. 202341060618, filed September 8, 2023, entitled “IMAGE STABILIZATION CONTROL FOR VARIABLE OPTICAL ZOOM IMAGING SYSTEMS,” which is expressly incorporated by reference herein in its entirety. TECHNICAL FIELD

[0002] Aspects of the present disclosure relate generally to image processing, and more specifically, to stabilizing image frames captured by a variable optical zoom (VOZ) system. Some features can enable and provide improved image processing, including improved stabilization of image frames captured by a VOZ system using optical image stabilization, electronic image stabilization, or a combination thereof. BACKGROUND

[0003] An image capture device is a device that can capture one or more digital images, whether still images for photographs or sequences of images for videos. Capture devices can be incorporated into a variety of devices. By way of example, an image capture device can comprise a standalone digital camera or digital video camera, a wireless communication device handset (such as a mobile telephone, cellular or satellite radio telephone) equipped with a camera, a personal digital assistant (PDA), a panel or tablet device, a gaming device, a computing device (such as a web camera, video surveillance camera), or other device having digital imaging or video capabilities. SUMMARY

[0004] The following summary of some aspects of the present disclosure is intended to provide an overview of the subject technology so as to provide a basic understanding of the discussed technology. This summary is not an extensive overview of all contemplated features of the present disclosure, and is not intended to identify key or critical elements of all aspects of the present disclosure nor is it intended to delineate the scope of any or all aspects of the present disclosure. Its sole purpose is to present some concepts of one or more aspects of the present disclosure in a general framework so as to provide an overview to the more detailed description to come later.

[0005] In some aspects, techniques for improving image stabilization of image frames captured by a VOZ system are provided. In certain aspects, movement information of the image capture device can be used to select between various image stabilization processes, adjust the image stabilization processes, or a combination thereof. In one particular aspect, a VOZ system can capture image frames at a first lens position and can stabilize these image frames to a target zoom level using a first image stabilization process. Movement information of the VOZ system can indicate that the VOZ system moved too much to properly stabilize the image frames. For example, the movement information can exceed a predetermined threshold associated with the first image stabilization process. A second lens position can be selected based on the movement information such that the image frames can be properly stabilized again. Subsequent image frames can then be stabilized from a second zoom level associated with the second lens position to the target zoom level. In certain cases, the first image stabilization process can be used to stabilize the image frames. In additional cases, a second, different image stabilization process can be used to stabilize these image frames.

[0006] In one aspect, a method includes receiving first image frames from a variable optical zoom (VOZ) system, the first image frames being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frames are stabilized at a target zoom level using a first image stabilization process; determining that movement information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the movement information exceeding the predetermined threshold; receiving second image frames captured by the VOZ system at the second lens position; and stabilizing the second image frames by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process to determine output image frames at the target zoom level based on the second image frames.

[0007] In additional aspects, a system includes a processor and a memory storing instructions that, when executed by the processor, cause the processor to perform operations comprising receiving first image frames from a variable optical zoom (VOZ) system, the first image frames being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frames are stabilized at a target zoom level using a first image stabilization process; determining that movement information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the movement information exceeding the predetermined threshold; receiving second image frames captured by the VOZ system at the second lens position; and stabilizing the second image frames by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process to determine output image frames at the target zoom level based on the second image frames.

[0008] In another aspect, an image capture device includes an image sensor; a memory storing processor-readable code; and at least one processor coupled to the memory and the image sensor. The at least one processor can be configured to execute the processor-readable code to cause the at least one processor to perform operations including: receiving a first image frame from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; determining that movement information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the movement information exceeding the predetermined threshold; receiving a second image frame captured by the VOZ system at the second lens position; and stabilizing the second image frame by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining an output image frame at the target zoom level based on the second image frame.

[0009] In another aspect, a non-transitory computer-readable medium stores instructions that, when executed by a processor, cause the processor to perform operations including: receiving a first image frame from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; determining that movement information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the movement information exceeding the predetermined threshold; receiving a second image frame captured by the VOZ system at the second zoom level; and stabilizing the second image frame by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining an output image frame at the target zoom level based on the second image frame.

[0010] The image processing methods described herein can be performed by an image capture device and / or on image data captured by one or more image capture devices. Image capture devices, which can capture one or more digital images, whether still image photographs or image sequences of a video, can be incorporated into a variety of devices. By way of example, image capture devices can include standalone digital cameras or digital video cameras, camera-equipped wireless communication devices handsets such as mobile telephones, cellular or satellite radio telephones, personal digital assistants (PDAs), panel or tablet devices, gaming devices, computing devices such as webcams, video surveillance cameras, or other devices having digital imaging or video capabilities.

[0011] The image processing techniques described herein can involve a digital camera having an image sensor and processing circuitry (e.g., an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a graphics processing unit (GPU), or a central processing unit (CPU)). An image signal processor (ISP) can include one or more of these processing circuits and be configured to perform operations to obtain image data for processing in accordance with the image processing techniques described herein and / or involved in the image processing techniques described herein. The ISP can be configured to control the capture of image frames from one or more image sensors and determine one or more image frames from the one or more image sensors to generate a view of a scene in an output image frame. The output image frame can be part of a sequence of image frames forming a video sequence. The video sequence can include other image frames received from the image sensor or other image sensors.

[0012] In an example application, an image signal processor (ISP) can receive instructions for capturing a sequence of image frames in response to loading of software, such as a camera application, to produce a preview display from an image capture device. The image signal processor can be configured to produce a single stream of output image frames based on image frames received from one or more image sensors. The single stream of output image frames can include raw image data from the image sensor, merged image data from the image sensor, or corrected image data processed by one or more algorithms within the image signal processor. For example, the image frames can be processed by an image post-processing engine (IPE) and / or other image processing circuitry to perform one or more of tone mapping, portrait lighting, contrast enhancement, gamma correction, etc. in the image signal processor processing the image frames obtained from the image sensor (which can have already performed some processing on the data prior to output to the image signal processor). The output image frames from the ISP can be stored in memory and retrieved by an application processor executing the camera application, which can perform further processing on the output image frames to adjust the appearance of the output image frames and render the output image frames on a display for viewing by a user.

[0013] After an output image frame representing a scene is determined by an image signal processor and / or an application processor (such as by the image processing techniques described in the various embodiments herein), the output image frame can be displayed on a device display as a single still image and / or as part of a video sequence, saved to a storage device as a picture or video sequence, transmitted over a network, and / or printed to an output medium. For example, an image signal processor (ISP) can be configured to obtain input frames of image data (e.g., pixel values) from one or more image sensors, and in turn produce corresponding output image frames (e.g., preview display frames, still image captures, frames for video, frames for object tracking, etc.). In other examples, the image signal processor can output image frames to various output devices and / or camera modules for further processing, such as for 3A parameter synchronization (e.g., auto-focus (AF), auto-white balance (AWB), and auto-exposure control (AEC)), producing video files via output frames, configuring frames for display, configuring frames for storage, transmitting frames over a network connection, etc. Generally, an image signal processor (ISP) can obtain incoming frames from one or more image sensors, as well as produce and output a stream of output frames to various output destinations.

[0014] In some aspects, output image frames can be produced by combining aspects of the image correction of the present disclosure with other computational photography techniques, such as high dynamic range (HDR) photography or multi-frame noise reduction (MFNR). In the case of HDR photography, a first image frame and a second image frame are captured using different exposure times, different apertures, different lenses, and / or other characteristics that can result in an improved dynamic range of a fused image when the two image frames are combined. In some aspects, this approach can be performed for MFNR photography, where a first image frame and a second image frame are captured using the same or different exposure times, and the first image frame and the second image frame are fused to generate a corrected first image frame that has reduced noise compared to the captured first image frame.

[0015] In some aspects, a device can include an image signal processor or processor (e.g., an application processor) that includes specific functionality for camera control and / or processing, such as enabling or disabling the merging module or otherwise controlling aspects of the image correction. The methods and techniques described herein can be performed entirely by the image signal processor or processor, or various operations can be split between the image signal processor and the processor, and in some aspects across additional processors.

[0016] The device can include one, two, or more image sensors, such as a first image sensor. When multiple image sensors are present, the configuration of these image sensors can be different. For example, the first image sensor can have a larger field of view (FOV) than the second image sensor, or the first image sensor can have a different sensitivity or a different dynamic range than the second image sensor. In one example, the first image sensor can be a wide-angle image sensor, and the second image sensor can be a telephoto image sensor. In another example, the first sensor is configured to obtain images through a first lens having a first optical axis, and the second sensor is configured to obtain images through a second lens having a second optical axis different from the first optical axis. Additionally or alternatively, the first lens can have a first magnification, and the second lens can have a second magnification different from the first magnification. Any of these or other configurations can be part of a lens cluster on a mobile device, such as where multiple image sensors and associated lenses are located in an offset position on a front or back side of the mobile device. Additional image sensors having larger, smaller, or the same field of view can be included. The image processing techniques described herein can be applied to image frames captured from any of the image sensors in a multi-sensor device.

[0017] In additional aspects of the disclosure, a device configured for image processing and / or image capture is disclosed. The apparatus includes means for capturing image frames. The apparatus also includes one or more means for capturing data representative of a scene, such as image sensors (including charge-coupled devices (CCD), Bayer filter sensors, infrared (IR) detectors, ultraviolet (UV) detectors, complementary metal-oxide-semiconductor (CMOS) sensors), and time-of-flight detectors. The apparatus can also include one or more means for concentrating and / or focusing light into one or more image sensors (including simple lenses, compound lenses, spherical lenses, and aspherical lenses). These components can be controlled to capture first and / or second image frames that are input to the image processing techniques described herein.

[0018] Other aspects, features, and details will become apparent to those of ordinary skill in the art upon examination of the following description in conjunction with the accompanying drawings. While some aspects can be discussed in the context of certain examples, various aspects can include one or more of the advantageous features discussed herein. In other words, although some aspects can be discussed with respect to particular examples, one or more of the advantageous features can be utilized in accordance with a variety of aspects. Similarly, although exemplary aspects can be discussed in the context of devices, systems, or methods, various aspects can be implemented in a variety of devices, systems, and methods.

[0019] The method can be embedded as computer program code in a computer readable medium, the computer program code comprising instructions for causing a processor to perform the steps of the method. In some embodiments, the processor can be part of a mobile device comprising: a first network adapter configured to transmit data, such as recorded images or video or streaming data, over a first network connection of a plurality of network connections; and a processor coupled to the first network adapter and a memory. The processor can cause the output image frames described herein to be transmitted over a wireless communication network, such as a 5G NR communication network.

[0020] The foregoing has outlined rather broadly the features and technical advantages of examples according to the disclosure in order that the detailed description that follows can be better understood. Additional features and advantages will be described hereinafter. The disclosed concepts and specific examples can be readily utilized as bases for modifying or designing other for carrying the same purposes of the present disclosure. Such equivalent constructions are not to depart from the scope of the claims. The characteristics of the concepts disclosed herein, both to the organization and method of operation, and the associated advantages will be better understood from the following description when considered in connection with the accompanying drawings. Each of the figures in the drawings is provided for purposes of illustration and description, and is not to be taken as a limitation on the definition of the claims.

[0021] While aspects and implementations are described in this application by illustration to some examples, those skilled in the art will understand that additional implementations and use cases can come about in many different arrangements and scenarios. The innovations described herein can be implemented across many differing platform types, devices, systems, shapes, sizes, and packaging arrangements. For example, aspects and / or uses can come about in terminal devices, vehicles, communication devices, computing devices, industrial equipment, retail / purchasing devices, medical devices, artificial intelligence (Al)-enabled devices, and / or the like. Some examples can or can not be specifically directed to use cases or applications, but can be applicable across any number of appropriate devices, platforms, systems, shapes, sizes, and packaging arrangements. The scope of the disclosure is not to be limited by the experiments described herein, but rather by the claims. The scope of the disclosure is to be measured in view of the claims that can be issued in this application and their equivalents. BRIEF DESCRIPTION OF DRAWINGS

[0022] Further understanding of the nature and advantages of the disclosure can be realized by reference to the following drawings. In the drawings, similar components or features can have the same reference label. Further, various components of the same type can be distinguished by adding a dash and a second label that distinguishes among the groups of components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

[0023] Figure 1 A block diagram of an example device for performing image capture from one or more image sensors is shown.

[0024] Figure 2 is a block diagram illustrating an example data flow path for image data processing in an image capture device in accordance with one or more implementations of the disclosure.

[0025] Figure 3 A flow diagram of an example method for processing image data to improve image stabilization of a VOZ system in accordance with some implementations of the disclosure is shown.

[0026] Figure 4 is a block diagram illustrating an example processor configuration for image data processing in an image capture device according to one or more embodiments of the present disclosure.

[0027] Figure 5 is a system for processing image frames to improve image stabilization of a VOZ system according to an aspect of the present disclosure.

[0028] Figure 6 depicts an image stabilization flow according to an aspect of the present disclosure.

[0029] Figure 7 depicts a VOZ system according to an aspect of the present disclosure.

[0030] The same reference numbers and designations in different drawings represent the same elements. DETAILED DESCRIPTION

[0031] The detailed description set forth below, in connection with the appended drawings and embodiments described herinin, is intended as a description of various configurations and is not intended to limit the scope of the disclosure. Rather, the detailed description includes specific details for the purpose of providing a thorough understanding of the inventive subject matter. It will be apparent to those skilled in the art, from this disclosure and with no further effort, that the

[0032] The present disclosure provides systems, apparatuses, methods, and computer-readable media that support image processing, including techniques for improving image stabilization of a VOZ system. Various image stabilization techniques, including optical and electronic stabilization techniques, can be used to stabilize image frames. Optical image stabilization techniques can produce higher quality images, but can only be able to perform lower degrees of image correction. Electronic image stabilization techniques can be able to perform greater degrees of correction, but the total amount of correction that can be applied to a received image frame can depend on the current zoom level of the system. A variable optical zoom imaging system can be able to change its zoom setting to a desired zoom level. This ability can result in a complication of the stabilization techniques, such as by changing the amount of correction needed depending on the current zoom level, changing the amount of correction that can be applied based on the current zoom level, or a combination thereof.

[0033] One solution to this problem is to utilize movement information to detect and respond to changes in the level of movement, thereby detecting and responding to stabilization requirements of an imaging system, including a VOZ system. Specifically, if the level of movement indicates that the VOZ system is moving too much to be corrected by the current image stabilization process, the image stabilization techniques can be adjusted. These adjustments can include changing the lens position, such as zooming out to enable greater degrees of electronic image stabilization. These adjustments can additionally or alternatively include changing the image stabilization process being used, such as transitioning from an optical image stabilization process to an electronic image stabilization process.

[0034] The disadvantages mentioned herein are merely representative and are included to emphasize the problems the inventors have identified in existing devices and sought to improve upon. The aspects of the device described below address some or all of these disadvantages, as well as other disadvantages known in the art. The improved aspects of the device described herein may offer additional benefits beyond those described above and may be used in applications other than those described above.

[0035] Specific embodiments of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages or benefits. These techniques can be adjusted in real time based on current motion information measured by an image capture device including a VOZ system. Therefore, these techniques enable improved image stabilization of images captured by these systems by utilizing techniques capable of responding to changes in the lens position of the VOZ system. Furthermore, these techniques enable more reliable use of higher-quality optical image stabilization techniques, relying only on electronic image stabilization (such as employing minimal cropping and upsampling) where possible and feasible. These techniques correspondingly improve the overall stability and image quality of images (especially image sequences) captured by VOZ systems.

[0036] In the description of the embodiments herein, numerous specific details (such as examples of specific components, circuits, and processes) are set forth to provide a thorough understanding of this disclosure. As used herein, the term "coupled" means a direct connection or a connection via one or more intermediate components or circuits. Furthermore, specific terminology is set forth in the following description and for purposes of explanation in order to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that practicing the teachings disclosed herein may not require these specific details. In other instances, known circuits and devices are illustrated in block diagram form to avoid obscuring the teachings of this disclosure.

[0037] Certain portions of the following detailed description are presented using other symbolic representations of programs, logic blocks, processes, and data bit operations within computer memory. In this disclosure, programs, logic blocks, processes, etc., are conceived as a self-consistent sequence of steps or instructions that produce a desired result. These steps are those that require physical manipulation of physical quantities. Although not strictly necessary, these physical quantities typically take the form of electrical or magnetic signals that can be stored, transferred, combined, compared, and otherwise manipulated within a computer system.

[0038] Example devices (such as smartphones) for capturing image frames using one or more image sensors may include a configuration of one, two, three, four, or more camera modules on the rear side (e.g., the side opposite the main user display) and / or the front side (e.g., the same side as the main user display). These devices may include one or more image signal processors (ISPs), computer vision processors (CVPs) (e.g., AI engines), or other suitable circuitry for processing the images captured by the image sensors. The one or more image signal processors (ISPs) may store the output image frames in memory (e.g., via a bus) and / or provide the output image frames to processing circuitry (e.g., an application processor). The processing circuitry may perform further processing, such as encoding, storing, transmitting, or other manipulations of the output image frames.

[0039] As used herein, a camera module may include an image sensor and certain other components coupled to the image sensor for acquiring a representation of a scene in image data comprising image frames. For example, a camera module may include other components of the camera, including a shutter, buffer, or additional readout circuitry for accessing individual pixels of the image sensor. In some embodiments, a camera module may include one or more components including an image sensor housed in a single package having an interface configured to couple the camera module to an image signal processor or other processor via a bus.

[0040] Figure 1 A block diagram of a device 100 for performing image capture from one or more image sensors is shown. Device 100 may include or be otherwise coupled to an image signal processor (e.g., ISP 112) for processing image frames from one or more image sensors, such as a first image sensor 101, a second image sensor 102, and a depth sensor 140. In some specific embodiments, device 100 may also include or be coupled to a processor 104 and a memory 106 storing instructions 108 (e.g., memory storing processor-readable code or a non-transitory computer-readable medium storing instructions). Device 100 may also include or be coupled to a display 114 and component 116. Component 116 may be used for user interaction, such as a touchscreen interface and / or physical buttons.

[0041] Component 116 may also include network interfaces for communicating with other devices, including a wide area network (WAN) adapter (e.g., WAN adapter 152), a local area network (LAN) adapter (e.g., LAN adapter 153), and / or a personal area network (PAN) adapter (e.g., PAN adapter 154). WAN adapter 152 may be a 4G LTE or 5G NR wireless network adapter. LAN adapter 153 may be an IEEE 802.11 WiFi wireless network adapter. PAN adapter 154 may be a Bluetooth wireless network adapter. Each of WAN adapter 152, LAN adapter 153, and / or PAN adapter 154 may be coupled to an antenna comprising multiple antennas configured for main and diversity reception and / or configured to receive a specific frequency band. In some embodiments, the antennas may be shared by WAN adapter 152, LAN adapter 153, and / or PAN adapter 154 for communication on different networks. In some implementations, WAN adapter 152, LAN adapter 153 and / or PAN adapter 154 may share circuitry and / or be packaged together, such as when LAN adapter 153 and PAN adapter 154 are packaged as a single integrated circuit (IC).

[0042] Device 100 may also include or be coupled to a power source 118 for use with device 100, such as a battery or an adapter for coupling device 100 to an energy source. Device 100 may also include or be coupled to... Figure 1 Additional features or components not shown. In one example, a wireless interface that may include multiple transceivers and a baseband processor in a radio frequency front-end (RFFE) may be coupled to or included in the WAN adapter 152 for use in a wireless communication device. In another example, an analog front-end (AFE) for converting analog image data to digital image data may be coupled between the first image sensor 101 or the second image sensor 102 and the processing circuitry in the device 100. In some embodiments, the AFE may be embedded in the ISP 112.

[0043] The device may include or be coupled to a sensor hub 150, which interfaces with sensors to receive data about the movement of device 100, data about the environment surrounding device 100, and / or other non-camera sensor data. One example non-camera sensor is a gyroscope, a device configured to measure rotation, orientation, and / or angular velocity to generate motion data. Another example non-camera sensor is an accelerometer, a device configured to measure acceleration, which can also be used to determine the speed and distance of travel by appropriately integrating the measured acceleration. In some aspects, a gyroscope in an electronic image stabilization system (EIS) may be coupled to the sensor hub. In another example, the non-camera sensor may be a Global Positioning System (GPS) receiver, a device used to process satellite signals, such as through triangulation and other techniques, to determine the location of device 100. Positioning can be tracked over time to determine additional motion information, such as velocity and acceleration. Data from one or more sensors may be accumulated by the sensor hub 150 into motion data. One or more of acceleration, velocity, and / or distance may be included in the motion data provided by sensor hub 150 to other components of device 100 (including ISP 112 and / or processor 104).

[0044] The ISP 112 can receive captured image data. In one embodiment, a local bus connection couples the ISP 112 to the first image sensor 101 and the second image sensor 102 of the first camera 103 and the second camera 105, respectively. In another embodiment, a wired interface couples the ISP 112 to an external image sensor. In yet another embodiment, a wireless interface couples the ISP 112 to either the first image sensor 101 or the second image sensor 102.

[0045] First image sensor 101 and second image sensor 102 are configured to capture image data representing scenes within the fields of view of first camera 103 and second camera 105, respectively. In some embodiments, first camera 103 and / or second camera 105 output analog data converted by an analog front-end (AFE) and / or analog-to-digital converter (ADC) in device 100 or embedded in ISP 112. In some embodiments, first camera 103 and / or second camera 105 output digital data. The digital image data may be formatted into one or more image frames, whether received from first camera 103 and / or second camera 105 or converted from analog data received from first camera 103 and / or second camera 105.

[0046] The first camera 103 may include a first image sensor 101 and a first lens 131. The second camera may include a second image sensor 102 and a second lens 132. Each of the first lens 131 and the second lens 132 may be controlled by an associated autofocus (AF) algorithm (e.g., AF 133) executed in the ISP 112, which adjusts the first lens 131 and the second lens 132 to focus on a specific focal plane located at a specific scene depth. AF 133 may be assisted by depth data received from the depth sensor 140. The first lens 131 and the second lens 132 focus light onto the first image sensor 101 and the second image sensor 102 respectively through one or more apertures for receiving light, one or more shutters for blocking light when outside the exposure window, and / or one or more color filter arrays (CFAs) for filtering light outside a specific frequency range. The first lens 131 and the second lens 132 may have different fields of view to capture different representations of the scene. For example, the first lens 131 may be an ultra-wide (UW) lens, and the second lens 132 may be a wide (W) lens. Multiple image sensors may include a combination of ultra-wide (high field of view (FOV)) sensors, wide sensors, long-range sensors, and ultra-long-range (low FOV) sensors.

[0047] Each of the first camera 103 and the second camera 105 can be configured through hardware configuration and / or software settings to obtain different but overlapping fields of view. In some configurations, the cameras are configured using different lenses with different magnifications, resulting in different fields of view for capturing different representations of the scene. The cameras can be configured such that the UW camera has a larger FOV than the W camera, the W camera has a larger FOV than the T camera, and the T camera has a larger FOV than the UT camera. For example, a camera configured for a wide FOV can capture a field of view in the range of 64 to 84 degrees, a camera configured for an ultra-side FOV can capture a field of view in the range of 100 to 140 degrees, a camera configured for a long-range FOV can capture a field of view in the range of 10 to 30 degrees, and a camera configured for an ultra-long-range FOV can capture a field of view in the range of 1 to 8 degrees.

[0048] In some implementations, one or more of the first camera 103 and / or the second camera 105 may be a variable aperture (VA) camera, wherein the aperture can be adjusted to set a specific aperture size. Example aperture sizes include f / 2.0, f / 2.8, f / 3.2, f / 8.0, etc. Larger aperture values ​​correspond to smaller aperture sizes, and smaller aperture values ​​correspond to larger aperture sizes. The variable aperture (VA) camera may have different characteristics that produce different representations of the scene based on the current aperture size. For example, the VA camera may capture image data with a depth of focus (DOF) corresponding to the current aperture size set for the VA camera.

[0049] In some embodiments, one or more of the first camera 103 and / or the second camera 105 may include a variable optical zoom (VOZ) system. The VOZ system may be able to adjust the focal length of one or more lenses associated with one or more image sensors to adjust the zoom level of the image captured by the image sensors. For example, Figure 7 A VOZ system 700 according to one aspect of this disclosure is depicted. The VOZ system 700 includes an image sensor 704 that receives light to capture an image, as discussed above. Specifically, the image sensor 704 receives light passing through a folded optical system formed by a prism 702 and a lens module 706. The lens module 706 includes a plurality of lenses 708, 710, 712 (only a subset thereof is numbered for clarity). Lenses 708, 710, 712 may include one or more convex lenses, concave lenses, or combinations thereof. The prism 702 bends light by 90° such that the light passes through the lens module 706 and is received by the image sensor 704. In some specific embodiments, the prism 702 may be movable or otherwise adjustable to compensate for movement of the device. For example, the prism 702 is capable of providing optical image stabilization by moving or being moved to compensate for movement of the device 100. To provide a variable optical zoom setting, the lens module 706 may be movable between the image sensor 704 and the prism 702. For example, the lens module 706 may be along the axis that connects the prism 702 to the image sensor 704 (such as along...). Figure 7 The lens module 706 moves axially along the axis extending from the center of lenses 708, 710, and 712. This movement of the lens module 706 enables different zoom levels of the image captured by the image sensor 704. For example, the VOZ system 700 may be able to have lens positions associated with zoom levels ranging from 1 to 4x, 2 to 5x, 1 to 10x, etc. In various embodiments, one or more of the lenses 708, 710, and 712 may be movable relative to each other (e.g., to adjust depth of field, focus, or other characteristics of the captured image). In some embodiments, the VOZ system 700 may also be referred to as a continuous optical zoom (COZ) system.

[0050] The ISP 112 processes image frames captured by the first camera 103 and the second camera 105. Although Figure 1Device 100 is illustrated as including a first camera 103 and a second camera 105, but any number of cameras (e.g., one, two, three, four, five, six, etc.) may be coupled to ISP 112. In some aspects, depth sensors (such as depth sensor 140) may be coupled to ISP 112. The output from depth sensor 140 may be processed in a manner similar to that of the first camera 103 and the second camera 105. Examples of depth sensors 140 include active sensors, including one or more of indirect time-of-flight (iToF), direct time-of-flight (dToF), light detection and ranging (LiDAR), mmWave, radio detection and ranging (radar), and / or hybrid depth sensors (such as structured light sensors). In embodiments without depth sensor 140, similar information about the depth or depth map of an object may be determined based on the parallax between the first camera 103 and the second camera 105, such as by using parallax depth measurement algorithms, stereo depth measurement algorithms, phase detection autofocus (PDAF) sensors, etc. In addition, any number of additional image sensors or image signal processors may be present in device 100.

[0051] In some embodiments, ISP 112 may execute instructions from memory, such as instructions 108 from memory 106, instructions stored in a separate memory coupled to or included in ISP 112, or instructions provided by processor 104. Additionally or alternatively, ISP 112 may include specific hardware (such as one or more integrated circuits (ICs)) configured to perform one or more operations described in this disclosure. For example, ISP 112 may include an image front-end (e.g., IFE 135), an image post-processing engine (e.g., IPE 136), an automatic exposure compensation (AEC) engine (e.g., AEC 134), and / or one or more engines for video analysis (e.g., EVA 137). The image pipeline may be formed by a sequence of one or more of IFE 135, IPE 136, and / or EVA 137. In some embodiments, the image pipeline in ISP 112 may be reconfigured by changing the connections between IFE 135, IPE 136, and / or EVA 137. AF 133, AEC 134, IFE 135, IPE 136 and EVA137 may each include dedicated circuitry and may be embodied as software or firmware executed by ISP 112 and / or a combination of hardware and software or firmware executed on ISP 112.

[0052] Memory 106 may include a non-transient or non-transitory computer-readable medium storing computer-executable instructions (as instructions 108) for performing all or part of one or more of the operations described in this disclosure. Instructions 108 may include a camera application (or other suitable application, such as a messaging application) to be executed by device 100 for taking pictures or videos. Instructions 108 may also include other applications or programs executed by device 100, such as an operating system and applications other than those for image or video generation. Executing a camera application, such as by processor 104, may enable device 100 to record images using the first camera 103 and / or the second camera 105 and the ISP 112.

[0053] In addition to instruction 108, memory 106 may also store image frames. The image frames may be output image frames stored by ISP 112. The output image frames may be accessed by processor 104 for further operation. In some embodiments, device 100 does not include memory 106. For example, device 100 may be circuitry including ISP 112, and the memory may be external to device 100. Device 100 may be coupled to external memory and configured to access that memory to write output image frames for display or long-term storage. In some embodiments, device 100 is a system-on-a-chip (SoC) that integrates ISP 112, processor 104, sensor hub 150, memory 106, and / or component 116 into a single package.

[0054] In some embodiments, at least one of the ISP 112 or processor 104 executes instructions to perform various operations described herein, including image stabilization of the VOZ system. For example, the execution of instructions may instruct the ISP 112 to begin or end the capture of image frames or sequences of image frames, wherein the capture includes corrections as described in the embodiments herein. In some embodiments, processor 104 may include one or more general-purpose processor cores 104A-N capable of executing instructions to control the operation of the ISP 112. For example, cores 104A-N may execute a camera application (or other suitable application for generating images or videos) stored in memory 106 that activates or deactivates the ISP 112 to capture image frames and / or controls the ISP 112 when applying image stabilization of the VOZ system to the image frames. The operation of cores 104A-N and ISP 112 may be based on user input. For example, a camera application executing on processor 104 may receive a user command to start a video preview display. Upon receiving the user command, video, including a sequence of image frames, is captured and processed via ISP 112 from first camera 103 and / or second camera 105 for display and / or storage. Image processing, such as that described herein, for determining “output” or “corrected” image frames, may be applied to one or more image frames in the sequence.

[0055] In some implementations, processor 104 may include an IC or other hardware (e.g., an artificial intelligence (AI) engine (such as AI engine 124) or other coprocessor) to offload certain tasks from cores 104A-N. AI engine 124 may be used to offload tasks related to face detection and / or object recognition, performed, for example, using machine learning (ML) or artificial intelligence (AI). AI engine 124 may be referred to as an artificial intelligence processing unit (AI PU). AI engine 124 may include hardware configured to perform and accelerate convolutional operations involved in performing machine learning algorithms, such as by executing predictive models such as artificial neural networks (ANNs) (including multilayer feedforward neural networks (MLFFNNs), recurrent neural networks (RNNs), and / or radial basis functions (RBFs)). The ANN executed by AI engine 124 has access to predefined training weights for performing operations on user data. The ANN may optionally be trained during operation of image capture device 100, such as through reinforcement training, supervised training, and / or unsupervised training. In some other implementations, device 100 does not include processor 104, such as when all the described functionality is configured in ISP 112.

[0056] In some embodiments, display 114 may include one or more suitable displays or screens that allow the user to interact and / or present a preview of an item (such as the output of the first camera 103 and / or the second camera 105) to the user. In some embodiments, display 114 is a touch-sensitive display. Input / output (I / O) components (such as component 116) may be or include any suitable mechanism, interface, or device to receive input (such as commands) from the user and provide output to the user via display 114. For example, component 116 may include (but is not limited to) a graphical user interface (GUI), keyboard, mouse, microphone, speaker, squeezable bezel, one or more buttons (such as a power button), slider, toggle key, switch, etc.

[0057] Although shown as coupled to each other via processor 104, components (such as processor 104, memory 106, ISP 112, display 114, and component 116) may be coupled to each other in various other arrangements, such as via one or more local buses, which are not shown for simplicity. An example of a bus used to interconnect components is a Peripheral Component Interface (PCI) Fast (PCIe) bus.

[0058] Although ISP 112 is illustrated as separate from processor 104, ISP 112 may be the core of processor 104, which is an application processor unit (APU) included in a system-on-a-chip (SoC), or otherwise included in processor 104. While device 100 is referenced in the examples herein to perform aspects of this disclosure, some device components may not be included. Figure 1 The details are shown to prevent obscuring aspects of this disclosure. Additionally, other components, the number of components, or combinations of components may be included in suitable equipment for performing aspects of this disclosure. Therefore, this disclosure is not limited to the configuration of a particular device or component, including device 100.

[0059] Operable Figure 1 An exemplary image capture device is provided to obtain improved images by better selecting and controlling image stabilization when capturing image frames using the VOZ system. Figure 2 An example method of operating one or more cameras (such as a first camera 103 and / or a second camera 105) is shown and described below.

[0060] Figure 2This is a block diagram illustrating an example data flow path for image data processing in an image capture device according to one or more embodiments of the present disclosure. The processor 104 of system 200 communicates with ISP 112 via a bidirectional bus and / or separate control and data lines. The processor 104 can control a first camera 103 via camera control 210. Camera control 210 may be a camera driver executed by processor 104 for configuring the first camera 103 (such as activating or deactivating image capture, configuring exposure settings, and / or configuring aperture size). Camera control 210 may be managed by a camera application 204 executing on processor 104. Camera application 204 provides user-accessible settings, allowing a user to specify individual camera settings or select a profile with corresponding camera settings. Camera control 210 communicates with the first camera 103 to configure the first camera 103 according to commands received from camera application 204. Camera application 204 may be, for example, a photography application, a document scanning application, a messaging application, or other applications that process image data acquired from the first camera 103.

[0061] Camera configuration may include parameters specifying, such as frame rate, image resolution, readout duration, exposure level, aspect ratio, aperture size, etc. The first camera 103 may apply the camera configuration and use it to acquire image data representing the scene. In some embodiments, the camera configuration may be adjusted to obtain different representations of the scene. For example, processor 104 may execute camera application 204 to instruct the first camera 103 via camera control 210 to set a first camera configuration for the first camera 103, acquire first image data from the first camera 103 operating with the first camera configuration, instruct the first camera 103 to set a second camera configuration for the first camera 103, and acquire second image data from the first camera 103 operating with the second camera configuration.

[0062] In some embodiments where the first camera 103 is a variable aperture (VA) camera system, the processor 104 can execute camera application 204 to instruct the first camera 103 to be configured to a first aperture size, acquire first image data from the first camera 103, instruct the first camera 103 to be configured to a second aperture size, and acquire second image data from the first camera 103. The aperture reconfiguration and the acquisition of the first and second image data can occur with little or no change in the scene captured at the first aperture size and the second aperture size. Example aperture sizes are f / 2.0, f / 2.8, f / 3.2, f / 8.0, etc. Larger aperture values ​​correspond to smaller aperture sizes, and smaller aperture values ​​correspond to larger aperture sizes. That is, f / 2.0 corresponds to an aperture size larger than f / 8.0.

[0063] Image data received from the first camera 103 can be processed in one or more blocks of the ISP 112 to determine an output image frame 230 that can be stored in memory 106 and / or otherwise provided to the processor 104. The processor 104 can further process the image data to apply effects to the output image frame 230. Effects may include background blur, lighting, color cast, and / or high dynamic range (HDR) blending. In some embodiments, effects can be applied in the ISP 112.

[0064] Image frames 230 output by ISP 112 may include a representation of the scene improved by various aspects of this disclosure, resulting in better image frame stabilization when captured with a VOZ system. Processor 104 may display these output image frames 230 to a user, and the improvements provided by the described processing implemented in ISP 112 and / or processor 104 enhance image quality and user experience by improving image frame stability. For example, one or more image stabilization processes 212 in ISP 112 may correct image data received from the first camera 103 when determining the output image frame 230.

[0065] Figure 2 System 200 can be configured to execute the reference. Figure 3 The described operation determines the output image frame 230. Figure 3 A flowchart is shown of an example method for processing image data to perform VOZ image stabilization according to some embodiments of the present disclosure. Figure 3 Capturing images in this way allows for an improved digital representation of the scene, resulting in photos or videos with higher image quality (IQ). (Reference) Figure 3 Each operation described may be performed by one or a combination of processor 104 (including core 104A-N or AI engine 124) and / or ISP 112.

[0066] Method 300 includes receiving a first image frame from a VOZ system (block 302). For example, device 100 may receive the first image frame from a VOZ system. The first image frame may be received, for example, from a bus coupled to the first camera 103 or from an analog front end (AFE) coupled to the first camera 103. The first image frame may alternatively be received from a wireless camera, wherein image data is received via one or more adapters among WAN adapter 152, LAN adapter 153, and / or PAN adapter 154. Image data may alternatively be received from memory location or network storage location, such as when image data was previously captured and is now retrieved from memory 106 and / or remote location via one or more adapters among WAN adapter 152, LAN adapter 153, and / or PAN adapter 154. In some embodiments, the capture of image data may be initiated by a camera application executing on processor 104, which causes camera control 210 to activate the first camera 103 to capture image data. The image data retrieved at box 302 can then be processed by ISP 112 and / or processor 104, or other components for processing the image data according to the operations described in one or more of the boxes below. In some embodiments, a first image frame is captured at a first lens position associated with a first zoom level of the VOZ system. In some embodiments, the image frame is stabilized at the target zoom level using a first image stabilization process. In some embodiments, the first image stabilization process may be an EIS process, an OIS process, or a combination thereof. In some embodiments, the image frame obtained from the first image stabilization process may be at the target zoom level.

[0067] Method 300 includes determining that motion information of the VOZ system exceeds a predetermined threshold associated with a first image stabilization process (box 304). For example, device 100 may determine that motion information of the VOZ system exceeds a predetermined threshold associated with a first image stabilization process. In some embodiments, motion information may be processed to determine the current level of motion of the VOZ system. In some embodiments, motion information includes an average of motion measurements, peak motion measurements, the number of failed image stabilization frames, or a combination thereof. In some embodiments, the predetermined threshold may be determined based on a first lens position (such as a first zoom level) and a first image stabilization process.

[0068] Method 300 includes determining the position of a second lens of the VOZ system based on motion information exceeding a predetermined threshold (block 306). For example, device 100 may determine the position of the second lens of the VOZ system based on motion information exceeding a predetermined threshold. The second lens position may be associated with a second zoom level of the VOZ system. In some specific implementations, the second lens position may be determined such that the captured second image frame has a margin of pixels relative to the target zoom level to correct for motion indicated by the motion information.

[0069] Method 300 includes receiving a second image frame captured by the VOZ system at a second lens position (block 308). For example, device 100 may receive a second image frame captured by the VOZ system at a second lens position. In some specific embodiments, device 100 may control camera system 500 to adjust or otherwise change the VOZ system from a first lens position to a second lens position.

[0070] Method 300 includes stabilizing a second image frame by applying at least one of a first image stabilization process and a second image stabilization process, thereby determining an output image frame at a target zoom level based on the second image frame (block 310). For example, device 100 may stabilize the second image frame by applying at least one of a first image stabilization process and a second image stabilization process, thereby determining an output image frame at a target zoom level based on the second image frame. The second image stabilization process may differ from the first image stabilization process. In some embodiments, the first image stabilization process determines the stabilized image frame at the target zoom level when applied to subsequent image frames captured by the VOZ system at the second lens position. In some embodiments, block 310 further includes determining a second image stabilization process for the VOZ system based on motion information and the target zoom level. In some embodiments, the first image stabilization process includes an optical image stabilization (OIS) process, and the second image stabilization process includes an electronic image stabilization (EIS) process.

[0071] Figure 4 This is a block diagram illustrating an example processor configuration for image data processing in an image capture device according to one or more embodiments of the present disclosure. Processor 104 or other processing circuitry may be configured to operate on the image data to perform... Figure 3 One or more operations of the method. Image data can be processed to determine one or more output image frames 410.

[0072] Processor 104 receives first image data, second image data, and motion information. In some embodiments, the first image data may be received directly from the image sensor or a memory coupled to the image sensor. In some embodiments, the first image data may be retrieved from a long-term storage device (such as flash memory or network location) storing previously captured or generated images. Processor 104 includes image frame receiving logic 404A, zoom determination logic 404B, zoom determination logic 404C, and image stabilization logic 404D.

[0073] Image frame receiving logic 404A can be configured to receive a first image frame from a variable optical zoom (VOZ) system. In some embodiments, the first image frame is captured at a first lens position of the VOZ system. In some embodiments, the image frame is stabilized at a target zoom level using a first image stabilization process. In some embodiments, the first image stabilization process can be an EIS process, an OIS process, or a combination thereof. In some embodiments, the image frame obtained from the first image stabilization process can be at the target zoom level.

[0074] The motion determination logic can be configured to determine that the motion information of the VOZ system exceeds a predetermined threshold associated with a first image stabilization process. In some embodiments, motion information may be processed to determine the current level of motion of the VOZ system. In some embodiments, the motion information includes an average of motion measurements, peak motion measurements, the number of failed image stabilization frames, or a combination thereof. In some embodiments, the predetermined threshold may be determined based on a first lens position and a first image stabilization process.

[0075] The zoom determination logic 404C can be configured to determine the position of the second lens of the VOZ system based on motion information exceeding a predetermined threshold. In some specific implementations, the second lens position can be determined such that the captured second image frame has a margin of pixels relative to the target zoom level to correct for motion indicated by the motion information.

[0076] The image frame receiving logic 404A can also be configured to receive a second image frame captured by the VOZ system at the second lens position. In some specific embodiments, the device 100 may control the camera system 500 to adjust or otherwise change the VOZ system from the first lens position to the second lens position.

[0077] Image stabilization logic 404D can be configured to stabilize a second image frame by applying at least one of a first image stabilization process and a second image stabilization process, thereby determining an output image frame at the target zoom level based on the second image frame. The second image stabilization process may differ from the first image stabilization process. In some embodiments, the first image stabilization process determines the stabilized image frame at the target zoom level when applied to subsequent image frames captured by the VOZ system at the second lens position. In some embodiments, a second image stabilization process for the VOZ system is further included, based on motion information and the target zoom level. In some embodiments, the first image stabilization process includes an optical image stabilization (OIS) process, and the second image stabilization process includes an electronic image stabilization (EIS) process.

[0078] refer to Figure 5An example operation is described, depicting a system 500 for controlling image stabilization of a VOZ system according to one aspect of this disclosure. System 500 includes a first image frame 504, a second image frame 506, a VOZ system 502, and a device 100. The VOZ system may be an exemplary implementation of a VOZ system 700. Device 100 includes a first lens position 508, a second lens position 510, a first image stabilization process 512, a target zoom level 514, and a second image stabilization process 516. The following functionalities may be implemented by this device using a processor 104, an ISP 112, or a combination thereof.

[0079] Device 100 may be configured to receive a first image frame 504 from VOZ system 502. The first image frame 504 may be part of a plurality of image frames received by VOZ system 502, such as a sequence of image frames captured at a first lens position 508. Image frame 504 may be captured at the first lens position 508 of VOZ system 502. In some specific implementations, the first lens position 508 may be associated with a first zoom level of VOZ system 502 (such as the current zoom level). For example, VOZ system 502 may currently be available for capturing image frames (such as part of a video frame sequence). Therefore, device 100 may receive the first image frame 504 and may react to control VOZ system 502 in real time.

[0080] In some embodiments, the first image frame 504 may be stabilized by a first image stabilization process 512. The type of image stabilization performed on the first image frame 504 may vary depending on the capabilities of the device 100, the VOZ system 502, or a combination thereof. The image stabilization processes 512, 516 used by the device may include one or more optical image stabilization (OIS) processes, one or more electronic image stabilization (EIS) processes, or one or more combinations thereof. OIS is a mechanical process that physically stabilizes a camera lens using a motion sensor. Movement of the device 100 is detected, and one or more lenses are moved to compensate for the movement. For example, as described above, the prism 702 of the VOX system may be moved to compensate for detected movement. EIS may correct for movement of the device by cropping consecutive image frames to capture the same or similar portions of the captured scene. The cropped area may then be processed and magnified to determine the final output image frame. In various specific embodiments, and at various times during the operation of the device 100, the image stabilization processes 512, 516 may include an EIS process, an OIS process, or a combination thereof. For example, device 100 can use both EIS and OIS, such as by moving the lens to compensate for some of the movement of device 100, while also using EIS to correct any additional movement that cannot be corrected by OIS.

[0081] Figure 6Image stabilization processes 600 and 610 according to exemplary aspects of this disclosure are depicted. Image stabilization process 600 may represent an OIS process, and image stabilization process 610 may represent an EIS process. In process 600, image 602, depicting a subject (such as a person) and a tree, is captured at a target zoom level. If the device (such as an image capture device) moves during the capture of an image frame, a lens can be moved to compensate for the movement of the device. Thus, images can be captured continuously at the target zoom level. In process 610, image 611 is captured at a lower zoom level (such as reducing image 602 relative to the target zoom level). A subset of image 611 can be cropped to form image 612. Image 612 can be formed by cropping image 611 at the target zoom level. Image 612 can then be enlarged and subsequently processed to form image 614. If the device moves during the capture of subsequent image frames, different portions of the image captured at the lower zoom level can be cropped to maintain focus on the same subject as image 614.

[0082] In some embodiments, the first image frame 504 from the VOZ system 502 and the first image stabilization process 512 may be at a target zoom level 514. In some embodiments, the target zoom level 514 may be the same as the first zoom level associated with the first lens position 508 (e.g., if OIS is used for the first image stabilization process 512). In additional or alternative embodiments, the target zoom level 514 may be different from the first zoom level (e.g., if EIS is used for the first image stabilization process 512).

[0083] Device 100 may be configured to determine that motion information of VOZ system 502 exceeds a predetermined threshold associated with first image stabilization process 512. Motion information may include one or more types of motion data, such as motion data measured by a sensor hub or otherwise collected. In some embodiments, motion information may be processed to determine the current level of motion of VOZ system 502. In some embodiments, motion information includes motion measurements over a previous time period, such as a previous 10 seconds, 5 seconds, 1 second, 0.1 seconds, etc. In additional or alternative embodiments, motion information may include summary statistics of motion measurements over a previous time period, such as the average of motion measurements, peak motion measurements, or combinations thereof. In further embodiments, motion information may include information about first image stabilization process 512, such as whether first image stabilization process 512 has failed. For example, motion information may include the number of failed image stabilization frames, the percentage of failed image stabilizations, or combinations thereof.

[0084] In some embodiments, a predetermined threshold may indicate when an image is captured under moving conditions that are unlikely to produce properly stable image frames (e.g., video frames that may produce jitter). In some embodiments, the predetermined threshold may be determined based on the first lens position 508 and the first image stabilization process 512. In some embodiments, the predetermined threshold may be determined based on the maximum degree of correction that the first image stabilization process 512 can perform at the target zoom level 514. The predetermined threshold may be indicated based on the maximum rotation (e.g., one degree movement, 5° movement, 10° movement, 20° movement) that can be corrected by the first image stabilization process 512 (e.g., the VOZ system 502). In additional or alternative embodiments, the predetermined threshold may be indicated as the maximum number of correctable moving pixels (e.g., the maximum number of margin pixels available at the target zoom level 514 when the VOZ system captures an image at the first lens position 508). In some specific implementations, the margin pixels may include pixels existing between the outer edge of a stable image frame at the target zoom level and the outer edge of the scene captured by the image sensor of the VOZ system 502 (such as at the first lens position 508 or the second lens position 510). For example, in Figure 6 In this context, edge pixels may include pixels from image 611 that are not included in the cropped image 612. Return to Figure 5 In a further embodiment, a predetermined threshold may be indicated based on measurements from one or more motion sensors (such as maximum permissible accelerometer measurements, maximum permissible gyroscope measurements, etc.). In some embodiments, different combinations of the stabilization process 512, the current lens position 508, and the target zoom level 514 may have different predetermined thresholds. For example, when the first zoom level and the target zoom level 514 differ by a larger amount (e.g., because a larger proportion of margin pixels are available for EIS correction), the image stabilization process 512 may be able to perform greater correction and stabilization. In some embodiments, the predetermined threshold may be determined by identifying the corresponding threshold for the current type of the first zoom level, the target zoom level 514, and the first image stabilization process 512. As a specific example, device 100 may maintain a table, database, or other record of thresholds for different combinations of image stabilization processes, VOZ system zoom levels, VOZ lens positions, and target zoom levels, and the predetermined threshold may be determined by accessing the corresponding threshold within the table, database, or other record.

[0085] Device 100 may be configured to determine a second lens position 510 of VOZ system 502 based on motion information exceeding a predetermined threshold. The second lens position 510 may be associated with a second zoom level. In some embodiments, the second lens position 510 may be determined such that the captured second image frame 506 has a margin of pixels relative to a target zoom level 514 to correct for motion indicated by the motion information. In some embodiments, device 100 may utilize the amount of motion indicated by the motion information to determine how much correction (such as how much margin) is needed to stabilize the image frame to the target zoom level 514. In an embodiment where device 100 maintains a lookup table of thresholds, the second lens position 510 may be determined by identifying a second zoom level that identifies the first image stabilization process 512, which has a threshold greater than the motion indicated by the motion information (e.g., greater than at least a predetermined margin, such as 5%, 10%, 25%, etc.). In additional or alternative embodiments, the second lens position 510 may be calculated based on the current level of motion indicated by the motion information. As a specific example, if the motion information indicates that the VOZ system 502 moves 20° vertically when capturing an image frame, the second lens position 510 can be determined such that even with the 20° movement, the subject remains in the frame at the target zoom level to achieve proper stabilization (e.g., making the target zoom level 514 occupy 40° less than the FOV of the VOZ system). In some specific implementations, the second zoom level can be determined to be smaller than the first zoom level (e.g., more reduced relative to the first zoom level). In such cases, the lower second lens position 510 can be significantly corrected by the image stabilization processes 512, 516.

[0086] Device 100 may be configured to receive a second image frame 506 captured by VOZ system 502 at a second lens position 510. The second image frame 506 may be part of a plurality of image frames received by VOZ system 502, such as a sequence of image frames captured at the second lens position 510. In some specific implementations, device 100 may control camera system 500 to use VOZ system 502 to capture subsequent image frames at a target zoom level 514 at the second lens position 510. For example, device 100 may determine the hardware configuration of VOZ system 502 indicating the second lens position 510 and may provide that hardware configuration to VOZ system 502. In response, VOZ system 502 may transition from a first lens position 508 to a second lens position 510 and continue capturing image frames as the second image frame 506. Device 100 may then receive the second image frame 506 from VOZ system 502.

[0087] Device 100 may be configured to determine one or more output image frames of image frame 230 output at target zoom level 514 based on second image frame 506. For example, device 100 may apply at least one of a first image stabilization process 512 and a second image stabilization process 516 to stabilize the second image frame 506. The second image stabilization process 516 may be different from the first image stabilization process 512. In some embodiments, device 100 may be able to use the same first image stabilization process 512 to determine the output image frame 230. For example, the first image stabilization process 512 may be an EIS process, and device 100 may continue to use EIS to determine the output image frame 230, such as having a larger correction level achieved by the second lens position 510.

[0088] In some implementations, a second image stabilization process 516 may be used separately from the first image stabilization process 512. For example, the second image stabilization process 516 may be selected if the motion information indicates a level of movement greater than that that the first image stabilization process can correct. For example, device 100 may determine the second image stabilization process 516 of the VOZ system 502 to be used with the second image frame 506 based on motion information and the target zoom level 514. In some specific implementations, the second image stabilization process 516 may be determined based on a predetermined threshold associated with the first stabilization process 512, such as a maximum threshold of the first stabilization process 512 within a stored range. The predetermined threshold may indicate the maximum level of movement that can be corrected by the first image stabilization process 512. The maximum level of movement may be indicated as the maximum angle of movement of the VOZ system 502, the maximum measurement from the sensor hub 150, the maximum number of failed image stabilization frames, or a combination thereof. In some specific implementations, different predetermined thresholds may be indicated for different target zoom levels 514. For example, higher zoom levels may be easier to move and therefore may have a lower level of potential stability. In such cases, higher zoom levels may have lower corresponding thresholds. In some implementations, separate, predetermined thresholds may be stored for individual target zoom levels. In additional or alternative implementations, predetermined thresholds may be defined based on the target zoom level 514.

[0089] In some implementations, the first image stabilization process 512 includes an optical image stabilization (OIS) process, and the second image stabilization process 516 includes an electronic image stabilization (EIS) process. For example, device 100 may initially use OIS 512 to stabilize the first image frame 504, and may switch to using the EIS process 516 in response to a motion level exceeding a maximum threshold of process 512. In some cases, the EIS process 516 may also include OIS stabilization techniques. For example, the image stabilization process 516 may be a hybrid image stabilization process using both optical and electronic image stabilization.

[0090] In one or more aspects, techniques for supporting image processing may include additional aspects, such as any single aspect or any combination of aspects described below or in conjunction with one or more other processes or devices described elsewhere herein. A first aspect provides a method comprising: receiving a first image frame from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; determining that motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the motion information exceeding the predetermined threshold; receiving a second image frame captured by the VOZ system at the second lens position; and stabilizing the second image frame by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining an output image frame at the target zoom level based on the second image frame.

[0091] In a second aspect, in conjunction with the first aspect, the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

[0092] In a third aspect, in conjunction with the second aspect, the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

[0093] In a fourth aspect, in conjunction with one or more of the first to third aspects, the position of the second lens is determined such that the captured second image frame has a margin of pixels relative to the target zoom level, in order to correct for movement indicated by the motion information.

[0094] In a fifth aspect, in conjunction with one or more of the first to fourth aspects, the method further includes determining a second image stabilization process for the VOZ system based on the motion information and the target zoom level.

[0095] In a sixth aspect, in conjunction with the fifth aspect, the first image stabilization process may include an optical image stabilization (OIS) process, and the second image stabilization process may include an electronic image stabilization (EIS) process.

[0096] In a first aspect, in combination with one or more of the first to sixth aspects, the first image stabilization process may include an electronic image stabilization process, and the output image frame may be determined by applying the first image stabilization process to the second image frame.

[0097] In the eighth aspect, in combination with one or more of the first to seventh aspects, the motion information may include an average value of motion measurements, peak motion measurements, the number of failed image stabilization frames, or a combination thereof.

[0098] A ninth aspect provides a system comprising a processor and a memory storing instructions, which, when executed by the processor, cause the processor to perform operations including: receiving a first image frame from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; determining that motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the motion information exceeding the predetermined threshold; receiving a second image frame captured by the VOZ system at the second lens position; and stabilizing the second image frame by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining an output image frame at the target zoom level based on the second image frame.

[0099] Additionally, the apparatus may perform or operate according to one or more aspects described below. In some embodiments, the apparatus includes a wireless device, such as a UE. In some embodiments, the apparatus includes a remote server (such as a cloud-based computing solution) that receives image data, processes it, and determines output image frames. In some embodiments, the apparatus may include at least one processor and memory coupled to the processor. The processor may be configured to perform the operations described herein with respect to the apparatus. In some other embodiments, the apparatus may include a non-transitory computer-readable medium having program code recorded thereon, and the program code may be executable by a computer to cause the computer to perform the operations described herein with reference to the apparatus. In some embodiments, the apparatus may include one or more components configured to perform the operations described herein. In some embodiments, a method of wireless communication may include one or more operations described herein with reference to the apparatus.

[0100] In a tenth aspect, in conjunction with the ninth aspect, the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

[0101] In the eleventh aspect, in conjunction with the tenth aspect, the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

[0102] In the twelfth aspect, in combination with one or more of the ninth to eleventh aspects, the position of the second lens is determined such that the captured second image frame has a margin of pixels relative to the target zoom level, in order to correct for movement indicated by the motion information.

[0103] In the thirteenth aspect, in combination with one or more of the ninth to twelfth aspects, the operation may further include determining the second image stabilization process of the VOZ system based on the motion information and the target zoom level.

[0104] In the fourteenth aspect, in conjunction with the thirteenth aspect, the first image stabilization process may include an optical image stabilization (OIS) process, and the second image stabilization process may include an electronic image stabilization (EIS) process.

[0105] In the fifteenth aspect, in conjunction with one or more of the ninth to fourteenth aspects, the first image stabilization process may include an electronic image stabilization process, and the output image frame may be determined by applying the first image stabilization process to the second image frame.

[0106] In the sixteenth aspect, in combination with one or more of the ninth to fifteenth aspects, the motion information may include an average value of motion measurements, peak motion measurements, the number of failed image stabilization frames, or a combination thereof.

[0107] A seventeenth aspect provides an image capture device, comprising: an image sensor; a memory storing processor-readable code; and at least one processor coupled to the memory and the image sensor. The at least one processor may be configured to execute the processor-readable code to cause the at least one processor to perform operations including: receiving a first image frame from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; determining that motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the motion information exceeding the predetermined threshold; receiving a second image frame captured by the VOZ system at the second lens position; and stabilizing the second image frame by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining an output image frame at the target zoom level based on the second image frame.

[0108] In the eighteenth aspect, in conjunction with the seventeenth aspect, the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

[0109] In the nineteenth aspect, in conjunction with the eighteenth aspect, the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

[0110] In the twentieth aspect, in conjunction with one or more of aspects seventeen to nineteen, the position of the second lens is determined such that the captured second image frame has a margin of pixels relative to the target zoom level, in order to correct for movement indicated by the motion information.

[0111] In the twenty-first aspect, in conjunction with one or more of aspects seventeen to twentieth, the operation may further include determining the second image stabilization process of the VOZ system based on the motion information and the target zoom level.

[0112] In a twenty-second aspect, in conjunction with the twenty-first aspect, the first image stabilization process may include an optical image stabilization (OIS) process, and the second image stabilization process may include an electronic image stabilization (EIS) process.

[0113] In the twentieth aspect, in conjunction with one or more of aspects seventeen to twenty-two, the first image stabilization process may include an electronic image stabilization process, and the output image frame may be determined by applying the first image stabilization process to the second image frame.

[0114] A twenty-fourth aspect includes a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations including: receiving a first image frame from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; determining that motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; determining a second lens position associated with a second zoom level of the VOZ system based on the motion information exceeding the predetermined threshold; receiving a second image frame captured by the VOZ system at the second zoom level; and stabilizing the second image frame by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining an output image frame at the target zoom level based on the second image frame.

[0115] In the twenty-fifth aspect, in conjunction with the twenty-fourth aspect, the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

[0116] In the twenty-sixth aspect, in conjunction with the twenty-fifth aspect, the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

[0117] In the twenty-seventh aspect, in combination with one or more of the twenty-fourth to twenty-sixth aspects, the position of the second lens is determined such that the captured second image frame has a margin of pixels relative to the target zoom level, in order to correct for movement indicated by the motion information.

[0118] In the twentieth aspect, in combination with one or more of the twentieth to twenty-seventh aspects, the operation may further include determining the second image stabilization process of the VOZ system based on the motion information and the target zoom level.

[0119] In the twentieth aspect, in conjunction with the twentieth aspect, the first image stabilization process may include an optical image stabilization (OIS) process, and the second image stabilization process may include an electronic image stabilization (EIS) process.

[0120] In the thirtieth aspect, in combination with one or more of the twenty-fourth to twenty-ninth aspects, the first image stabilization process may include an electronic image stabilization process, and the output image frame may be determined by applying the first image stabilization process to the second image frame.

[0121] In the accompanying drawings, a single block can be described as performing one or more functions. The one or more functions performed by this block can be performed in a single component or across multiple components, and / or can be performed using hardware, software, or a combination of hardware and software. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps are described below in general terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art may implement the described functionality in different ways for each specific application, but such specific implementation decisions should not be construed as departing from the scope of this disclosure. Additionally, the example device may include components other than those shown, including well-known components such as processors, memory, etc.

[0122] The aspects of this disclosure are applicable to any electronic device that includes, is coupled to, or otherwise processes data from one, two, or more image sensors capable of capturing image frames (or “frames”). The terms “output image frame,” “modified image frame,” and “corrected image frame” can refer to an image frame that has been processed by any of the techniques disclosed to adjust the raw image data received from the image sensor. Furthermore, aspects of the disclosed techniques can be implemented for processing image data received from image sensors having the same or different capabilities and characteristics, such as resolution, shutter speed, or sensor type. Additionally, aspects of the disclosed techniques can be implemented in devices for processing image data, whether or not the device includes or is coupled to an image sensor. For example, the disclosed techniques may include operations performed by a processing device in a cloud computing system that retrieves image data previously recorded by a separate device having an image sensor for processing.

[0123] Unless explicitly stated otherwise in the following discussion, it should be understood that throughout this application, the use of terms such as “access,” “receive,” “transmit,” “use,” “select,” “determine,” “normalize,” “multiply,” “average,” “monitor,” “compare,” “apply,” “update,” “measure,” “derive,” “set,” “generate,” etc., refers to the actions and processes of a computer system or similar electronic computing device that manipulate data represented as physical (electronic) quantities in the registers and memories of the computer system and transform them into other data similarly represented as physical quantities in the registers, memories, or other such information storage, transmission, or display devices of the computer system. The use of different terms to refer to actions or processes of a computer system does not necessarily indicate different operations. For example, “determining” data can refer to “generating” data. Similarly, “determining” data can refer to “retrieving” data.

[0124] The terms "device" and "apparatus" are not limited to one or a specific number of physical objects (such as a smartphone, a camera controller, a processing system, etc.). As used herein, a device can be any electronic device having one or more components that can implement at least some parts of this disclosure. Although the description and examples herein use the term "device" to describe various aspects of this disclosure, the term "device" is not limited to a particular configuration, type, or number of objects. As used herein, an apparatus can include a device or part of a device for performing the described operations.

[0125] Certain components in a device or apparatus described as “parts for access,” “parts for receiving,” “parts for transmitting,” “parts for using,” “parts for selecting,” “parts for determining,” “parts for normalizing,” “parts for multiplying,” or other similarly named terms referring to one or more operations on data (such as image data) may refer to processing circuitry (e.g., application-specific integrated circuit (ASIC), digital signal processor (DSP), graphics processing unit (GPU), central processing unit (CPU), computer vision processor (CVP), or neural signal processor (NSP)) configured to perform the described functions by means of hardware, software, or a combination of hardware configured by software.

[0126] Those skilled in the art will understand that information and signals can be represented using any of a variety of different techniques and skills. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0127] The components, functional blocks, and modules described herein with respect to the accompanying figures cited above include processors, electronic devices, hardware devices, electronic components, logic circuits, memory, software code, firmware code, and so on, or any combination thereof. Software should be interpreted broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, programs, and / or functions, whether referred to as software, firmware, middleware, microcode, hardware description languages, or other terms. Furthermore, the features discussed herein may be implemented via dedicated processor circuitry, via executable instructions, or a combination thereof.

[0128] Those skilled in the art should understand that: reference Figure 3 and Figure 4 One or more boxes (or operations) described may be combined with one or more boxes (or operations) described in another figure in the reference diagram. For example, Figure 3 One or more boxes (or operations) can be connected with Figure 1 , Figure 2 and Figure 5 A combination of one or more boxes (or operations). For example, with... Figure 4 One or more associated boxes can be connected with and Figure 1 , Figure 2 and Figure 5 A combination of one or more associated boxes (or operations).

[0129] Those skilled in the art will also recognize that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above in general terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such specific implementation decisions should not be construed as departing from the scope of this disclosure. Those skilled in the art will also readily recognize that the order or combination of components, methods, or interactions described herein is merely illustrative, and that components, methods, or interactions of various aspects of this disclosure can be combined or performed in ways other than those illustrated and described herein.

[0130] The various exemplary logics, logic blocks, modules, circuits, and algorithmic processes described in conjunction with the specific implementations disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. The interchangeability of hardware and software has been generally described in terms of functionality and illustrated in the various exemplary components, blocks, modules, circuits, and processes described above. Whether such functionality is implemented in hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0131] Hardware and data processing means for implementing the various exemplary logic units, logic blocks, modules, and circuits described herein can be implemented or executed using general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic units, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, or any conventional processor, controller, microcontroller, or state machine. In some embodiments, the processor may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In some embodiments, specific processes and methods may be performed by circuitry specific to a given function.

[0132] In one or more aspects, the described functionality may be implemented in hardware, digital electronic circuits, computer software, firmware, including the structures disclosed in this specification and their structural equivalents or any combination thereof. Specific implementations of the subject matter described in this specification may also be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus.

[0133] If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted through a computer-readable medium. The processes of the methods or algorithms disclosed herein can be implemented in a processor-executable software module that can reside on a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that can be implemented to transfer a computer program from one location to another. Storage media can be any available medium accessible to a computer. By way of example and not limitation, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible to a computer. Additionally, any connection may be appropriately referred to as a computer-readable medium. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically reproduce data, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operation of a method or algorithm may reside as a set of code and instructions or any combination of code and instructions on a machine-readable medium and a computer-readable medium that may be incorporated into a computer program product.

[0134] Various modifications to the specific embodiments described in this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to some other specific embodiments without departing from the spirit or scope of this disclosure. Therefore, the claims are not intended to be limited to the specific embodiments shown herein, but are to be granted the broadest scope consistent with this disclosure, the principles disclosed herein, and the novel features.

[0135] Additionally, those skilled in the art will readily recognize that, for the convenience of describing the accompanying drawings, contrasting terms such as “upper” and “lower” or “front” and “back” or “top” and “bottom” or “forward” and “backward” are sometimes used, indicating relative positions on a correctly oriented page corresponding to the orientation of the drawings, and may not reflect the correct orientation of any device as implemented.

[0136] Certain features described in this specification in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as operating in certain combinations and even originally claimed in this way, one or more features from the claimed combination may be removed from that combination in some cases, and the claimed combination may be for sub-combinations or variations thereof.

[0137] Similarly, although operations are depicted in a specific order in the figures, this should not be construed as requiring such operations to be performed in the indicated specific order or sequential order, or to perform all illustrated operations to achieve the desired result. Furthermore, the figures may schematically depict one or more example processes in the form of flowcharts. However, other operations not depicted may be combined with the schematically illustrated example processes. For example, one or more additional operations may be performed before, after, simultaneously with, or between any illustrated operation. In some contexts, multitasking and parallel processing are advantageous. Moreover, the separation of the various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. Additionally, some other embodiments also fall within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired result.

[0138] As used herein (including the claims), the term "or" in a list of two or more items means that any one of the listed items may be used alone, or any combination of two or more listed items may be used. For example, if a composition is described as containing component A, B, or C, the composition may contain A alone; B alone; C alone; a combination of A and B; a combination of A and C; a combination of B and C; or a combination of A, B, and C. Additionally, as used herein (including the claims), "or" in a list of items beginning with "at least one of" indicates a separate list, such that a list such as "at least one of A, B, or C" refers to A or B or C or AB or AC or BC or ABC (i.e., A and B and C) or any combination of any of these items.

[0139] The term “substantially” is defined as being largely but not necessarily entirely what is specified (and includes what is specified; for example, substantially 90 degrees includes 90 degrees and substantially parallel includes parallel), as understood by one of ordinary skill in the art. In any specific implementation of the disclosure, the term “substantially” may be used in place of the “[percentage]” of the specified content, where the percentage includes 0.1%, 1%, 5%, or 10%.

[0140] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method, the method comprising: A first image frame is received from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; Determine that the motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; The position of the second lens associated with the second zoom level of the VOZ system is determined based on the movement information exceeding the predetermined threshold. Receive a second image frame captured by the VOZ system at the second lens position; as well as The second image frame is stabilized by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining the output image frame at the target zoom level based on the second image frame.

2. The method of claim 1, wherein the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

3. The method of claim 2, wherein the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

4. The method of claim 1, wherein the position of the second lens is determined such that the second image frame is captured having margin pixels relative to the target zoom level to correct for movement indicated by the motion information.

5. The method according to claim 1, further comprising determining a second image stabilization process of the VOZ system based on the motion information and the target zoom level.

6. The method of claim 5, wherein the first image stabilization process includes an optical image stabilization (OIS) process, and the second image stabilization process includes an electronic image stabilization (EIS) process.

7. The method of claim 1, wherein the first image stabilization process includes an electronic image stabilization process, and wherein the output image frame is determined by applying the first image stabilization process to the second image frame.

8. The method of claim 1, wherein the motion information includes an average value of motion measurements, a peak motion measurement, the number of failed image stabilization frames, or a combination thereof.

9. A system comprising: processor; and The memory stores instructions that, when executed by the processor, cause the processor to perform operations including: A first image frame is received from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; Determine that the motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; The position of the second lens associated with the second zoom level of the VOZ system is determined based on the movement information exceeding the predetermined threshold. Receive a second image frame captured by the VOZ system at the second lens position; as well as The second image frame is stabilized by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining the output image frame at the target zoom level based on the second image frame.

10. The system of claim 9, wherein the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

11. The system of claim 10, wherein the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

12. The system of claim 9, wherein the position of the second lens is determined such that the second image frame is captured having margin pixels relative to the target zoom level to correct for movement indicated by the motion information.

13. The system of claim 9, wherein the operation further comprises determining a second image stabilization process of the VOZ system based on the motion information and the target zoom level.

14. The system of claim 13, wherein the first image stabilization process includes an optical image stabilization (OIS) process, and the second image stabilization process includes an electronic image stabilization (EIS) process.

15. The system of claim 9, wherein the first image stabilization process includes an electronic image stabilization process, and wherein the output image frame is determined by applying the first image stabilization process to the second image frame.

16. The system of claim 9, wherein the motion information includes an average value of motion measurements, peak motion measurements, the number of failed image stabilization frames, or a combination thereof.

17. An image capturing device, the image capturing device comprising: Image sensor; Memory, the memory storing processor-readable code; and At least one processor, coupled to the memory and the image sensor, is configured to execute processor-readable code to cause the at least one processor to perform operations, said operations including: A first image frame is received from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; Determine that the motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; The position of the second lens associated with the second zoom level of the VOZ system is determined based on the movement information exceeding the predetermined threshold. Receive a second image frame captured by the VOZ system at the second lens position; and The second image frame is stabilized by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining the output image frame at the target zoom level based on the second image frame.

18. The image capture device of claim 17, wherein the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

19. The image capture device of claim 18, wherein the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

20. The image capturing device of claim 17, wherein the position of the second lens is determined such that the second image frame is captured having a margin of pixels relative to the target zoom level to correct for movement indicated by the motion information.

21. The image capture device of claim 17, wherein the operation further comprises determining a second image stabilization process of the VOZ system based on the motion information and the target zoom level.

22. The image capture device of claim 21, wherein the first image stabilization process includes an optical image stabilization (OIS) process, and the second image stabilization process includes an electronic image stabilization (EIS) process.

23. The image capture device of claim 17, wherein the first image stabilization process includes an electronic image stabilization process, and wherein the output image frame is determined by applying the first image stabilization process to the second image frame.

24. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations, the operations including: A first image frame is received from a variable optical zoom (VOZ) system, the first image frame being captured at a first lens position associated with a first zoom level of the VOZ system, and wherein the first image frame is stabilized at a target zoom level using a first image stabilization process; Determine that the motion information of the VOZ system exceeds a predetermined threshold associated with the first image stabilization process; The position of the second lens associated with the second zoom level of the VOZ system is determined based on the movement information exceeding the predetermined threshold. Receive a second image frame captured by the VOZ system at the second zoom level; as well as The second image frame is stabilized by applying at least one of the first image stabilization process and a second image stabilization process different from the first image stabilization process, thereby determining the output image frame at the target zoom level based on the second image frame.

25. The non-transitory computer-readable medium of claim 24, wherein the predetermined threshold is determined based on the first zoom level and the first image stabilization process.

26. The non-transitory computer-readable medium of claim 25, wherein the predetermined threshold indicates the maximum movement that can be corrected at the first lens position by the first image stabilization process.

27. The non-transitory computer-readable medium of claim 24, wherein the position of the second lens is determined such that the second image frame is captured having margin pixels relative to the target zoom level to correct for movement indicated by the motion information.

28. The non-transitory computer-readable medium of claim 24, wherein the operation further comprises determining a second image stabilization process of the VOZ system based on the motion information and the target zoom level.

29. The non-transitory computer-readable medium of claim 28, wherein the first image stabilization process includes an optical image stabilization (OIS) process, and the second image stabilization process includes an electronic image stabilization (EIS) process.

30. The non-transitory computer-readable medium of claim 24, wherein the first image stabilization process includes an electronic image stabilization process, and wherein the output image frame is determined by applying the first image stabilization process to the second image frame.

Images