Zoom camera

Abstract

Zoom cameras are disclosed for seamless, smooth transition between a wide-angle field (FOV) camera and a scanning telephoto camera by correcting prism image differences, matching telephoto and wide-angle fields, positioning an associated telephoto field, and zooming a wide-angle frame to the positioned telephoto field.

Description

[0001] This application is a divisional application of the application with application number 202180013654.6 (PCT application number PCT / IB2021 / 062305), application date December 24, 2021, and invention title "Video Support in a Multi-Aperture Mobile Camera with a Scanning Zoom Camera".

[0002] Related Applications

[0003] This application claims the priority of U.S. Provisional Patent Application No. 63 / 130,655, filed on December 26, 2020, the entire text of which is incorporated herein by reference. Technical Field

[0004] The embodiments disclosed herein generally relate to various mobile cameras, particularly video support in multi-aperture cameras in the presence of scanning cameras. Background Art

[0005] Multi-cameras are standard in modern mobile electronic handheld devices ("mobile devices" such as smartphones, tablets, etc.), and dual-cameras are a subclass thereof. Multi-cameras typically include a wide angle or wide field of view (FOV W ) camera ("wide camera" or "WC") and one or more additional cameras, or have a narrower field of view (FOV) (with a "native telephoto field of view ("native FOV T , native FOV T or n-FOV T ") telephoto or "tele" camera), or have an ultra-wide field of view (FOV UW ).

[0006] A "scanning tele camera" ("STC") covers or "scans" a segment of a scene larger than the n-FOV T . This segment is called the "scanning telephoto field of view" or "s-FOV T ". The field of view scanning can be performed by rotating one or more optical path folding elements ("OPFE"). For example, the scanning tele camera (STC) is described in U.S. Patent No. 10,578,948 jointly owned by the inventors of this application.

[0007] For a dual camera that includes a wide-angle camera (WC) and a telephoto camera, when the user zooms in or out in the video operation mode (image stream), a wide-angle image or a telephoto image is displayed to the user. When switching the "active" camera, that is, switching the camera that displays the output image (from telephoto to wide-angle, or vice versa), the user will see a jump or interruption in the video. This jump is caused by the different points of view (POV) of two or more cameras. The point of view of a camera is defined as a unit vector, which is fully described by an origin and a direction. The origin is the center of the aperture of the corresponding camera. The direction of the point of view is given by the linear connection between the origin and the center of the specific field of view of the corresponding camera. In known dual cameras incorporated in mobile devices, the origins of the two cameras are slightly different, for example, 5 - 25 millimeters.

[0008] "Smoothing transition" (ST) in the displayed video is a software function that minimizes the jump when switching between image streams of different cameras (with different points of view). In current multi-cameras, the center of the telephoto field of view (FOV T ) is substantially the same as the center of the wide-angle field of view (FOV W ) (that is, the center of the telephoto field of view is substantially the same as the center of the wide-angle field of view). For mobile devices such as smartphones that incorporate multi-cameras, the point of view of the wide-angle camera and the point of view of the telephoto camera are substantially parallel to the normal of the rear surface of the smartphone. However, for a scanning telephoto camera (STC), the n-FOV T and the point of view are not fixed, but they scan within the scanning telephoto field of view (s-FOV T ). Therefore, generally speaking, when incorporated in a smartphone, the center of the n-FOV T is not the same as the center of the wide-angle field of view, and the point of view of the scanning telephoto camera (STC) is not parallel to the normal of the rear surface of the smartphone.

[0009] It is necessary and beneficial to have a smoothing transition feature for smooth switching between multiple wide-angle images ("WIs") and multiple scanning telephoto images ("STIs") in the video image stream. SUMMARY OF THE INVENTION

[0010] The present disclosure relates to systems and methods for obtaining smooth (i.e., seamless) transitions between cameras or viewpoints, which minimize jump effects in video operation modes during zoom-in when switching from displaying wide-angle image data to scanning long-focus image data, or during zoom-out when switching from displaying scanning long-focus image data to wide-angle image data. Hereinafter, for simplicity, "wide-angle image data" may be replaced by "wide-angle image" or "WI", and "scanning long-focus image data" may be replaced only by "scanning long-focus camera (STC) data", "scanning long-focus image", or "STI".

[0011] In various examples, a plurality of zoom cameras are provided, including: a wide-angle camera having a wide-angle field of view and configured to output a plurality of wide-angle images (WIs); a scanning long-focus camera (STC) having a long-focus field of view (n-FOV T ) that is narrower than the wide-angle field of view and is configured to output a plurality of scanning long-focus images; and a camera controller operatively coupled to the wide-angle camera and the scanning long-focus camera and configured to stream a video image stream that displays a continuous zoom-in action towards a non-central region or a continuous zoom-out action away from the non-central region within the wide-angle field of view, wherein the plurality of video images have a smooth transition when switching from displaying a plurality of wide-angle images to displaying a plurality of scanning long-focus images or vice versa.

[0012] In some examples, the smooth transition may be achieved by one or more of the following: performing a rotation correction of the plurality of scanning long-focus images, performing a registration between the plurality of wide-angle images and the plurality of scanning long-focus images, or performing a positioning between the plurality of wide-angle images and the plurality of scanning long-focus images to perform position matching. The positioning between the plurality of wide-angle images and the plurality of scanning long-focus images improves the accuracy of position matching by more than 2.5 times compared to the position matching before the positioning.

[0013] In some examples, the smooth transition can be achieved in one of the following ways: when switching from displaying multiple wide-angle images to displaying multiple scanned telephoto images, moving the multiple scanned telephoto images according to the distance of an object in a region of interest (ROI) of a scanned telephoto image relative to the multiple wide-angle images, and / or when switching from displaying multiple scanned telephoto images to displaying multiple wide-angle images, moving the multiple wide-angle images according to the distance of an object in a region of interest of a wide-angle image relative to the multiple scanned telephoto images; applying blurring to the multiple wide-angle images and / or the multiple scanned telephoto images; mixing the multiple wide-angle images and the multiple scanned telephoto images; matching the scale and / or brightness and / or color between the multiple wide-angle images and the multiple scanned telephoto images; or cropping the multiple wide-angle images such that the respective cropping offsets of the multiple wide-angle images displayed in sequence are on a line. The cropping of the multiple wide-angle images may include cropping such that the distance between the respective cropping offsets of the multiple wide-angle images displayed in sequence changes linearly with a zoom factor, or cropping such that the distance between the respective cropping offsets of the multiple wide-angle images displayed in sequence changes according to the square of a zoom factor. A cropping offset and / or a cropping center and / or a cropping coefficient may be selected such that a selected object is included in a cropped scanned telephoto image displayed to a user. The selected object may be located at a specific position in the cropped scanned telephoto image.

[0014] In some examples, the smooth transition can be achieved by cropping the multiple wide-angle images such that the respective coordinates of the center of a wide-angle field of view of the multiple wide-angle images displayed in sequence are on a line. The cropping of the multiple wide-angle images may include cropping such that the distance between the respective coordinates of the center of the wide-angle field of view of the multiple wide-angle images displayed in sequence changes linearly with a zoom factor, or cropping such that the distance between the respective coordinates of the center of the wide-angle field of view of the multiple wide-angle images displayed in sequence changes according to the square of a zoom factor.

[0015] In some examples, the smooth transition can be achieved by cropping the multiple wide-angle images such that a specific target area is always displayed in the video image stream, or by cropping the multiple wide-angle images such that the respective coordinates of a specific target area in the multiple wide-angle images displayed in sequence are on a line. The cropping may be such that the distance between the respective coordinates of the specific target area in the multiple wide-angle images displayed in sequence changes linearly, or according to a square law.

[0016] In some examples, the switch from displaying multiple wide-angle images to displaying multiple scanned telephoto images can be performed with an upward shift of the ZF value (ZF 向上 ) and the switch from displaying multiple scanned telephoto images to displaying multiple wide-angle images is at a downward shift of the ZF value (ZF向下 ) is executed with the upward transfer ZF value ≥ the downward transfer ZF value.

[0017] In some examples, the switch from displaying multiple scanned long - focal - length images to displaying multiple wide - angle images can be made with a downward transfer ZF value (ZF 向下 ) and the downward transfer ZF value (ZF 向下 ) depends on the native FOV within the wide - angle field of view (FOV W ) at a viewpoint, that is, ZF T (n - FOV T ) such that ZF 向下 = ZF 向下 (x, y), where ZF 向下 (the central point of view (POV)) is the downward transfer ZF value of the central point of view within the wide - angle field of view (FOV W ), and ZF 向下 (the edge point of view (POV)) is the downward transfer ZF value of the edge point of view within the wide - angle field of view, and where ZF 向下 (central point of view) < ZF 向下 (edge point of view).

[0018] In some examples, the switch from displaying multiple wide - angle images to displaying multiple scanned long - focal - length images can be made with an upward transfer ZF value (ZF 向上 ) where ZF 向上 does not depend on the viewpoints of the native long - focal - length field of view (n - FOV W ) within the wide - angle field of view (FOV T ), and where [ZF 向下 (x, y)] 最大 is the maximum value of ZF T (x, y) for all possible viewpoints within the scanned - type long - focal - length camera (STC)'s scanned field of view (s - FOV 向下 ), where ZF 向上 ≥ [ZF 向下 (x, y)] 最大 .

[0019] In some examples, the switch from displaying multiple wide - angle images to displaying multiple scanned long - focal - length images can be made with ZF 向上 and depends on the viewpoints of the n - FOV within the wide - angle field of view, where ZF T = ZF 向上 = ZF 向 up(x, y).

[0020] In some examples, the switch from displaying multiple wide - angle images to displaying multiple scanned long - focal - length images can be made with ZF 向上 and the switch from displaying multiple scanned long - focal - length images to displaying multiple wide - angle images can be made with ZF 向下Execute, where ZF in video shooting mode 向上 and / or ZF 向下 has a value that is 5%-30% greater than ZF in still shooting mode 向上 and / or ZF 向下 . The value of ZF 向上 and / or ZF 向下 can depend on the wide angle shown to the user and the aspect ratio of the multiple scanned telephoto images respectively. ZF in digital image stabilization video mode 向上 and / or ZF 向下 can have a value that is 5%-30% greater than ZF in non-image stabilization video mode 向上 and / or ZF 向下 .

[0021] In some examples, in digital image stabilization video mode, the image stabilization ability at the central viewing point can be 5%-30% greater than the image stabilization ability at the edge viewing point.

[0022] In some examples, smooth transition can be achieved by cropping the multiple scanned telephoto images such that a specific position of a selected object in the cropped scanned telephoto images shown to a user in two consecutive images of the video image stream does not change by >10 pixels. In certain examples, the specific position of the selected object may not change by >5 pixels. In some examples, the specific position of the selected object in the cropped scanned telephoto images shown to the user is selected according to aesthetic criteria.

[0023] In some examples, the camera controller can additionally be configured to evaluate a non-switching criterion before switching from displaying multiple wide angle images to displaying multiple scanned telephoto images, and if the non-switching criterion is met, not switch from displaying multiple wide angle images to displaying multiple scanned telephoto images. The non-switching criterion can be selected from a group consisting of: motion blur, electronic noise, jelly effect, defocus blur, and incorrect image alignment or obstacles. The non-switching criterion can include a significant incorrect position of a region of interest (ROI) in the scanned telephoto image relative to its position in a wide angle image or the lack of the region of interest in the scanned telephoto image; an imperfect roll correction of the scanned telephoto image; a check to determine whether multiple target coordinates are included in a scan field of view (s-FOV T ) of the scanned telephoto camera; a check to determine whether multiple target coordinates are close to multiple edges of the s-FOV T of the scanned telephoto camera to prevent jumping back and forth between the wide angle image and the scanned telephoto image; or a check to determine whether multiple target coordinates are moving faster than an expected maximum prism scan speed.

[0024] Zoom cameras above or below may be included in smartphones.

[0025] In some examples, a method is provided, comprising: using a device having a wide angle field of view (FOV W ) to output a plurality of wide-angle images (WI); using an n-FOV having a narrower field of view than the wide-angle field of view T A scanning telephoto camera (STC) is used to output a plurality of scanning telephoto images; and a camera controller is configured, the camera controller being operably coupled to the wide-angle camera and the scanning telephoto camera to stream a video image stream, the video image stream showing a continuous zoom-in action toward a non-central area within the wide-angle field of view or a continuous zoom-out action away from the non-central area, and providing a smooth transition of the plurality of video images when switching from displaying the plurality of wide-angle images to displaying the plurality of scanning telephoto images or vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Non-limiting examples of the disclosed subject matter will be described below with reference to the accompanying drawings listed after this paragraph. The same structure, element, or component appearing in multiple drawings will be labeled with the same reference numerals in the drawings in which they appear. The drawings and descriptions are intended to illustrate and clarify the embodiments disclosed herein and should not be construed as limiting in any way.

[0027] Figure 1 An embodiment of a mobile device including multiple cameras disclosed herein is schematically shown;

[0028] Figure 2 Different optical path folding element (OPFE) positions in the object field and their respective fields of view (FOV) are shown;

[0029] Figure 3 The main steps of the method disclosed herein for positioning a scanning telephoto camera (STC) image within a wide-angle image are shown in the form of a flow chart;

[0030] Figure 4A shows the positioning of the T image within the W image in one view;

[0031] Figure 4B shows the positioning of the T image within the W image in another view;

[0032] Figure 5A shows a zoomed-in scene displayed in a smooth transition video sequence created by a first method embodiment;

[0033] Figure 5B Shows Figure 5AMultiple images in an enlarged scene that have been digitally scaled to have the same aspect ratio;

[0034] Figure 5C shows Figure 5B some of the images and the target positions in;

[0035] Figure 6 shows the main steps of the cropping method in the form of a flowchart;

[0036] Figure 7 shows another embodiment of a method disclosed herein for implementing a smooth transition experience in a multi-camera system that includes at least one camera with scanning capabilities. Detailed Description

[0037] Figure 1 Schematically shows an embodiment of a mobile device (e.g., a smartphone) labeled 100 that includes a dual camera. The dual camera includes a scanning telephoto camera ("STC") 110 having a given native telephoto field of view (FOV T )(“n-FOV T ”) and an effective or scanned FOV T (“scanning telephoto field of view (s-FOV T )”). The scanning telephoto camera 110 can be a folding camera that includes an optical path folding element (OPFE) 118, a telephoto lens module 112 having a telephoto lens, and a telephoto image sensor 114. The telephoto lens can have a fixed effective focal length (EFL) that provides a fixed zoom factor (ZF), or an adjustable (variable) effective focal length that provides an adjustable zoom factor. The adaptation of the EFL can be discrete or continuous. The scanning telephoto camera (STC) 110 also includes a lens actuator 116 for moving the lens module 112 for focusing and / or optical image stabilization (OIS); an optical path folding element actuator 122 for actuating the optical path folding element 118 for optical image stabilization and / or for scanning the n-FOV T into a specific point of view (POV) within the scanning telephoto field of view (s-FOV T ); and a first memory 124. The memory 124 can be an EEPROM (electrically erasable programmable read-only memory). In some embodiments, the first memory 124 can store first calibration data. The EFL of the scanning telephoto camera (STC) 110 is 8 millimeters to 50 millimeters or greater, the diagonal n-FOV T is 10 - 40 degrees, and the f-value is approximately f / # = 1.5 - 6.

[0038] n-FOVT The scanning of T is performed at a finite maximum speed, i.e., some setup time is required. N-FOV T The scanning of T can be performed on a time scale of about 1 - 30 milliseconds for scanning 2° - 5°, and on a time scale of about 10 - 80 milliseconds for scanning 10 - 25°. In some embodiments, the scanning of the long focal length field of view (s-FOV T ) can cover about 50% of the area of the wide field of view (FOV W ). In some embodiments, the scanning of the long focal length field of view can cover 100% or more of the area of the wide field of view.

[0039] In some embodiments, the n-FOV T scanning can be performed by actuating two or more optical path folding elements instead of a single optical path folding element, as described in International Patent Application No. PCT / IB2021 / 059843 jointly owned by the inventors of the present application.

[0040] The dual camera also includes a wide-angle camera ("WC") 130, which has a wide field of view greater than the n-FOV of the scanning long focal length camera (STC) 110 T . The wide-angle camera (WC) 130 includes a wide-angle lens module 132 having a wide-angle lens and a wide-angle image sensor 134. The second lens actuator 136 can move the lens module 132 for focusing and / or optical image stabilization. In some embodiments, the second calibration data can be stored in the second memory 138. In other embodiments, the first calibration data and the second calibration data can be stored in the third memory 170. The first and second calibration data can include calibration data between the wide-angle camera (WC) 130 and the scanning long focal length camera (STC) 110. The EFL of the wide-angle camera (WC) may be 2.5 - 20 millimeters, the diagonal field of view is 50 - 130 degrees, and the f / # is about 1.0 - 2.5.

[0041] The mobile device 100 may also include an application processor (AP) 140. The application processor 140 may include a wide-angle image signal processor (ISP) 144 and a telephoto image signal processor 142. The application processor 140 may also include a camera controller 150 having a sensor control unit 152, a user control unit 160, a video processing unit 154, and a post-processing unit 146, all of which are operatively coupled to image sensors 114 and 134. The user control unit 160 may include an operation mode selection function 162 for selecting whether to capture an image or video, a region of interest (ROI) function 164 for selecting a region of interest, the “target coordinates” of the region of interest or its specific viewpoint (the “specific viewpoint” and “target coordinates” are used interchangeably hereinafter), and a zoom factor (ZF) module 166 for selecting a zoom factor. The region of interest may be a section within the wide-angle field of view or the scanned telephoto field of view selected by the user or an algorithm. For the user, the region of interest may be of higher value than other segments, for example, because it contains a specific object or a specific combination of objects. Generally, the wide-angle camera (WC) focuses on one of the multiple regions of interest, and the scanned telephoto camera (STC) turns towards and / or focuses on one of the multiple regions of interest.

[0042] In use, the application processor 140 may receive respective wide-angle and scanned telephoto camera (STC) image data from cameras 110 and 130, and provide camera control signals to cameras 110 and 130.

[0043] The sensor control unit 152 is operatively connected to the two image signal processors (142 and 144) and the user control unit 160, and may be used to select which image sensor is operable according to the zoom factor and provide sensor control signals. The video processing unit 154 may be configured to evaluate a non-switching criterion to make a decision regarding the video output. Specifically, when evaluating the non-switching criterion, if the non-switching criterion is met, the module 154 may be configured to output a scaled video output image that includes only wide-angle image data during the zoom operation. The post-processing module 146 may be used for image processing, including denoising, sharpening, scaling, etc.

[0044] Herein and hereinafter, unless otherwise specified, the following definitions will be used:

[0045] Input image: An image provided by the telephoto image signal processor 142 or the wide-angle image signal processor 144.

[0046] Scanned telephoto image (STI): An image provided by the telephoto image signal processor 142.

[0047] Wide-angle image (WI): An image provided by the wide-angle image signal processor 144.

[0048] Output image: The image shown to the user (usually a specific image of a video stream).

[0049] Tele output image: Based on the scanned tele image data and the output image provided in step 608 ( Figure 6 ) or step 708 ( Figure 7 ).

[0050] Wide output image: According to the wide-angle image data and the output image provided in step 608 ( Figure 6 ) or step 708 ( Figure 7 ).

[0051] Output image data switching: In the image video stream, the action of switching from showing the tele output image or the wide output image to the user to showing the wide output image or the tele output image to the user respectively.

[0052] "Image data" and "data" or "image" are used interchangeably hereinafter.

[0053] Zoom in and out in the still camera mode

[0054] Defined as follows: TFOV = tan(camera FOV / 2). "Low zoom factor (ZF)" refers to all zoom factors that meet ZF < TFOV W / TFOV T ."High zoom factor (ZF)" refers to all zoom factors that meet ZF > TFOV W / TFOV T ."Transfer zoom factor (Transfer ZF)" or "ZFT" refers to the zoom factor that meets ZF = TFOV W / TFOV T . ZFT represents the minimum zoom factor that allows output image data switching. In one embodiment, zoom in and out in the still mode can be performed as follows:

[0055] Zoom in: When the ZF is slightly higher than ZFT in the low ZF 向上 , the output image is a digitally scaled wide output image. When the zoom factor is transferred upward (up-transfer ZF, "ZF 向上 "), and ZF 向上 ≥ZFT, when performing the switching of the output image data from showing the wide output image to showing the tele output image, move and correct the scanned tele image as described herein to achieve a smooth transition (ST). For ZF ≥ ZF 向上 , the output is a digitally scaled tele output image.

[0056] Zoom out: When the ZF is high 向下When slightly above ZFT, the output image is a digitally zoomed telephoto output image. When down-transferring the zoom factor (down-transfer ZF, "ZF 向下 "), ZF 向下 ≥ZFT, when performing the switching of output image data from displaying the telephoto output image to displaying the wide-angle output image, move and correct the wide-angle image as described herein to achieve a smooth transition. For ZF≤ZF 向下 , the output is a digitally zoomed wide-angle output image.

[0057] "Slightly above ZFT" means that the zoom factor is about 1%-25% higher than ZFT.

[0058] In some examples, a global registration algorithm can be used to achieve a smooth transition.

[0059] In some embodiments, ZF 向上 and / or ZF 向下 can be a fixed zoom factor, while ZF 向上 ≤ZF 向下 or ZF 向上 ≥ZF 向下 .

[0060] In other embodiments, ZF 向上 and / or ZF 向下 may not be fixed zoom factors, but they may vary ("dynamic transfer ZF"), for example, depending on the viewing point or target coordinates (i.e., the position of n-FOV T within s-FOV T ), i.e., ZF 向下 =ZF 向下 (x, y) and / or ZF 向上 =ZF 向上 (x, y), where "(x, y)" represents the coordinates or viewing point within s-FOV T .

[0061] In some embodiments, in the central region of s-FOV T , ZF 向下 may be less than that in the edge region of s-FOV T , i.e., ZF 向下 (central region)<ZF 向下 (edge region). "Central region" and "edge region" are defined as mutually exclusive. In one example, referring to the s-FOV Figure 2 shown in T 200, the central region can be defined by all viewing points within the pointed rectangular box, and the rectangular box has the same size as s-FOV T200 with the same aspect ratio and center, but only 50% of the height and width of 200 respectively. Any viewing point that points to 200 but not to the central region is included in the edge region. In other examples, the rectangular box may have only 30% of the height and width of 200, or may have up to 80% of the height and width of 200, and the definitions of the central region and the edge region will change accordingly. In a scanning telephoto camera (STC) based on a single optical path folding element, the scanned telephoto image captured in the central region of the s-FOV contains a larger field of view section in the object domain than the scanned telephoto image captured in the edge region. This is because in order to obtain a rectangular corrected scanned telephoto image with a specific aspect ratio from the unprocessed (i.e., uncorrected, uncropped, etc.) scanned telephoto image at the central position within the s-FOV, less cropping amount (i.e., less loss of scene information) is required compared to obtaining a rectangular corrected scanned telephoto image with the same specific aspect ratio from the unprocessed scanned telephoto image at the edge position within the s-FOV. The object domain is defined as the actual (or physical) scene, i.e., the scene captured by an ideal camera with a sufficiently large field of view and without any aberrations and distortions. That is to say, the object domain corresponds to the appearance of the scene as it appears to a human observer. Therefore, the ZF in the central region and the ZF can be smaller than that in the edge region. T 200 and the width of 50%. Any viewing point that points to 200 but not to the central region is included in the edge region. T 200 but does not point to the central region is included in the edge region. In other examples, the rectangular box may have only 30% of the height and width of 200, or may have up to 80% of the height and width of 200, and the definitions of the central region and the edge region will change accordingly. In a scanning telephoto camera (STC) based on a single optical path folding element, the scanned telephoto image captured in the central region of the s-FOV contains a larger field of view section in the object domain than the scanned telephoto image captured in the edge region. T 200 of the height and width, or may have up to 80% of the height and width of 200, and the definitions of the central region and the edge region will change accordingly. In a scanning telephoto camera (STC) based on a single optical path folding element, the scanned telephoto image captured in the central region of the s-FOV contains a larger field of view section in the object domain than the scanned telephoto image captured in the edge region. T 200 of the height and width, the central region and the edge region of the definition will change accordingly. In a scanning telephoto camera (STC) based on a single optical path folding element, the scanned telephoto image captured in the central region of the s-FOV contains a larger field of view section in the object domain than the scanned telephoto image captured in the edge region. T of the central region captured scanning telephoto image contains in the object domain than in the edge region captured scanning telephoto image larger field of view section. This is because in order to from the s-FOV T within the central position of the unprocessed (i.e., uncorrected, uncropped, etc.) scanning telephoto image obtained with a specific aspect ratio of the rectangular corrected scanning telephoto image, and from the s-FOV T within the edge position of the unprocessed scanning telephoto image obtained with the same specific aspect ratio of the rectangular corrected scanning telephoto image, requires less cropping amount (i.e., less scene information loss). The object domain is defined as the actual (or physical) scene, that is, by having a sufficiently large field of view and no aberration and distortion of the ideal camera captured scene. That is to say, the object domain corresponds to the appearance of the scene, as it is in the human observer's view. Therefore, the ZF of the central region 向上 and ZF 向下 can be less than the edge region.

[0062] The advantage of using dynamic transfer ZF is that at least in the central region of the s-FOV, the image quality advantage of using a scanning telephoto camera (STC) can be enjoyed at a lower ZF. In the still photography mode, using dynamic up and down conversion zoom factors may be particularly beneficial. In some embodiments with fixed or dynamic ZF T and / or ZF 向上 and / or ZF 向下 In some embodiments, the ZF 向上 and / or ZF 向下 in the video photography mode (i.e., when the user captures a video stream) may be 5% - 30% larger than the ZF 向上 and / or ZF 向下 in the still photography mode. In some embodiments, the ZF 向上 and / or ZF 向下 may depend on the aspect ratio of the wide-angle output image and / or the telephoto output image.

[0063] In some embodiments, where ZF 向上 = ZF 向上(x, y), ZF 向上 (central region) <ZF 向上 (peripheral region).

[0064] In some embodiments, ZF 向上 can be fixed, while ZF 向下 can be dynamic, i.e., ZF 向下 = ZF 向下 (x, y). Fixed ZF 向上 can satisfy ZF 向上 ≥ [ZF 向下 (x, y)] 最大 , i.e., fixed ZF 向上 is defined by the maximum value of ZF 向下 (x, y) when considering all viewpoints within the s-FOV T . The maximum value of ZF 向下 (x, y) is the maximum value of ZF 向下 . This can be beneficial because it can prevent the need to perform a switch in output image data from multiple scanned telephoto images to multiple wide-angle images even when the zoom factor does not change. An example of such an undesirable situation involves switching the output image data from multiple wide-angle images to a scanned telephoto image at the central position at the target coordinates (where ZF 向上 can be less than the peripheral region) and then using a scanned telephoto camera (STC) to track a moving object. The object (i.e., the target coordinates) may now move to a more peripheral position within the s-FOV T , where the n-FOV T of this peripheral position is not large enough, so the field of view defined by the corresponding ZF can no longer be supported by multiple scanned telephoto images. Due to the incomplete scanned telephoto image data, a switch in output image data from multiple scanned telephoto images to multiple wide-angle images needs to be performed even though ZF has not changed. Using a fixed ZF 向上 to satisfy ZF 向上 ≥ [ZF 向下 (x, y)] 最大 can be particularly useful in video mode.

[0065] Smooth transition

[0066] To achieve smooth transition, matching of the position, scale, brightness, and color of the output image can be performed before and / or after switching the output image data. However, in many cases, image position matching between the entire wide-angle image and the scanned telephoto image is impossible, for example, because of parallax. Thus, in the smooth transition as disclosed herein, position matching can be achieved only in the region of the region of interest, while scale, brightness, and color are matched for the entire output image region. Specifically, when switching from displaying multiple wide-angle images to displaying multiple scanned telephoto images, multiple scanned telephoto images can be moved according to the distance of an object in the region of interest of the scanned telephoto images relative to the multiple wide-angle images, and / or when switching from displaying scanned telephoto images to displaying wide-angle images, by moving multiple wide-angle images according to the distance of an object in the region of interest of the wide-angle images relative to the multiple scanned telephoto images.

[0067] Figure 2 Shows different optical path folding element positions in the object domain and their respective n-FOVs T , as described in more detail in International Patent Application No. PCT / IB2021 / 056311. The object domain is different from the "image domain", which is defined as the scene with aberration and distortion captured by the camera. In the present invention, the image domain is defined as the scene captured by a scanned telephoto camera (STC) such as camera 110. Box 200 represents the s-FOV T , i.e., the smallest rectangular field of view that contains all the scanned telephoto camera (STC) image data in all viewpoints and can be achieved using a specific scanned telephoto camera (STC) in the target domain. The n-FOVs T for three different optical path folding element positions (0, 1, and 2) are represented by 202-0, 202-1, and 202-2. The n-FOV T 202-0 of the optical path folding element "zero position" is defined as the n-FOV that produces an object or scene image without viewpoint aberration T . That is, at the zero position, the object in the object domain is the same as the object image in the image domain (except for scaling, image sensor noise, etc.). Generally speaking, in the zero position, the n-FOV T and the center of the wide-angle field of view overlap. As shown in the figure, the n-FOVs T at any other position (such as 202-1 and 202-2) are not horizontal rectangles (such as 202-0), but arbitrary quadrilaterals. The same rectangular object is represented by 204-0, 204-1, and 204-2 in the n-FOVs T 202-0, 202-1, and 202-2 respectively.

[0068] Figure 3Shows the main steps of the method disclosed herein for localizing a scanned telephoto image within a wide-angle image; a scanned telephoto image is captured in step 302. Before capture, the n-FOV T is scanned towards a specific viewing point. As described in International Patent Application No. PCT / IB2021 / 056311, in step 304, the scanned telephoto image is corrected. By using suitable cropping selection criteria, the content contained in the corrected and trimmed scanned telephoto image, as well as the positions of multiple objects in the corrected and trimmed scanned telephoto image, can be manipulated. The cropping selection criteria refer to the specific criteria for using a specific scanned telephoto image captured at a specific n-FOV T to correct and crop the specific scanned telephoto image. Cropping includes defining a cropping offset and / or a cropping center and / or a cropping factor. The cropping offset is the position of the upper left corner of the image in the object domain. The cropping factor is typically given by 1 / ZF.

[0069] In some examples, the cropping can be modified so that a selected object is included in the cropped scanned telephoto image.

[0070] In other examples, the cropping can be modified so that the selected object is located at a specific position in the cropped scanned telephoto image.

[0071] In some examples, for video mode, the cropping can be modified so that the change in the position of the selected object in two consecutive frames of the video stream does not exceed 10%, 5%, or 1% of the width and / or height of the telephoto sensor.

[0072] In other examples, the position of the selected object in two consecutive frames can vary by less than 50 pixels or less than 20 pixels, or even less than 5 pixels (assuming a pixel size of 1 micron).

[0073] In yet another example, the cropping can be modified so that the position of the selected object in consecutive frames of the video stream changes only slowly, for example, at a frequency not higher than 10 pixels per second or 5 pixels per second or even 2 pixels per second (assuming a pixel size of 1 micron).

[0074] In other examples, the cropping can be modified so that the position of the selected object meets aesthetic criteria. The aesthetic criteria can be based on, for example, the position of the selected object in the cropped scanned telephoto image (aesthetic frame), such as the rule of thirds of a photographer, whether additional objects are included or excluded in the cropped scanned telephoto image, such as the leading line method of a photographer, etc. For example, as Figure 4A-B As described, in step 306, the positioning of the scanned telephoto image within the wide-angle image is performed. In step 308, image registration known in the art is performed between the cropped scanned telephoto image and the wide-angle image. Output image data switching can be performed in step 310, as outlined in more detail below. The same steps can be performed for the zoom-out case, where the output image data can be switched from the telephoto output image data to the wide-angle output image data. In some examples, the switching step may include the mixing or fusing of multiple scanned telephoto images and multiple wide-angle images.

[0075] Figure 4A -B shows the details of the positioning of the scanned telephoto image within the wide-angle image performed in step 306, as described in more detail in International Patent Application No. PCT / IB2020 / 061461 jointly owned by the inventors of the present application.

[0076] In Figure 4A the estimated viewpoint within the wide-angle field of view 404 shows the estimate 402 of the n-FOV T The estimated viewpoint refers to the viewpoint estimated based on calibration data. The estimate of the n-FOV T depends on calibration and may not be accurate enough in terms of matching multiple scanned telephoto images with multiple wide-angle images. Generally, before positioning, the image point coordinates of the same object point in the scanned telephoto image and the wide-angle image may deviate by more than 25, 50, or even 100 pixels. Assume that the size of one pixel is about 1 micron. Positioning (step 306) is performed to reduce this deviation, that is, to achieve a precise n-FOV T estimate with higher accuracy. The positioning includes:

[0077] 1. Select a search area 406a as shown in Figure 4A . The search area 406 can be defined by using the center of the n-FOV T estimate and (for example, symmetrically) embedding it in a rectangular area, where the area of the rectangle can be twice, three times, or four times the area covered by the n-FOV T estimate.

[0078] 2. Crop the search area from the wide-angle image.

[0079] 3. Template matching, where the source is represented by the cropped search area from the wide-angle image, and the template can be represented by the scanned telephoto image. Template matching can be performed by the correlation of the template at different positions or over the entire search area. The position with the highest matching value can indicate the best estimate of the n-FOV T position within the wide-angle field of view. This more accurate precise n-FOV T estimate is in Figure 4BIt is represented as 408 in the figure and can be further used to execute steps 308 and 310. Generally, after positioning, the deviation of the image points of the same object point between the wide-angle image and the scanned telephoto image may be less than 5 pixels, or even less than 2 pixels, that is, the accuracy is increased by 5, 10 or even 20 times.

[0080] Figure 5A -C shows a smooth transition video sequence created by the first cropping method ("Method 1") disclosed herein. Hereinafter, the index "0" represents the initial state, the index "f" represents the final (or target) state, and the index "t" represents some intermediate state. "State" refers to an image defined by ZF, the cropping coefficient, and the cropping offset. The "final state" refers to the last image shown in the smooth transition video sequence, which contains the wide-angle output image data. When further zooming in, the output image data switching can be performed.

[0081] Figure 5A It shows how to display the magnified scene in the smooth transition video sequence created by Method 1. Image 502 can represent the first image in the smooth transition video sequence. Image 502 may have ZF = 1.0, that is, it contains the full wide-angle field of view, or it may have ZF>1. Images 504-512 represent subsequent images in the smooth transition video sequence towards the "target" position 514, with coordinates (xf, xf) and the target ZF ("ZFf"). The target position 514 is the position towards which the magnification is directed. The n-FOV of the scanned telephoto camera (STC) T is scanned so that the target position 514 is located at the center of the n-FOV T The position 514 and ZFf can be selected by the user or the program before or at the start of the magnification action. Images 504-512 are wide-angle output images. The cropping offsets of images 504-512 are represented by image frames 516-528 respectively. The advantage of Method 1 is that the cropping is selected such that the target position 514 is included in each of images 502-512, that is, the target position 514 is shown throughout the smooth transition video sequence.

[0082] Figure 5B It shows images 504-512 that are subsequently displayed to the user in the magnified smooth transition video sequence, that is, they are digitally scaled to have the same aspect ratio.

[0083] Figure 5CDisplays image 502, image 512, and target position 514. Reference numeral 532 represents the initial X coordinate X0 of target position 514 in the coordinate system of image 502, and the number 534 represents the target X coordinate Xf of target position 514 in the coordinate system of image 512. The Y coordinates Y0 and Yf are defined correspondingly. The target position 514 is defined in the object domain. The widths and heights of images 502 and 512 (measured along the x-axis and y-axis respectively) are given by W502 and W512, H502 and H512 respectively. The relative positions (“relPos 0， x”) and (“relPosf ， x”) can be derived from 532 and 534 by relPos 0， x = X0 / W502 and relPosf ， x = xf / W512. The relative positions in the Y direction can be defined correspondingly. By gradually modifying the relative coordinates of the target position (in the X-axis and Y-axis), smooth transformation can be achieved. At the target ZFf, the relative position of target position 514 is relPosf ， x = relPosf ， y = 1 / 2, that is, the target position 514 coincides with the center of the output image.

[0084] Method 1 can be used similarly for reduction.

[0085] Figure 6 Shows the main steps of Method 1. It can start from the initial (or “0”) state, use a video stream with ZF0 < ZFT and display a wide-angle output image (such as images 502 - 512). The positioning of the n-FOV T within the wide-angle field of view (for example Figure 4A - as described in -B) is performed.

[0086] Generally, the relative position at some intermediate state (“t”) can be calculated according to relPost = (Xt / Wt, Yt / Ht), where Wt and Ht are the width and height of the output image displayed at the intermediate state t respectively, where Wt = W / ZFt and Ht = H / ZFt, to define the cropping factor, where W and H are the width and height of the untrimmed wide-angle image respectively.

[0087] The initial state relative position relPos0 is calculated from the target coordinates (X0, Y0) in the coordinate system of image 502 as relPos0 = (X0 / W0, Y0 / H0), where W0 = W / ZF0 and H0 = H / ZF0 are the width and height of the output image displayed at the initial state 0 respectively. The conversion slope S from relPos0 to the relative position in the target state relPosf is derived from the following formula:

[0088]

[0089] where ZF P is a preset and fixed ZF value that satisfies ZF P ≥ ZFT.

[0090] In step 602, the program or user uses a ZF (ZFt1) to trigger a command to zoom to the first intermediate state ("t1"), where ZFt1 ≥ ZFT, enabling the execution of output image data switching (from wide-angle image data to scanned telecamera (STC) image data). The initial output image can be an image showing the entire wide-angle field of view (i.e., having ZF0 = 1), or an image having 1 < ZF0 < ZFT, which shows a (digitally zoomed) section of the wide-angle field of view.

[0091] In step 604, the relative position update is calculated according to relPost1 = relPos0 + (ZFt1 - ZF0). Generally, the relative position update can be relPost = relPost -1 + (ZFt - ZFt -1 ) · S.

[0092] In step 606, the cropping factor and cropping offset are calculated. The cropping offset (X C , Y C ) is calculated by first calculating the relative coordinates as follows: Xt1 = relPost1,x · Wt1 and Yt1 = relPost 1， y · Ht1, where relPost ， x and relPost ， y are the x-value and y-value of relPost respectively. The position of the cropping offset (X C , Y C ) is calculated by subtracting Xt1 and Yt1 from the target coordinates (in the coordinate system of image t1).

[0093] In step 608, an image based on the wide-angle image data and cropped according to the cropping offset (X C , Y C ) and having an image size (Wt, Ht) is output and displayed in a smoothly transitioning video sequence. If further zooming is performed, the process can start again from step 602.

[0094] In some examples, the following output image data switching can be from using only wide-angle image input to using only scanned telephoto image input.

[0095] In the second cropping method ("Method 2"), smooth transition can be achieved by linearly modifying the cropping offset. One of them can consider the initial cropping offset (X C，0 , YC，0 ) and a linear connection between the target cropping offset (X C， f, Y C， f). In Method 2, when zooming in or out, the cropping offset is always located on this linear connection, as shown in Figure 5A . In some examples, the position of the cropping offset on the linear connection is linearly determined. In other examples, the position is determined by the square law. For example, when zooming from ZF = 1 to ZF = 2, the difference in the cropping offset position (i.e., the distance between the respective cropping offsets) on the linear connection is 2 times greater than when zooming from ZF = 2 to ZF = 4.

[0096] In a third cropping method (“Method 3”), smooth transition can be achieved by linearly modifying the center of the field of view shown in the smooth transition video stream. One possibility is to consider the linear connection between the initial center of the field of view and the target center of the field of view. In Method 3, when zooming in or out, the center of the field of view may always be located on this linear connection. The position of the cropping offset on the linear connection can be determined by the linear law or the square law.

[0097] Figure 7 An embodiment of the smooth transition method disclosed herein is shown. For each image of the video stream, in step 702, the wide-angle input image and the telephoto input image, as well as the viewpoints of the scanning telephoto camera (STC), are retrieved. The telephoto input image can be pre-corrected to align with the wide-angle viewpoint or corrected in step 702. Optionally, in step 702, calibration data can be used to convert the OFPE position to an n-FOV T estimate. For example, in the case of changing the n-FOV by scanning T , a registration procedure can be performed to calculate the translation and rotation parameters of the current scanned telephoto image relative to the wide-angle image. The translation can compensate for parallax, and the rotation can compensate for the residual rotation between the scanned telephoto image and the wide-angle image, which may be caused by imperfect calibration.

[0098] In some examples, the multi-camera may include image stabilization (IS), such as optical image stabilization (OIS) or electronic image stabilization (EIS) for a scanning telephoto camera (STC) and / or a wide-angle camera (WC). Since image stabilization may shift the wide-angle image and the scanning telephoto image relative to each other, in step 702, image stabilization control input data may be read, such as sensor data provided by an inertial measurement unit (IMU), or image stabilization control output data, such as a command to move a particular optical component by a given amount. The data may be converted to an expected pixel offset in the wide-angle image and the scanning telephoto image, and this conversion may be used to compensate for any undesired effects that the image stabilization mechanism may have on the smooth transition of the video sequence. As is well known, for electronic image stabilization, the input image is cropped and shifted so that, in two or more consecutive output images of the video stream, a selected object or the entire scene remains located at the same (or similar) particular position. To prevent jumps between the scanning telephoto camera (STC) and the wide-angle camera (WC), the ZF 向上 and / or ZF 向下 value in the digital image stabilization video mode may be 2.5% - 50% greater than the ZF 向上 and / or ZF 向下 value in the non-image stabilization video mode. Additionally, the ability to perform electronic image stabilization (e.g., measured by the maximum number of pixels that can be moved) may be 2.5% - 50% greater at the center viewing point than at the edge viewing point. Here, image stabilization can be used to achieve two different goals. The first goal may be to stabilize the wide-angle camera (WC) or the scanning telephoto camera (STC) at a frequency of approximately 50 - 100 Hz and higher, for example, to mitigate the user's hand shake. The first goal may be to stably construct the scene, i.e., to stabilize the wide-angle camera (WC) or the scanning telephoto camera (STC) at a frequency of approximately 20 Hz and lower (e.g., 1 Hz) to maintain a selected scene (of the object domain) within the wide-angle field of view or n-FOV T or at a particular position within the wide-angle field of view or n-FOV T In step 704, a wide-angle input image or a telephoto input image is selected as the output image. The wide-angle input image data may be selected for the following cases:

[0099] for any low ZF, or

[0100] for high ZF, but meeting one or more non-switching criteria. Evaluating the non-switching criteria avoids switching the output image data in situations where a smooth transition cannot be achieved or may be disadvantageous. After capturing a scanning telephoto image, the following non-switching criteria may be evaluated:

[0101] 1. Low-scan long-focus image quality. For example, due to defocus (defocus blur) of a scanning long-focus camera (STC), or due to motion blur, jelly effect artifacts, or other artifacts known in the art, low image quality may be caused by large electronic noise from low scene illumination.

[0102] 2. Inappropriate scanning long-focus image composition. For example, this may include the following scenarios: (i) the scene shown in the scanning long-focus image is significantly different from the scene shown in the wide-angle image, and in a smoothly transitioning video stream, it is output to the user before the scanning long-focus image, or (ii) the scanning long-focus image shows a scene that is semantically significantly inconsistent with the scene shown in the wide-angle image (e.g., motion occurring between the captured scanning long-focus image and the wide-angle image), or (iii) the scanning long-focus image does not show the region of interest (e.g., an obstacle) included in the wide-angle image, or it does show it at a significantly incorrect location relative to the region of interest in the wide-angle image (i.e., the target coordinates in the wide-angle camera (WC) coordinate system are significantly different from the target coordinates in the scanning long-focus camera (STC) coordinate system).

[0103] 3. Imperfect scanning long-focus image correction, such as imperfect rolling correction of the scanning long-focus image.

[0104] Before capturing a scanning long-focus image or scanning to specific target coordinates for the n-FOV T the following non-switching criteria can be evaluated:

[0105] 1. Effective scanning long-focus camera (STC) scanning range: Evaluate whether the target coordinates are included in the s-FOV T or not.

[0106] 2. Preventing jumps between the scanning long-focus camera (STC) and the wide-angle camera (WC): Evaluate whether the target coordinates are close to certain thresholds at the edge of the s-FOV T . In this case, even a small relative movement of the region of interest and the scanning long-focus camera (STC) may result in a situation where the region of interest can no longer be fully covered by the scanning long-focus image, so the wide-angle image must be used. To prevent jumps between the scanning long-focus camera (STC) and the wide-angle camera (WC), if the target coordinates are located at a distance of 5% or 10% or 20% from the s-FoV T (in the horizontal or vertical direction), then the output data of the scanning long-focus image cannot be switched to.

[0107] 3. Effective region of interest speed: Evaluate the speed of the region of interest (and the corresponding target coordinates) to analyze whether the n-FOV T can follow this motion.

[0108] In step 706, the input image selected for output is cropped. For example, based on the selected input image, the current ZF, image cropping required for the input image, etc. are calculated. The calculation can be performed according to cropping method 1, cropping method 2, or cropping method 3. The generated cropped image can be displayed (or output) to the user.

[0109] Optionally, and depending on, for example, a) the selected input image, b) the cropping of the input image, and / or c) the image registration parameters, a smoothly transformed output image can be provided in step 708.

[0110] In other examples, the methods disclosed herein may not include the action of cropping, but may only provide the parameters calculated as described herein, and the actions of steps 706 and 708 (such as cropping, providing a video, etc.) may be performed on different programs or processors, such as on dedicated hardware (HW) that supports hardware acceleration.

[0111] To compensate for the resolution differences between multiple scanned telephoto images and multiple wide-angle images, blurring can be applied to the output image.

[0112] Rotation correction can also be applied in step 708. The lens distortion difference between the wide-angle camera (WC) and the scanned telephoto camera (STC) is another challenge. For smooth transformation, distortion correction can be applied to all input images, or alternatively, digital distortion can be applied only to the telephoto input images or only to the wide-angle input images so that they match the corresponding other input images.

[0113] Unless otherwise specified, the expression "and / or" is used between the last two members of the list of options for selection to indicate that it is appropriate to select one or more of the listed options and selection can be made.

[0114] It should be understood that when a claim or specification refers to "a (a, an)" element, such a reference should not be construed as meaning only one of a plurality of elements.

[0115] All patents, patent applications, and publications mentioned in this specification are hereby incorporated by reference in their entirety to the same extent as if each individual patent, patent application, or publication was specifically and individually indicated to be incorporated by reference into this specification. Additionally, any reference documents cited or identified in this application should not be construed as admitting that such reference documents can be used as prior art for this disclosure.

Claims

1. A zoom camera, characterized in that, The zoom camera includes: A wide-angle camera having a wide-angle field of view FOVW and operable to output a plurality of wide-angle images WI; A foldable telephoto camera FTC includes an optical path folding element OPFE, and the foldable telephoto camera has a native telephoto field of view n-FOV T , the native telephoto field of view n-FOV is narrower than the wide-angle field of view FOVW, and the foldable telephoto camera FTC is operable to output a plurality of folded telephoto images FTI and perform optical image stabilization OIS or scan a scene having the native telephoto field of view n-FOV T ; and A camera controller, operably coupled to the wide-angle camera and the foldable telephoto camera FTC, and configured to stream a video image stream that is displayed within the wide-angle field of view FOV W with a continuous zoom-in action towards a non-central region or a continuous zoom-out action away from the non-central region Wherein when switching from displaying the plurality of wide-angle images to displaying the plurality of folded telephoto images or vice versa, the plurality of video images have a smooth transition.

2. The zoom camera according to claim 1, characterized in that, The wide-angle camera includes a wide-angle camera lens, wherein the wide-angle camera is operable to perform optical image stabilization OIS by actuating the wide-angle camera lens.

3. The zoom camera according to claim 1, characterized in that, Data from an inertial measurement unit IMU is used to calculate the expected pixel offsets in the plurality of wide-angle images WI and the plurality of folded telephoto images FTI.

4. The zoom camera according to claim 1, characterized in that, The zoom camera is further configured to perform electronic image stabilization EIS.

5. The zoom camera according to claim 4, characterized in that, Data from an inertial measurement unit IMU is used to calculate the expected pixel offsets in the plurality of wide-angle images WI and the plurality of scanned telephoto images STI.

6. The zoom camera according to claim 1, characterized in that, The smooth transition is achieved by performing a rotation correction of the plurality of folded telephoto images FTI.

7. The zoom camera according to claim 1, characterized in that, The smooth transition is achieved by performing a registration between the plurality of wide-angle images WI and the plurality of folded telephoto images FTI.

8. The zoom camera according to claim 1, characterized in that, The smooth transition is achieved by performing positioning between the plurality of wide-angle images WI and the plurality of folded telephoto images FTI to perform position matching.

9. The zoom camera according to claim 1, characterized in that, The smooth transition is achieved by applying blur to the plurality of wide-angle images WI and / or the plurality of folded telephoto images FTI.

10. The zoom camera according to claim 1, characterized in that, The smooth transition is achieved by blending the plurality of wide-angle images WI and the plurality of folded telephoto images FTI.

11. The zoom camera according to claim 1, characterized in that, The switching from displaying the plurality of wide-angle images WI to displaying the plurality of scanned telephoto images STI is performed by shifting a ZF value ZF upward 向上 wherein the switching from displaying the plurality of scanned telephoto images STI to displaying the plurality of wide-angle images WI is performed by shifting a ZF value ZF downward 向下 wherein ZF 向上 ≥ZF 向下 .

12. The zoom camera according to claim 11, characterized in that, The zoom camera is further configured to perform electronic image stabilization (EIS), wherein the ZF in the video image stream stabilized by the EIS is 向上 and / or the ZF 向下 has a value that is 2.5% to 50% greater than the ZF 向上 or the ZF 向下 in the video image stream stabilized without EIS.

13. The zoom camera according to claim 1, characterized in that, The foldable telephoto camera FTC is operable to hold a selected scene in a specific position within the native telephoto field of view n-FOV T and wherein actuation of the optical image stabilization OIS is performed at a high frequency of 50 Hz or higher and actuation for holding the selected scene in the specific position within the native telephoto field of view n-FOV T is performed at a low frequency of 20 Hz or lower.

14. The zoom camera according to claim 1, characterized in that, To provide a smooth transition, the zoom camera performs one or more computational steps, and wherein hardware acceleration is used in at least one of the one or more computational steps.

15. The zoom camera according to claim 1, characterized in that, The effective focal length of the scanning telephoto camera STC is from 8 mm to 50 mm.

16. The zoom camera according to claim 1, characterized in that, The effective focal length of the wide-angle camera is from 2.5 mm to 20 mm.

17. The zoom camera according to claim 1, characterized in that, The off-center region within the wide-angle field of view FOVW is selected by the user.

18. The zoom camera according to claim 1, characterized in that, The off-center region within the wide-angle field of view FOVW is selected by a program.

19. The zoom camera according to any one of claims 1 to 18, characterized in that, The zoom camera is included in a mobile device.

20. The zoom camera according to claim 19, characterized in that, The mobile device is a smartphone.

Images