Multi-camera zoom method, device and storage medium

Abstract

The present disclosure relates to a multi-camera zoom method, device, and storage medium. The method includes: determining a first zoom ratio based on a zoom ratio change operation for a first camera; determining a first mapping matrix based on the first zoom ratio; wherein the first mapping matrix represents: a mapping relationship between a first preview image corresponding to an initial zoom ratio of the first camera and a second preview image corresponding to the first zoom ratio; the first preview image and the second preview image are images of different zoom ratios for the same target object; based on the first mapping matrix, transforming the first preview image into the second preview image; and outputting the second preview image.

Description

Technical Field

[0001] The present disclosure relates to the field of camera technology, and in particular to a multi-camera zoom method, device, and storage medium. Background Art

[0002] Because a single camera has a limited shooting distance and range, mobile terminals are now equipped with multiple cameras to meet the demands of multiple viewpoints, multiple scenes, and multiple shooting distances, enhancing their shooting capabilities. These cameras can be used for zoom shooting to meet the demands of various application scenarios. However, due to the different positions of multiple cameras, when the camera used to display the preview image on a mobile terminal changes, the preview image inevitably experiences jitter, making it difficult to achieve smooth zooming. Summary of the Invention

[0003] The present disclosure provides a multi-camera zoom method, device, and storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a multi-camera zoom method is provided, including:

[0005] Determining a first zoom ratio according to a zoom ratio change operation on the first camera;

[0006] Determining a first mapping matrix based on the first zoom magnification; wherein the first mapping matrix represents: a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification; the first preview image and the second preview image are images of the same target object at different zoom magnifications;

[0007] transforming the first preview image into the second preview image based on the first mapping matrix;

[0008] The second preview image is output.

[0009] Optionally, determining a first mapping matrix according to the first zoom ratio includes:

[0010] determining, based on the image coordinate system of the first camera and the image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera; wherein the second mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a third preview image corresponding to an initial zoom magnification of the second camera; the first preview image and the third preview image being images of the same target object at different zoom magnifications;

[0011] According to the first zoom magnification, the second mapping matrix is ​​interpolated to obtain the first mapping matrix corresponding to the first zoom magnification.

[0012] Optionally, the method comprises:

[0013] determining a target mode of the second mapping matrix according to the attribute information of the target object;

[0014] The determining, according to the image coordinate system of the first camera and the image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera includes:

[0015] The second mapping matrix between the first camera and the second camera is determined in the target manner according to the image coordinate system of the first camera and the image coordinate system of the second camera.

[0016] Optionally, determining a target mode of the second mapping matrix according to the attribute information of the target object includes:

[0017] When the attribute information indicates that the target object does not include a stereoscopic object, determining the target mode to be a first mode; in the first mode, determining the second mapping matrix according to the image coordinate system of the first camera and the image coordinate system of the second camera;

[0018] When the attribute information indicates that the target object includes a stereoscopic object, the target mode is determined to be the second mode; under the second mode, the second mapping matrix is ​​determined based on the image coordinate system of the first camera, the image coordinate system of the second camera, and combined with the calibration parameters and / or camera coordinate systems of the first camera and the second camera.

[0019] Optionally, determining the second mapping matrix between the first camera and the second camera according to the image coordinate system of the first camera and the image coordinate system of the second camera includes:

[0020] If the target mode is the second mode, determining a rotation matrix and a translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera based on the extrinsic parameters of the first camera and the extrinsic parameters of the second camera;

[0021] Determining a first transformation relationship between a camera coordinate system of the first camera and a camera coordinate system of the second camera based on the rotation matrix and the translation matrix;

[0022] Determining a second transformation relationship between an image coordinate system of the first camera and an image coordinate system of the second camera based on the first transformation relationship and internal parameters of the first camera and the second camera;

[0023] Based on the second transformation relationship, a second mapping matrix that satisfies alignment of feature points in the first preview image and the third preview image is determined.

[0024] Optionally, performing interpolation processing on the second mapping matrix according to the first zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification includes:

[0025] Obtaining a first mapping matrix corresponding to an initial zoom magnification of the first camera;

[0026] Based on the first zoom magnification, linear interpolation processing is performed on the first mapping matrix and the second mapping matrix corresponding to the initial zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification.

[0027] Optionally, the method further includes:

[0028] determining whether the first zoom ratio indicated by the zoom ratio changing operation on the first camera satisfies a switching condition for the second camera;

[0029] The determining the first zoom ratio according to the zoom ratio change operation for the first camera includes:

[0030] If the switching condition of the second camera is not met, the first zoom ratio is determined.

[0031] Optionally, a switching condition for the second camera is: the first zoom ratio is within a zoom ratio range of the second camera.

[0032] Optionally, the method comprises:

[0033] If the first zoom ratio indicated by the zoom ratio changing operation on the first camera meets the switching condition of the second camera, the second camera is switched to be used for shooting.

[0034] According to a second aspect of an embodiment of the present disclosure, a multi-camera zoom device is provided, comprising:

[0035] A first determining module, configured to determine a first zoom ratio according to a zoom ratio changing operation on the first camera;

[0036] a second determining module configured to determine a first mapping matrix based on the first zoom magnification; wherein the first mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification; the first preview image and the second preview image being images of the same target object at different zoom magnifications;

[0037] The transformation module is configured to transform the first preview image into the second preview image based on the first mapping matrix, and output the second preview image.

[0038] Optionally, the second determining module is configured to:

[0039] determining, based on an image coordinate system of the first camera and an image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera; wherein the second mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a third preview image corresponding to an initial zoom magnification of the second camera; the first preview image and the third preview image being images of the same target object at different zoom magnifications;

[0040] According to the first zoom magnification, the second mapping matrix is ​​interpolated to obtain the first mapping matrix corresponding to the first zoom magnification.

[0041] Optionally, the second determining module is configured to:

[0042] determining a target mode of the second mapping matrix according to the attribute information of the target object;

[0043] The second mapping matrix between the first camera and the second camera is determined in the target manner according to the image coordinate system of the first camera and the image coordinate system of the second camera.

[0044] Optionally, the first determining module is configured to:

[0045] When the attribute information indicates that the target object does not include a stereoscopic object, determining the target mode to be a first mode; in the first mode, determining the second mapping matrix according to the image coordinate system of the first camera and the image coordinate system of the second camera;

[0046] When the attribute information indicates that the target object includes a stereoscopic object, the target mode is determined to be the second mode; under the second mode, the second mapping matrix is ​​determined based on the image coordinate system of the first camera, the image coordinate system of the second camera, and combined with the calibration parameters and / or camera coordinate systems of the first camera and the second camera.

[0047] Optionally, the second determining module is configured to:

[0048] If the target mode is the second mode, determining a rotation matrix and a translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera based on the extrinsic parameters of the first camera and the extrinsic parameters of the second camera;

[0049] Determining a first transformation relationship between a camera coordinate system of the first camera and a camera coordinate system of the second camera based on the rotation matrix and the translation matrix;

[0050] Determining a second transformation relationship between an image coordinate system of the first camera and an image coordinate system of the second camera based on the first transformation relationship and internal parameters of the first camera and the second camera;

[0051] Based on the second transformation relationship, a second mapping matrix that satisfies alignment of feature points in the first preview image and the third preview image is determined.

[0052] Optionally, the second determining module is configured to:

[0053] Obtaining a first mapping matrix corresponding to an initial zoom magnification of the first camera;

[0054] Based on the first zoom magnification, linear interpolation processing is performed on the first mapping matrix and the second mapping matrix corresponding to the initial zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification.

[0055] Optionally, the first determining module is further configured to:

[0056] determining whether the first zoom ratio indicated by the zoom ratio changing operation on the first camera satisfies a switching condition for the second camera;

[0057] If the switching condition of the second camera is not met, the first zoom ratio is determined.

[0058] Optionally, a switching condition for the second camera is: the first zoom ratio is within a zoom ratio range of the second camera.

[0059] Optionally, the first determining module is configured to:

[0060] If the first zoom ratio indicated by the zoom ratio changing operation on the first camera meets the switching condition of the second camera, the second camera is switched to be used for shooting.

[0061] According to a third aspect of the embodiments of the present disclosure, a multi-camera zoom device is provided, comprising:

[0062] processor;

[0063] a memory for storing executable instructions;

[0064] The processor is configured to: when executing the executable instructions stored in the memory, implement the steps in the multi-camera zoom method described in the first aspect of the embodiment of the present disclosure.

[0065] According to the fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of a multi-camera zoom device, the multi-camera zoom device is enabled to perform the steps in the multi-camera zoom method described in the first aspect of an embodiment of the present disclosure.

[0066] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0067] The disclosed embodiment determines a first zoom magnification indicated by a zoom magnification change operation for a first camera, and determines, based on the first zoom magnification, a first mapping matrix that reflects a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification. Based on the first mapping matrix, the first preview image is converted into a second preview image corresponding to the first zoom magnification. During a zooming process, as the first zoom magnification and the first mapping matrix corresponding to the first zoom magnification continuously change, continuous and natural zooming of the first preview image is achieved, thereby achieving a smooth zooming effect, reducing preview screen jitter, and improving user experience.

[0068] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0070] Figure 1 The figure is a flowchart of a camera switching method according to an exemplary embodiment.

[0071] Figure 2 A schematic diagram of a multi-camera zoom method according to an exemplary embodiment Figure 1 .

[0072] Figure 3A schematic diagram of a multi-camera zoom method according to an exemplary embodiment Figure 2 .

[0073] Figure 4 The figure is a schematic diagram showing zoom ratio ranges of multiple cameras according to an exemplary embodiment.

[0074] Figure 5 A schematic diagram of a multi-camera zoom method according to an exemplary embodiment Figure 3 .

[0075] Figure 6 The figure is a schematic structural diagram of a multi-camera zoom device according to an exemplary embodiment.

[0076] Figure 7 The figure is a block diagram of a multi-camera zoom device according to an exemplary embodiment. DETAILED DESCRIPTION

[0077] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0078] With the advancement of terminal imaging technology, more and more terminals are equipped with multiple cameras, forming multi-camera systems to enhance terminal imaging capabilities. The different cameras in a multi-camera system typically operate at different zoom magnification ranges. To achieve continuous zoom, it is necessary to switch cameras within certain zoom magnification ranges. Due to the differences in the position of different cameras, when the camera used to display the preview image on the terminal changes, the preview image will inevitably jitter, making it difficult to achieve smooth switching during continuous zoom.

[0079] In the related technology, a virtual camera is constructed, and based on the mapping relationship between the current camera and the virtual camera, the image captured by the current camera is converted into the image of the virtual camera corresponding to each zoom ratio, so as to achieve smooth switching of the camera switching process.

[0080] Specifically, if Figure 1 As shown, Figure 1 FIG1 is a flow chart illustrating a camera switching method according to an exemplary embodiment. The method includes:

[0081] Step S101: If the current camera zoom reaches a preset switching magnification, a start instruction is sent to the target camera;

[0082] Step S102: Control the current camera to zoom within the zoom range of the target camera. During the zooming process, the image captured by the current camera is converted into an image captured by a virtual camera corresponding to each zoom factor. The virtual camera is constructed based on the parameters of the current camera, the parameters of the target camera, and the zoom factors.

[0083] Step S103: If the image formed by the virtual camera is aligned with the image of the target camera, the current camera is switched to the target camera.

[0084] In the above method, when the current camera reaches the preset switching magnification, the camera screen is not switched immediately. Instead, the target camera is notified to start up first, and during the startup of the target camera, the current camera is controlled to continue zooming; and during the zooming process, a virtual camera determined according to the parameters of the current camera and the target camera is used to map the image captured by the current camera. Finally, when the image formed by the virtual camera is aligned with the image screen of the target camera, the current camera screen is switched to the target camera screen.

[0085] During the entire process, a virtual camera is set during the zoom process, and the image of the real camera is transformed into the virtual camera. The images between different cameras are transitioned through the virtual camera, so that the target camera can relay the zoom of the current camera continuously and naturally, and gradually converge from one camera to another during the relay zoom process, achieving the purpose of smooth transition switching of cameras.

[0086] However, this method uses the intrinsic and extrinsic parameters of the virtual camera to achieve image-to-image plane conversion, without considering the depth information that the image may contain; when the shooting scene has rich parallax levels, it is difficult to achieve a smooth switching effect.

[0087] Based on this, an embodiment of the present disclosure provides a multi-camera zoom method. Figure 2 A schematic diagram of a multi-camera zoom method according to an exemplary embodiment Figure 1 ,like Figure 2 As shown, the method includes:

[0088] Step S201, determining a first zoom ratio according to a zoom ratio change operation on a first camera;

[0089] Step S202: determining a first mapping matrix based on the first zoom magnification; wherein the first mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification; the first preview image and the second preview image being images of the same target object at different zoom magnifications;

[0090] Step S203: transforming the first preview image into the second preview image based on the first mapping matrix;

[0091] Step S204: output the second preview image.

[0092] In an embodiment of the present disclosure, the multi-camera zoom method can be performed by a multi-camera zoom device, which can be configured in a terminal device, and the terminal device has at least two cameras; here, the terminal device may include: a smartphone, a tablet computer or a laptop computer, etc.

[0093] In step S201, a zoom ratio change operation for a first camera is received, and a first zoom ratio indicated by the zoom ratio change operation is determined; the first zoom ratio may be a zoom ratio of a currently displayed image relative to a displayed image when the first camera is just started.

[0094] Here, the first camera is any camera currently in use in the terminal device, and the zoom ratio change operation refers to a zoom operation performed by the user on the preview image currently displayed on the terminal device (ie, the preview image captured by the first camera).

[0095] It should be noted that the zoom ratio change operation can be a sliding operation of the user on the interface of the terminal device displaying the preview image during the shooting process, or it can be a stretching operation on the focus control or a pressing operation on the focus button; for example, a focus case can be set in the terminal device, and when the focus button is pressed, different zoom ratios and magnification areas can be determined. This disclosure does not make specific limitations.

[0096] In step S202, a first mapping matrix for converting a first preview image corresponding to an initial zoom magnification of a first camera into a second preview image corresponding to the first zoom magnification is determined according to the first zoom magnification.

[0097] It can be understood that the first camera starts image acquisition by turning on the preview, thereby capturing a temporary image, namely the first preview image. Usually, the preview image is an image stored only in the cache of the terminal device. After obtaining the first zoom ratio, the first preview image needs to be processed to obtain a second preview image. The second preview image is displayed on the terminal device at the first zoom ratio. Different first zoom ratios correspond to different zoom ratios. For example, if the first zoom ratio is 2X, the first preview image needs to be displayed at a magnification of 2x; therefore, it is necessary to determine a mapping matrix between the first preview image and the preview image after being magnified by 2 times, so that the first preview image can be processed based on the mapping matrix.

[0098] In step S203, after the first mapping matrix is ​​determined, mapping processing is performed on the first preview image to obtain a second preview image corresponding to the first zoom magnification.

[0099] The first mapping matrix is ​​used to participate in the mapping process of the first preview image, that is, the first preview image corresponding to the initial zoom magnification of the first camera is mapped to the second preview image corresponding to the first zoom magnification.

[0100] In some embodiments, each pixel in the first preview image may be processed using a first mapping matrix to obtain corresponding mapped pixel points, thereby obtaining a second preview image corresponding to the first zoom factor.

[0101] In step S204, after obtaining the second preview image corresponding to the first zoom ratio, the second preview image is output through the display screen of the terminal device.

[0102] The disclosed embodiment determines a first zoom magnification indicated by a zoom magnification change operation for a first camera, and determines, based on the first zoom magnification, a first mapping matrix that reflects a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification. Based on the first mapping matrix, the first preview image is converted into a second preview image corresponding to the first zoom magnification. During a zooming process, as the first zoom magnification and the first mapping matrix corresponding to the first zoom magnification continuously change, continuous and natural zoom processing of the first preview image is achieved, thereby achieving a smooth zooming effect and improving user experience.

[0103] Optionally, determining a first mapping matrix according to the first zoom ratio in step S202 includes:

[0104] determining, based on an image coordinate system of the first camera and an image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera; wherein the second mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a third preview image corresponding to an initial zoom magnification of the second camera; the first preview image and the third preview image being images of the same target object at different zoom magnifications;

[0105] According to the first zoom magnification, the second mapping matrix is ​​interpolated to obtain the first mapping matrix corresponding to the first zoom magnification.

[0106] In the embodiment of the present disclosure, since the first preview image and the third preview image are images of the same target object at different zoom magnifications, a mapping relationship between the first preview image corresponding to the initial zoom magnification of the first camera and the third preview image corresponding to the initial zoom magnification of the second camera, i.e., a second mapping matrix between the first camera and the second camera, can be determined by obtaining the first coordinates of the target object in the image coordinate system of the first preview image and the second coordinates of the target object in the image coordinate system of the third preview image; and based on the first coordinates and the second coordinates of the target object.

[0107] It should be noted that the terminal device may involve switching cameras when taking photos or shooting videos. The first camera is any camera currently in use in the terminal device, and the second camera is any camera in the terminal device that is to take over from the first camera.

[0108] A camera's image coordinate system refers to the coordinate system of images captured by the camera. This image coordinate system is used to describe the projection and transmission relationship between the target object captured by the camera and the image coordinate system during imaging. Different images may have different image coordinate systems, and the coordinates of the target object in different image coordinate systems may also differ.

[0109] The first zoom ratio is between the initial zoom ratio of the first camera and the initial zoom ratio of the second camera. To achieve a smooth zoom effect, the second mapping matrix between the first camera and the second camera can be interpolated based on the first zoom ratio to determine the first mapping matrix corresponding to the first zoom ratio. Thus, throughout the zoom process, the first preview image of the first camera is converted into the second preview image corresponding to the first zoom ratio according to the first mapping matrix corresponding to the first zoom ratio. As the first zoom ratio continues to change, the second preview image is gradually aligned with the third preview image corresponding to the second camera using the second preview image, thereby reducing preview image jitter.

[0110] Optionally, the method comprises:

[0111] determining a target mode of the second mapping matrix according to the attribute information of the target object;

[0112] The determining, according to the image coordinate system of the first camera and the image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera includes:

[0113] The second mapping matrix between the first camera and the second camera is determined in the target manner according to the image coordinate system of the first camera and the image coordinate system of the second camera.

[0114] In the embodiment of the present disclosure, the attribute information of the target object can be obtained, and the target mode can be determined based on the attribute information, so that different processing methods can be used to obtain the second mapping matrix between the first camera and the second camera for target objects with different attributes.

[0115] The attribute information may include: contour information and depth information; based on the contour information and depth information of the target object, determining the spatial structure information of the target object, and determining the second mapping matrix between the first camera and the second camera through a pre-set processing method corresponding to the spatial structure information of the target object.

[0116] It can be understood that since the contour information of the target object can be used to describe the shape of the target object; the depth information can be used to describe the distance between each feature point in the target object and the camera, the two-dimensional spatial structure information and / or three-dimensional spatial structure information of the target object can be determined based on the contour information and depth information of the target object.

[0117] Optionally, determining a target mode of the second mapping matrix according to the attribute information of the target object includes:

[0118] When the attribute information indicates that the target object does not include a stereoscopic object, determining the target mode to be a first mode; in the first mode, determining the second mapping matrix according to the image coordinate system of the first camera and the image coordinate system of the second camera;

[0119] When the attribute information indicates that the target object includes a stereoscopic object, the target mode is determined to be the second mode; under the second mode, the second mapping matrix is ​​determined based on the image coordinate system of the first camera, the image coordinate system of the second camera, and combined with the calibration parameters and / or camera coordinate systems of the first camera and the second camera.

[0120] In an embodiment of the present disclosure, if the attribute information indicates that the target object does not include a three-dimensional object, that is, the target object is a planar object, a second mapping matrix between the first camera and the second camera will be determined in a preset first manner.

[0121] Since the target object is a planar object, the first preview image corresponding to the first camera does not contain depth information difference, so the first preview image of the first camera can be directly scaled to obtain an image whose feature points are completely aligned with the third preview image corresponding to the second camera.

[0122] It should be noted that the depth information can be the distance between each point in the plane where the target object is located and the camera. In the camera coordinate system, the line perpendicular to the imaging plane and passing through the center of the mirror is the Z axis. If the coordinates of the target object in the camera coordinate system are (X, Y, Z), then the Z value is the depth information between the target object and the camera.

[0123] Since the target object is a planar object, the distance between each point in the plane where the target object is located and the camera is the same, that is, the Z value is the same; according to the mapping relationship between the image coordinate system of the first camera and the image coordinate system of the second camera, not only the zoom ratio relationship between the first camera and the second camera can be reflected, but also the relative position information between the first camera and the second camera can be reflected.

[0124] If the attribute information indicates that the target object includes a three-dimensional object, a second mapping matrix between the first camera and the second camera is determined in a preset second manner.

[0125] Since the target object is a three-dimensional object and due to the difference in posture between the first camera and the second camera, the target object in the first preview image of the first camera is not only scaled relative to the target object in the third preview image of the second camera, but may also be rotated, translated, and the like.

[0126] To more accurately describe the mapping relationship between the first preview image and the third preview image, when determining the mapping relationship between the image coordinate system of the first camera and the image coordinate system of the second camera, it is necessary to consider the inherent characteristics of the first camera and the second camera, as well as the positions of the first camera and the second camera relative to the target object.

[0127] Therefore, when it is determined that the target object is a three-dimensional object, it is necessary to determine the second mapping matrix based on the image coordinate system of the first camera and the image coordinate system of the second camera, and in combination with the calibration parameters and / or camera coordinate systems of the first camera and the second camera.

[0128] The disclosed embodiment determines a target method for obtaining a second mapping matrix through attribute information of a target object. When the target object is a planar object, the second mapping matrix is ​​determined directly based on the mapping relationship between the image coordinate system of the first camera and the image coordinate system of the second camera, thereby reducing the amount of data processing and improving processing efficiency. When the target object is a three-dimensional object, the second mapping matrix is ​​determined based on the image coordinate system of the first camera and the image coordinate system of the second camera, in combination with the calibration parameters and / or camera coordinate system of the first camera and the second camera, fully considering the influence of the posture difference between the first camera and the second camera on the second mapping matrix, so as to ensure that an accurate second mapping matrix is ​​obtained.

[0129] Optionally, determining the second mapping matrix between the first camera and the second camera according to the image coordinate system of the first camera and the image coordinate system of the second camera includes:

[0130] If the target mode is the second mode, determining a rotation matrix and a translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera based on the extrinsic parameters of the first camera and the extrinsic parameters of the second camera;

[0131] Determining a first transformation relationship between a camera coordinate system of the first camera and a camera coordinate system of the second camera based on the rotation matrix and the translation matrix;

[0132] Determining a second transformation relationship between an image coordinate system of the first camera and an image coordinate system of the second camera based on the first transformation relationship and internal parameters of the first camera and the second camera;

[0133] Based on the second transformation relationship, a second mapping matrix is ​​determined that satisfies the alignment of feature points in the first preview image and the third preview image.

[0134] In an embodiment of the present disclosure, the external parameters of the camera may include the rotation matrix and translation matrix of the camera. Here, the rotation matrix describes the direction of the world coordinate system relative to the camera coordinate system of the camera, and the translation matrix describes the position of the space origin in the camera coordinate system of the camera.

[0135] Since the world coordinate system is fixed, the rotation matrix and translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera can be determined by using the extrinsic parameters of the first camera and the extrinsic parameters of the second camera;

[0136] The rotation matrix here describes the direction of the camera coordinate system of the first camera relative to the camera coordinate system of the second camera; the translation matrix describes the position of the origin of the camera coordinate system of the first camera in the camera coordinate system of the second camera.

[0137] It should be noted that the world coordinate system is a user-defined coordinate system for the three-dimensional world. It is introduced to describe the position of the target object in the real world and is an absolute coordinate system in the objective three-dimensional world. The camera coordinate system is a coordinate system established based on the camera. It is introduced to describe the position of the target object relative to the camera from the camera's perspective.

[0138] After determining the rotation matrix and translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera, the first transformation relationship between the coordinates of the target object in the camera coordinate system of the first camera and the coordinates of the target object in the camera coordinate system of the second camera can be determined based on the rotation matrix and the translation matrix, thereby reducing the impact caused by the difference in posture between the first camera and the second camera during the zoom process.

[0139] The camera's internal parameters may include an internal parameter matrix, i.e., focal length, center principal point coordinates, etc., which reflect the camera's inherent characteristics. Based on the camera's internal parameters, the transformation relationship between the camera's camera coordinate system and the camera's image coordinate system can be determined.

[0140] Therefore, by obtaining the internal parameters of the first camera and the internal parameters of the second camera, according to the first transformation relationship between the camera coordinate system of the first camera and the camera coordinate system of the second camera, as well as the internal parameters of the first camera and the second camera, the second transformation relationship between the image coordinate system of the first camera and the image coordinate system of the second camera can be determined.

[0141] The second mapping matrix is ​​determined based on a second transformation relationship between the image coordinate system of the first camera and the image coordinate system of the second camera; and feature points of the target object in the first preview image processed by the second mapping matrix are aligned with feature points of the target object in the third preview image.

[0142] Here, aligning the feature points of the target object in the first preview image with the feature points of the target object in the third preview image may mean that the vertical coordinates of the feature points of the target object in the first preview image are the same as the vertical coordinates of the feature points of the target object in the third preview image.

[0143] It should be noted that since the positions of the first camera and the second camera are different on the terminal device, even if the first preview image captured by the first camera is mapped to a preview image corresponding to the initial zoom magnification of the second camera through the second mapping matrix, there may still be parallax between the preview image and the third preview image actually captured by the second camera. Therefore, the vertical coordinates of the feature points of the target object in the first preview image are the same as the vertical coordinates of the feature points of the target object in the third preview image, and there is a certain difference between the horizontal coordinates of the feature points of the target object in the first preview image and the horizontal coordinates of the feature points of the target object in the third preview image. That is, when the captured target object is a three-dimensional object, when the first camera is switched to the second camera, there is only a parallax difference between the first preview image of the first camera and the third preview image of the second camera, and the zoom is smooth.

[0144] Optionally, performing interpolation processing on the second mapping matrix according to the first zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification includes:

[0145] Obtaining a first mapping matrix corresponding to an initial zoom magnification of the first camera;

[0146] Based on the first zoom magnification, linear interpolation processing is performed on the first mapping matrix and the second mapping matrix corresponding to the initial zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification.

[0147] In the embodiment of the present disclosure, since the first zoom magnification is any zoom magnification between the initial zoom magnification of the first camera and the initial zoom magnification of the second camera, in order to accurately estimate the first mapping matrix corresponding to the first zoom magnification, the first mapping matrix corresponding to the initial zoom magnification of the first camera and the second mapping matrix corresponding to the initial zoom magnification of the second camera can be obtained; based on the first mapping matrix corresponding to the initial zoom magnification of the first camera and the second mapping matrix corresponding to the initial zoom magnification of the second camera, the association between the zoom magnification and the mapping matrix is ​​determined, and then the first mapping matrix corresponding to the first zoom magnification is estimated based on the association between the zoom magnification and the mapping matrix and the first zoom magnification.

[0148] Optionally, the method comprises:

[0149] determining whether the first zoom ratio indicated by the zoom ratio changing operation on the first camera satisfies a switching condition for the second camera;

[0150] The step S201 of determining the first zoom ratio according to the zoom ratio change operation for the first camera includes:

[0151] If the switching condition of the second camera is not met, the first zoom ratio is determined.

[0152] In an embodiment of the present disclosure, based on a received zoom ratio change operation for a first camera, it is determined whether a first zoom ratio corresponding to the zoom ratio change operation satisfies a switching condition for a second camera; if the first zoom ratio does not satisfy the switching condition for the second camera, the first camera is continued to be used for shooting, and a first preview image captured by the first camera is converted into a second preview image corresponding to the first zoom ratio.

[0153] Here, the switching condition of the second camera can be determined according to the zoom ratio range of the second camera. For example, the switching condition of the second camera can be that the first zoom ratio is greater than the minimum zoom ratio of the second camera, or that the first zoom ratio is less than the maximum zoom ratio of the second camera.

[0154] In some embodiments, a switching condition for the second camera is that the first zoom ratio is within a zoom ratio range of the second camera.

[0155] When the first zoom ratio does not meet the switching condition of the second camera, it means that the first zoom ratio is within the zoom range of the first camera, and the first camera can continue to be used for shooting.

[0156] It is understandable that different cameras correspond to different zoom ranges. The currently used camera can be determined according to the zoom range of the first zoom ratio to obtain a preview image matching the first zoom ratio.

[0157] Optionally, the method comprises:

[0158] If the first zoom ratio indicated by the zoom ratio changing operation on the first camera meets the switching condition of the second camera, the second camera is switched to be used for shooting.

[0159] In the embodiment of the present disclosure, if the first zoom ratio meets the switching condition of the second camera, it means that the first zoom ratio is within the zoom range of the second camera, and the second camera is switched to be used for shooting.

[0160] In some embodiments, if the first zoom ratio satisfies a switching condition for the second camera, the second camera is activated and switched to be used for shooting.

[0161] It should be noted that since a large current consumption will be caused when the camera is turned on, in order to reduce power, the terminal device only starts a camera when it is needed. When it is not needed, the camera is in the off state. Therefore, when the first zoom ratio meets the switching conditions of the second camera, the terminal device can send a start-up instruction to the second camera to start the second camera and switch to use the second camera for shooting.

[0162] The present disclosure also provides the following embodiments:

[0163] Figure 3 A schematic diagram of a multi-camera zoom method according to an exemplary embodiment Figure 2 , the method comprising:

[0164] Step S301: determining a second mapping matrix between the first camera and the second camera according to the image coordinate system of the first camera and the image coordinate system of the second camera;

[0165] In this example, the first camera is the current camera currently shooting, the second camera is the target camera to be switched, and the second mapping matrix between the first camera and the second camera can be the alignment mapping matrix between the current camera and the target camera.

[0166] It should be noted that the zoom magnification range of the current camera is adjacent to the zoom magnification range of the target camera; for example, Figure 4 As shown, Figure 4 FIG. 1 is a schematic diagram showing zoom ratio ranges of multiple cameras according to an exemplary embodiment, wherein the zoom ratio range of the current camera is 1X to mX, and the zoom ratio range of the target camera is mX to nX.

[0167] Here, the alignment mapping matrix H between the current camera and the target camera is a 3×3 matrix in the following form:

[0168]

[0169] The values ​​of the elements in the alignment mapping matrix H may be determined by the intrinsic parameter matrix of the current camera and the intrinsic parameter matrix of the target camera, as well as the rotation matrix between the current camera and the target camera;

[0170] It should be noted that since the target object in the preview image captured by the current camera and the target object in the preview image captured by the target camera may involve translation, rotation or scaling changes of the target object, in the process of determining the alignment mapping matrix, in order to facilitate calculation and reduce the amount of calculation, homogeneous coordinates can be introduced to change the image from planar two-dimensional coordinates to three-dimensional coordinates, so that the translation, rotation or scaling changes can be uniformly expressed in the form of matrix multiplication.

[0171] Based on this, h8=1 is determined based on the homogeneous coordinates of the target object, and other element values ​​are related to the intrinsic parameter matrix of the current camera and the intrinsic parameter matrix of the target camera, as well as the rotation matrix between the current camera and the target camera.

[0172] It is understood that the alignment mapping matrix represents a mapping relationship between a first preview image corresponding to the initial zoom magnification of the current camera and a third preview image corresponding to the initial zoom magnification of the target camera, wherein the first preview image and the third preview image are images of the same target object at different zoom magnifications.

[0173] In some embodiments, a target method for obtaining the second mapping matrix can be determined based on attribute information of the target object; and the second mapping matrix between the first camera and the second camera can be determined using the target method based on the image coordinate system of the first camera and the image coordinate system of the second camera.

[0174] For target objects with different attributes, different methods may be used to determine the alignment mapping matrix.

[0175] In some embodiments, when the attribute information indicates that the target object does not include a stereoscopic object, the target method is determined in a first manner. Under the first manner, the second mapping matrix is ​​determined solely based on the image coordinate system of the first camera and the image coordinate system of the second camera.

[0176] In this example, if the attribute information indicates that the target object does not include a three-dimensional object, that is, the target object is a planar object, since the target object is a planar object, the first preview image captured by the current camera and the third preview image captured by the target camera do not contain depth information. Therefore, the alignment mapping matrix can be determined directly based on the mapping relationship between the image coordinate system of the current camera and the image coordinate system of the target camera.

[0177] It can be understood that since the target object is a planar object, the mapping relationship between the image coordinate system of the current camera and the image coordinate system of the target camera, that is, the homography matrix between the current camera plane and the target camera plane; the homography matrix can be determined as the alignment mapping matrix between the current camera and the target camera.

[0178] It should be noted that the homography matrix describes the mapping relationship between points on the same plane in different images.

[0179] The first preview image captured by the current camera is mapped using the homography matrix, and the mapped first preview image is completely aligned with the third preview image captured by the target camera.

[0180] In other embodiments, when the attribute information indicates that the target object includes a stereoscopic object, the target mode is determined to be the second mode. Under the second mode, the second mapping matrix is ​​determined based on the image coordinate system of the first camera, the image coordinate system of the second camera, and in combination with the calibration parameters and / or camera coordinate systems of the first camera and the second camera.

[0181] In this example, if the attribute information indicates that the target object includes a three-dimensional object, that is, the parallax layers of the shooting scene are relatively rich, considering that there are differences in the postures of the current camera and the target camera, the position and posture of the target object in the first preview image and the target object in the third preview image may be different. Therefore, when determining the alignment mapping matrix between the current camera and the target camera, not only the mapping relationship between the image coordinate system of the current camera and the image coordinate system of the target camera should be considered, but also the mapping relationship between the camera coordinate system of the current camera and the camera coordinate system of the target camera should be considered.

[0182] It should be noted that camera calibration is the process of converting from the world coordinate system to the image coordinate system. The calibration process can be divided into two parts: the first part is to convert from the world coordinate system to the camera coordinate system; the second part is to convert from the camera coordinate system to the image coordinate system. Camera calibration parameters can include intrinsic and extrinsic parameters. Intrinsic parameters can be understood as parameters related to focal length. Intrinsic parameters describe the conversion relationship between the camera coordinate system and the image coordinate system. Intrinsic parameters can be used to eliminate distortion and make the resulting image more accurate. Extrinsic parameters describe the conversion relationship between the camera coordinate system and the world coordinate system.

[0183] In some embodiments, determining a second mapping matrix between the first camera and the second camera according to the image coordinate system of the first camera and the image coordinate system of the second camera includes:

[0184] If the target mode is the second mode, determining a rotation matrix and a translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera based on the extrinsic parameters of the first camera and the extrinsic parameters of the second camera;

[0185] Determining a first transformation relationship between a camera coordinate system of the first camera and a camera coordinate system of the second camera based on the rotation matrix and the translation matrix;

[0186] Determining a second transformation relationship between an image coordinate system of the first camera and an image coordinate system of the second camera based on the first transformation relationship and internal parameters of the first camera and the second camera;

[0187] Based on the second transformation relationship, a second mapping matrix is ​​determined that satisfies the alignment of feature points in the first preview image and the third preview image.

[0188] Here, by obtaining the extrinsic parameters of the current camera and the extrinsic parameters of the target camera; since the extrinsic parameters of the current camera are used to describe the mapping relationship between the camera coordinate system of the current camera and the world coordinate system, the extrinsic parameters of the target camera are used to describe the mapping relationship between the camera coordinate system of the target camera and the world coordinate system; according to the extrinsic parameters of the current camera and the extrinsic parameters of the target camera, the rotation matrix and translation matrix required for switching the camera coordinate system of the current camera to the camera coordinate system of the target camera can be determined.

[0189] The rotation matrix describes the direction of the coordinate axes of the camera coordinate system of the current camera relative to the coordinate axes of the camera coordinate system of the target camera; the translation matrix describes the position of the origin of the camera coordinate system of the current camera in the camera coordinate axes of the target camera.

[0190] After determining the first mapping relationship between the camera coordinate system of the current camera and the camera coordinate system of the target camera, the second mapping relationship between the image coordinate system of the current camera and the image coordinate system of the target camera can be determined based on the first mapping relationship.

[0191] Since the internal parameters describe the conversion relationship between the camera coordinate system and the image coordinate system, the second mapping relationship between the image coordinate system of the current camera and the image coordinate system of the target camera can be determined based on the internal parameters of the current camera and the target camera, and the first mapping relationship, thereby determining the second mapping matrix, i.e., the row alignment matrix, based on the second mapping relationship.

[0192] In some embodiments, the first transformation relationship between the camera coordinate system of the first camera and the camera coordinate system of the second camera may be:

[0193] P1=RP2+T;

[0194] Among them, P1 is the coordinate of the feature point of the target object in the camera coordinate system of the first camera; P2 is the coordinate of the feature point of the target object in the camera coordinate system of the second camera; R is the rotation matrix between the first camera and the second camera; T is the translation matrix between the first camera and the second camera.

[0195] Accordingly, the second mapping matrix between the first camera and the second camera may be:

[0196]

[0197] Wherein, H is the second mapping matrix, i.e., the row alignment matrix; K1 is the internal parameter of the first camera; is the inverse matrix corresponding to the intrinsic parameter of the second camera; and R is the rotation matrix between the first camera and the second camera.

[0198] Based on the row alignment matrix, the first preview image captured by the current camera is mapped, and the first preview image after the mapping process has a difference with the third preview image captured by the target camera only in parallax.

[0199] Step S302: Determining, based on the zoom magnification change operation on the first camera, whether a first zoom magnification indicated by the zoom magnification change operation on the first camera satisfies a switching condition for the second camera; if the first zoom magnification does not satisfy the switching condition, interpolating the second mapping matrix based on the first zoom magnification to obtain a first mapping matrix corresponding to the first zoom magnification;

[0200] Here, the switching condition of the second camera is: the first zoom ratio is within the zoom ratio range of the second camera.

[0201] In this example, the first zoom magnification may be the current zoom magnification of the current camera after the zoom magnification change operation; the first mapping matrix corresponding to the first zoom magnification is a transition alignment mapping matrix corresponding to the current zoom magnification; the transition alignment mapping matrix represents a mapping relationship between a first preview image corresponding to the initial zoom magnification of the first camera and a second preview image corresponding to the current zoom magnification.

[0202] According to the alignment mapping matrix between the current camera and the target camera and the current zoom magnification of the first camera, a transition alignment mapping matrix corresponding to the current zoom magnification is determined, so that the first preview image is converted into the second preview image using the transition alignment mapping matrix.

[0203] In some embodiments, a first mapping matrix corresponding to an initial zoom ratio of the first camera may be obtained; based on the first zoom ratio, linear interpolation may be performed on the first mapping matrix corresponding to the initial zoom ratio and the second mapping matrix to obtain a first mapping matrix corresponding to the first zoom ratio.

[0204] In this example, since the zoom magnification range of the current camera is 1X to mX, the zoom magnification range of the target camera is mX to nX.

[0205] If the current zoom ratio indicated by the zoom ratio change operation does not meet the switching condition of the target camera, that is, the current zoom ratio is not within the zoom ratio range of the target camera; that is, the current zoom ratio f∈[1,m].

[0206] The first mapping matrix H1 corresponding to the initial zoom ratio of the current camera is a unit matrix, that is:

[0207]

[0208] The alignment mapping matrix H between the current camera and the target camera:

[0209]

[0210] Based on the current zoom magnification, linear interpolation is performed on the first mapping matrix H1 corresponding to the initial zoom magnification and the alignment mapping matrix H to determine the transition alignment mapping matrix H corresponding to the current zoom magnification. f , as shown below:

[0211]

[0212] Wherein, the α is the current zoom ratio.

[0213] It can be understood that as the current zoom ratio α changes, the transition alignment mapping matrix H corresponding to the current zoom ratio changes. f The various parameters in also change evenly, so that the second preview image corresponding to the current zoom magnification is gradually aligned with the third preview image of the target camera by using the transition alignment mapping matrix.

[0214] Step S303: transforming the first preview image of the first camera into a second preview image corresponding to the first zoom factor based on the first mapping matrix; and outputting the second preview image;

[0215] In this example, based on the determined transition alignment mapping matrix corresponding to the current zoom ratio, the first preview image captured by the current camera is converted into a second preview image corresponding to the current zoom ratio, and the second preview image is output in response to the zoom ratio change operation.

[0216] Step S304: If the first zoom ratio indicated by the zoom ratio change operation on the first camera satisfies a switching condition for the second camera, the second camera is switched to be used for shooting.

[0217] In this example, when the current zoom ratio indicated by the zoom ratio changing operation reaches the zoom ratio range of the target camera, that is, when the current zoom ratio reaches mX, the target camera will be switched to be used for shooting.

[0218] For example, Figure 5 As shown, Figure 5 A schematic diagram of a multi-camera zoom method according to an exemplary embodiment Figure 3 The method comprises:

[0219] Step S401, calculating the alignment mapping matrix from the current camera to the target camera;

[0220] Step S402: Calculate a transition alignment mapping matrix at the current zoom ratio based on the alignment mapping matrix from the current camera to the target camera and the current zoom ratio, and gradually align the preview image of the current camera to the target camera using the transition alignment mapping matrix.

[0221] Step S403: When the zoom ratio reaches the ratio range of the target camera, the camera used to display the preview is switched to achieve smooth zoom of multiple cameras.

[0222] In this way, it can effectively solve the problem of poor smooth switching effect when the shooting scene has rich parallax levels, achieve smooth zoom effect of multiple cameras in complex application scenarios, smooth visual effect, and improve user experience.

[0223] The disclosed embodiment also provides a multi-camera zoom device. Figure 6 FIG. 1 is a structural diagram of a multi-camera zoom device according to an exemplary embodiment. Figure 6 As shown, the multi-camera zoom device 100 includes:

[0224] A first determining module 101 is configured to determine a first zoom ratio according to a zoom ratio change operation performed on a first camera;

[0225] A second determining module 102 is configured to determine a first mapping matrix based on the first zoom magnification; wherein the first mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification; the first preview image and the second preview image are images of the same target object at different zoom magnifications;

[0226] The transformation module 103 is configured to transform the first preview image into the second preview image based on the first mapping matrix, and output the second preview image.

[0227] Optionally, the second determining module 102 is configured to:

[0228] determining, based on an image coordinate system of the first camera and an image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera; wherein the second mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a third preview image corresponding to an initial zoom magnification of the second camera; the first preview image and the third preview image being images of the same target object at different zoom magnifications;

[0229] According to the first zoom magnification, the second mapping matrix is ​​interpolated to obtain the first mapping matrix corresponding to the first zoom magnification.

[0230] Optionally, the second determining module 102 is configured to:

[0231] determining a target mode of the second mapping matrix according to the attribute information of the target object;

[0232] The second mapping matrix between the first camera and the second camera is determined in the target manner according to the image coordinate system of the first camera and the image coordinate system of the second camera.

[0233] Optionally, the second determining module 102 is configured to:

[0234] When the attribute information indicates that the target object does not include a stereoscopic object, determining the target mode to be a first mode; in the first mode, determining the second mapping matrix according to the image coordinate system of the first camera and the image coordinate system of the second camera;

[0235] When the attribute information indicates that the target object includes a stereoscopic object, the target mode is determined to be the second mode; under the second mode, the second mapping matrix is ​​determined based on the image coordinate system of the first camera, the image coordinate system of the second camera, and combined with the calibration parameters and / or camera coordinate systems of the first camera and the second camera.

[0236] Optionally, the second determining module 102 is configured to:

[0237] If the target mode is the second mode, determining a rotation matrix and a translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera based on the extrinsic parameters of the first camera and the extrinsic parameters of the second camera;

[0238] Determining a first transformation relationship between a camera coordinate system of the first camera and a camera coordinate system of the second camera based on the rotation matrix and the translation matrix;

[0239] Determining a second transformation relationship between an image coordinate system of the first camera and an image coordinate system of the second camera based on the first transformation relationship and internal parameters of the first camera and the second camera;

[0240] Based on the second transformation relationship, a second mapping matrix that satisfies alignment of feature points in the first preview image and the third preview image is determined.

[0241] Optionally, the second determining module 102 is configured to:

[0242] Obtaining a first mapping matrix corresponding to an initial zoom magnification of the first camera;

[0243] Based on the first zoom magnification, linear interpolation processing is performed on the first mapping matrix and the second mapping matrix corresponding to the initial zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification.

[0244] Optionally, the first determining module 101 is configured to:

[0245] determining whether the first zoom ratio indicated by the zoom ratio changing operation on the first camera satisfies a switching condition for the second camera;

[0246] If the switching condition of the second camera is not met, the first zoom ratio is determined.

[0247] Optionally, a switching condition for the second camera is: the first zoom ratio is within a zoom ratio range of the second camera.

[0248] Optionally, the first determining module 101 is configured to:

[0249] If the first zoom ratio indicated by the zoom ratio changing operation on the first camera meets the switching condition of the second camera, the second camera is switched to be used for shooting.

[0250] Figure 7 8 is a block diagram of a multi-camera zoom device according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a mobile computer, etc.

[0251] Reference Figure 7 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0252] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0253] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0254] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0255] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0256] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0257] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0258] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor assembly 814 can also detect changes in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0259] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0260] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0261] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the apparatus 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0262] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0263] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A multi-camera zoom method, characterized in that: include: Determining a first zoom ratio according to a zoom ratio change operation on the first camera; Determining a first mapping matrix based on the first zoom magnification; wherein the first mapping matrix represents: a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification; the first preview image and the second preview image are images of the same target object at different zoom magnifications; transforming the first preview image into the second preview image based on the first mapping matrix; outputting the second preview image; The determining of a first mapping matrix according to the first zoom ratio includes: determining, based on the image coordinate system of the first camera and the image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera; wherein the second mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a third preview image corresponding to an initial zoom magnification of the second camera; the first preview image and the third preview image being images of the same target object at different zoom magnifications; According to the first zoom magnification, the second mapping matrix is ​​interpolated to obtain the first mapping matrix corresponding to the first zoom magnification.

2. The method according to claim 1, characterized in that The method comprises: determining a target mode of the second mapping matrix according to the attribute information of the target object; The determining, according to the image coordinate system of the first camera and the image coordinate system of the second camera, a second mapping matrix between the first camera and the second camera includes: The second mapping matrix between the first camera and the second camera is determined in the target manner according to the image coordinate system of the first camera and the image coordinate system of the second camera.

3. The method according to claim 2, characterized in that The determining, according to the attribute information of the target object, a target mode of the second mapping matrix includes: When the attribute information indicates that the target object does not include a stereoscopic object, determining the target mode to be a first mode; in the first mode, determining the second mapping matrix according to the image coordinate system of the first camera and the image coordinate system of the second camera; When the attribute information indicates that the target object includes a stereoscopic object, the target mode is determined to be the second mode; under the second mode, the second mapping matrix is ​​determined based on the image coordinate system of the first camera, the image coordinate system of the second camera, and combined with the calibration parameters and / or camera coordinate systems of the first camera and the second camera.

4. The method according to claim 3, characterized in that The determining, according to the image coordinate system of the first camera and the image coordinate system of the second camera, the second mapping matrix between the first camera and the second camera includes: If the target mode is the second mode, determining a rotation matrix and a translation matrix required to switch the camera coordinate system of the first camera to the camera coordinate system of the second camera based on the extrinsic parameters of the first camera and the extrinsic parameters of the second camera; Determining a first transformation relationship between a camera coordinate system of the first camera and a camera coordinate system of the second camera based on the rotation matrix and the translation matrix; Determining a second transformation relationship between an image coordinate system of the first camera and an image coordinate system of the second camera based on the first transformation relationship and internal parameters of the first camera and the second camera; Based on the second transformation relationship, a second mapping matrix that satisfies alignment of feature points in the first preview image and the third preview image is determined.

5. The method according to claim 1, wherein The interpolation processing is performed on the second mapping matrix according to the first zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification, including: Obtaining a first mapping matrix corresponding to an initial zoom magnification of the first camera; Based on the first zoom magnification, linear interpolation processing is performed on the first mapping matrix and the second mapping matrix corresponding to the initial zoom magnification to obtain the first mapping matrix corresponding to the first zoom magnification.

6. The method according to claim 1, characterized in that The method comprises: determining whether the first zoom ratio indicated by the zoom ratio changing operation on the first camera satisfies a switching condition for the second camera; The determining the first zoom ratio according to the zoom ratio change operation for the first camera includes: If the switching condition of the second camera is not met, the first zoom ratio is determined.

7. The method according to claim 6, characterized in that A switching condition for the second camera is that the first zoom ratio is within a zoom ratio range of the second camera.

8. The method according to claim 6, characterized in that The method comprises: If the first zoom ratio indicated by the zoom ratio changing operation on the first camera meets the switching condition of the second camera, the second camera is switched to be used for shooting.

9. A multi-camera zoom device, characterized in that: The device comprises: A first determining module, configured to determine a first zoom ratio according to a zoom ratio changing operation on the first camera; a second determining module configured to determine a first mapping matrix based on the first zoom magnification; wherein the first mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a second preview image corresponding to the first zoom magnification; the first preview image and the second preview image being images of the same target object at different zoom magnifications; a transformation module, configured to transform the first preview image into the second preview image based on the first mapping matrix; and output the second preview image; The second determining module is further configured to determine a second mapping matrix between the first camera and the second camera based on the image coordinate system of the first camera and the image coordinate system of the second camera; the second mapping matrix represents a mapping relationship between a first preview image corresponding to an initial zoom magnification of the first camera and a third preview image corresponding to an initial zoom magnification of the second camera; the first preview image and the third preview image are images of the same target object at different zoom magnifications; and the first mapping matrix corresponding to the first zoom magnification is obtained by interpolating the second mapping matrix based on the first zoom magnification.

10. A multi-camera zoom device, characterized in that: include processor; a memory for storing executable instructions; The processor is configured to: implement the multi-camera zoom method according to any one of claims 1 to 8 when executing the executable instructions stored in the memory.

11. A non-transitory computer-readable storage medium, characterized in that When the instructions in the storage medium are executed by a processor of a multi-camera zoom device, the multi-camera zoom device is enabled to perform the multi-camera zoom method according to any one of claims 1 to 8.

Images