Camera zooming method and device, electronic equipment and storage medium

Abstract

The invention discloses a camera zooming method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a to-be-started second camera in response to the zooming operation of a first camera; when the zooming operation meets a camera switching condition, starting a second camera, and generating a transition animation based on a first image of the first camera and a second image of the second camera; and switching the first camera to the second camera based on the transition animation until the second image is displayed.

Description

Technical Field

[0001] The present application relates to the field of camera technology, and in particular to a camera zoom method and device, electronic equipment and storage medium. Background Art

[0002] With the development of electronic devices such as mobile phones and tablets, in order to obtain a better photo-taking experience, more and more cameras are installed on electronic devices, and each camera has a different focal length, such as wide-angle camera, standard camera and telephoto camera.

[0003] In the related art, due to the performance limitation of the chip of the electronic device, all cameras cannot be turned on at the same time. Therefore, during the switching process of cameras with different focal lengths, the camera may not be pulled up in time, resulting in a stagnant picture. As a result, the zooming between multiple cameras is not smooth enough, which seriously affects the smoothness of the continuous zooming process of multiple cameras. Summary of the invention

[0004] The embodiments of the present application provide a camera zoom method and device, an electronic device and a storage medium, which can solve the problem of smoothness in the continuous zoom process of multiple cameras.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a camera zoom method, comprising:

[0007] In response to a zoom operation of the first camera, determining a second camera to be turned on;

[0008] When the zoom operation meets the camera switching condition, the second camera is turned on, and a transition animation is generated based on the first image of the first camera and the second image of the second camera;

[0009] The first camera is switched to the second camera based on a transition animation until the second image is displayed.

[0010] In a second aspect, an embodiment of the present application provides a camera zoom device, comprising:

[0011] a determining unit, configured to determine a second camera to be turned on in response to a zoom operation of the first camera;

[0012] A generating unit, configured to start the second camera when the zoom operation satisfies the camera switching condition, and generate a transition animation based on the first image of the first camera and the second image of the second camera;

[0013] A switching unit is used to switch the first camera to the second camera based on a transition animation until the second image is displayed.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program that can be executed on the processor, and the processor is used to implement the method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the method described in the first aspect.

[0017] The embodiment of the present application provides a camera zoom method and device, an electronic device and a storage medium, which can respond to the zoom operation of the first camera to determine the second camera to be turned on; when the zoom operation meets the camera switching condition, the second camera is turned on, and a transition animation is generated based on the first image of the first camera and the second image of the second camera, and the first camera is switched to the second camera based on the transition animation until the second image is displayed. That is to say, the embodiment of the present application can adopt a camera double-opening strategy, when the zoom operation meets the camera switching condition, the corresponding second camera can be turned on according to the zoom ratio corresponding to the zoom operation, so that the second camera to be switched can be turned on in advance, avoiding the problem of image stagnation caused by the untimely opening of the camera when the camera is switched; in addition, during the camera switching process, a transition animation is generated based on the image of the currently turned on first camera and the image of the second camera to be switched, so that the problem of unsmooth zoom caused by the difference in field of view caused by the limit on the number of chip camera openings and the physical placement of the camera can also be solved, so that the preview effect can be guaranteed and the smoothness of continuous switching between cameras with different focal lengths is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1A This is a schematic diagram of the arrangement of common cameras;

[0019] Figure 1B This is a schematic diagram of the field of view of common cameras;

[0020] Figure 2 A schematic diagram of a camera zoom method according to an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 2 ;

[0022] Figure 4A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 3 ;

[0023] Figure 5 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 4 ;

[0024] Figure 6 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 5 ;

[0025] Figure 7 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 6 ;

[0026] Figure 8 A schematic diagram of the structure of a camera zoom device provided in an embodiment of the present application;

[0027] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0030] It should also be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0031] It should also be noted that in the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0032] For ease of understanding, the following briefly describes the related technologies of the embodiments of the present application. The related technologies here can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and all of them belong to the protection scope of the embodiments of the present application.

[0033] For example, Figure 1A The following is a schematic diagram of the arrangement of common cameras. Figure 1A As shown, the first camera 11 , the second camera 12 , and the third camera 13 are arranged at the upper left corner of the electronic device 10 .

[0034] For example, Figure 1B is a schematic diagram of the field of view of common cameras, such as Figure 1B As shown, the first camera corresponds to the first field of view angle 14, the second camera corresponds to the second field of view angle 15, and the third camera corresponds to the third field of view angle 16. Figure 1B It can be seen that the field of view (FOV) of each camera is different.

[0035] In the related art, due to the performance limitations of the chip of the mobile device, all cameras cannot be turned on at the same time. Therefore, during the camera switching process of multiple focal lengths, you may frequently encounter the situation where the camera is not pulled up in time, resulting in the picture stagnating. In addition, because there is a physical distance between the placement of the cameras, the FOV of each camera is different. At the moment of switching the camera, the preview screen will show a sudden change in FOV. The above two reasons will affect the smoothness of the continuous zoom process of multiple cameras.

[0036] Based on this, the embodiment of the present application provides a camera zoom method and device, an electronic device and a storage medium, which can respond to the zoom operation of the first camera to determine the second camera to be turned on; when the zoom operation meets the camera switching condition, the second camera is turned on, and a transition animation is generated based on the first image of the first camera and the second image of the second camera, and the first camera is switched to the second camera based on the transition animation until the second image is displayed. That is to say, the embodiment of the present application can adopt a camera double-opening strategy, when the zoom operation meets the camera switching condition, the corresponding second camera can be turned on according to the zoom ratio corresponding to the zoom operation, so that the second camera to be switched can be turned on in advance, avoiding the problem of image stagnation caused by the untimely opening of the camera when the camera is switched; in addition, during the camera switching process, a transition animation is generated based on the image of the currently turned on first camera and the image of the second camera to be switched, so that the problem of unsmooth zoom caused by the difference in field of view caused by the limit on the number of chip camera openings and the physical placement of the camera can also be solved, so that the preview effect can be guaranteed and the smoothness of continuous switching between cameras with different focal lengths is improved.

[0037] The embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.

[0038] In one embodiment of the present application, Figure 2 Schematic diagram 1 of a camera zoom method implementation process provided in an embodiment of the present application. Figure 2 As shown, the camera zoom method may include the following steps:

[0039] Step 101: In response to a zoom operation of a first camera, determine a second camera to be turned on.

[0040] In an embodiment of the present application, the camera zoom method is applied to an electronic device, which may include multiple cameras, such as a first camera, a second camera, etc.

[0041] Here, the first camera is the camera currently being used by the electronic device. The first camera may be any camera in the electronic device. In addition, the second camera is the camera to be switched. The second camera may be any camera in the electronic device except the first camera.

[0042] It should be noted that in the embodiment of the present application, the camera may also be referred to as a "camera", which has basic functions such as image acquisition and capture. After the image is acquired through the lens, the photosensitive component circuit and control component in the camera process the image and convert it into a digital signal that can be recognized by the device, and then the image is displayed on the device screen.

[0043] It should also be noted that in the embodiments of the present application, the cameras (including the first camera, the second camera and other cameras mentioned later) have corresponding FOV and zoom ratio intervals. FOV refers to the range of scenes that the camera can capture, usually expressed in angles. The size of the FOV determines the field of view of the optical instrument. The larger the FOV, the larger the field of view. The FOV can be any suitable size, for example, 120°, 150°, etc. The zoom ratio interval can be any suitable size, for example, [0.6, 1], [1, 3], etc.

[0044] In some embodiments, the zoom ratio interval corresponding to the camera represents the interval in which the camera zoom effect is better, that is, the actual zoom range of the camera is greater than the zoom ratio interval corresponding to the camera. Exemplarily, the actual zoom range of the camera is [0.4, 1.2], but the zoom ratio interval corresponding to the camera can be set to [0.6, 1], but there is no limitation.

[0045] In addition, the zoom operation may refer to an operation of adjusting the focal length of the camera lens, and the zoom operation may change the viewing angle and distance of the captured image. In some embodiments, the zoom operation is implemented by any suitable operation, such as a sliding operation, a clicking operation, and the like.

[0046] In some embodiments, the zoom operation can be triggered according to a zoom instruction, and the device can perform a zoom operation on the camera according to the received zoom instruction.

[0047] In some embodiments, there is a correspondence between the zoom operation and the camera. A first correspondence between the zoom operation and the camera can be established. Thus, after receiving the zoom operation of the first camera, the second camera can be determined and turned on according to the first correspondence.

[0048] In some embodiments, the zoom operation has a corresponding target magnification, and a second correspondence between the zoom operation and the target magnification can be established, so that after receiving the zoom operation of the first camera, the target magnification corresponding to the zoom operation can be determined according to the second correspondence.

[0049] In some embodiments, the zoom operation has a corresponding zoom mode, and the zoom mode corresponding to the zoom operation can be determined to determine the second camera to be turned on based on the zoom mode. The zoom mode includes but is not limited to one of the following: a sliding zoom mode and a click zoom mode.

[0050] Step 102: When the zoom operation meets the camera switching condition, turn on the second camera, and generate a transition animation based on the first image of the first camera and the second image of the second camera.

[0051] Here, the camera switching condition is a condition for determining whether to switch the camera. The camera switching condition may include, but is not limited to, one of the following: the zoom operation is greater than a preset duration, the zoom ratio reaches a switching threshold, etc. The preset duration may be any suitable size, for example, 0.2s (seconds), 0.5s, etc. The switching threshold may be any suitable size, for example, 1, 0.5, etc.

[0052] In some embodiments, since the zoom ratio of the camera may be in a state of continuous change, the current zoom ratio at different times during the zoom operation may be determined. When the current zoom ratio reaches the switching threshold, it is determined that the zoom operation meets the camera switching condition. When the current zoom ratio does not reach the switching threshold, it is determined that the zoom operation does not meet the camera switching condition. The current zoom ratio may be any suitable size, for example, 1, 4, etc.

[0053] In some embodiments, different zoom ratio intervals correspond to different switching thresholds, and a third corresponding relationship between the zoom ratio interval and the switching threshold can be established, so that the switching threshold corresponding to the first camera can be determined based on the zoom ratio interval corresponding to the first camera according to the third corresponding relationship.

[0054] In some embodiments, the switching threshold may be a median value corresponding to the zoom factor interval. The median value corresponding to the zoom factor interval refers to the middle value of the zoom factor in the zoom factor interval. The median value corresponding to the zoom factor interval may be any suitable size, for example, 0.6, 1, etc. Exemplarily, the zoom factor interval is [0.5, 1.5], and the median value corresponding to the zoom factor interval may be 1.

[0055] The first image refers to the image captured by the first camera. The second image refers to the image captured by the second camera. The transition animation is the animation played by the device during the period of switching from the first camera to the second camera.

[0056] In some embodiments, the transition animation includes at least one transition image frame. This is because the switching from the first camera to the second camera is completed within a time period, during which the first camera will capture at least one first image frame, and the second camera will capture at least one second image frame, and the transition animation is generated based on the first image and the second image, so the transition animation includes at least one transition image frame.

[0057] In some embodiments, when generating a transition animation based on a first image and a second image, it is necessary to capture the first image and the second image at the same capture moment to obtain the first image and the second image at the capture moment, and then generate the transition animation based on the first image and the second image at at least one capture moment.

[0058] In some embodiments, when the zoom operation does not meet the camera switching condition, the first image may be cropped to obtain a cropped image, and then the cropped image may be enlarged to obtain a preview image, and the preview image may be displayed.

[0059] The method for generating the transition animation may include but is not limited to: generating the transition animation based on the first image and the second image by calling a preset function, generating the transition animation based on the first image and the second image by using video editing software, etc. The preset function may include but is not limited to: at least one of setInterval, transition, keyframes, etc. The video editing software may include but is not limited to: at least one of Adobe Premiere Pro, Final Cut Pro, After Effects, etc.

[0060] In some embodiments, the first image and the second image may be superimposed to generate a transition animation, and the method of superimposing the first image and the second image may include but is not limited to: superimposing the first image and the second image by multiplication mixing, superimposing the first image and the second image by channel mixing, etc. Among them, multiplication mixing can achieve image superposition by multiplying the pixel values ​​of the two images. Channel mixing achieves image superposition by mixing the RGB channels of the image respectively.

[0061] In some embodiments, at least one first feature point of the first image and at least one second feature point of the second image can be determined, and then feature point matching is performed on the at least one first feature point and the at least one second feature point to determine at least one feature point pair, so that the first image and the second image are aligned based on the at least one feature point pair to obtain a transition image, and finally a transition animation is generated based on the transition image.

[0062] Step 103: Switch the first camera to the second camera based on a transition animation until the second image is displayed.

[0063] Here, switching the first camera to the second camera based on the transition animation means that during the process of switching the first camera to the second camera, the picture displayed by the device is the transition animation, and after switching to the second camera, the picture displayed by the device is the second image.

[0064] In some embodiments, after switching to the second camera, the first camera can be switched to an off state, and only the second camera can be kept on, thereby reducing power consumption of the device.

[0065] The embodiment of the present application provides a camera zoom method, which can determine the second camera to be turned on in response to the zoom operation of the first camera; when the zoom operation meets the camera switching condition, the second camera is turned on, and a transition animation is generated based on the first image of the first camera and the second image of the second camera, and the first camera is switched to the second camera based on the transition animation until the second image is displayed. That is to say, the embodiment of the present application can adopt a camera double-opening strategy, and when the zoom operation meets the camera switching condition, the corresponding second camera can be turned on according to the zoom ratio corresponding to the zoom operation, so that the second camera to be switched can be turned on in advance to avoid the problem of image stagnation caused by the untimely opening of the camera when the camera is switched; in addition, during the camera switching process, a transition animation is generated based on the image of the currently turned on first camera and the image of the second camera to be switched, so that the problem of unsmooth zoom caused by the difference in field of view caused by the limit on the number of chip camera openings and the physical placement of the camera can also be solved, so that the preview effect can be guaranteed and the smoothness of continuous switching between cameras with different focal lengths is improved.

[0066] In another embodiment of the present application, Figure 3 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 2 .like Figure 3 As shown, when the first image of the first camera and the second image of the second camera generate a transition animation, the camera zoom method may include the following steps:

[0067] Step 121: Determine at least one first feature point of the first image and at least one second feature point of the second image.

[0068] Here, the first feature point refers to a feature point in the first image. The number of the first feature point is at least one. The second feature point refers to a feature point in the second image. The number of the second feature point is at least one. In some embodiments, the number of the first feature points and the number of the second feature points may be the same or different.

[0069] The method for determining the feature points of an image (including the first image and the second image) may include but is not limited to: determining the feature points of the image through a feature point detection network, determining the feature points of the image through edge detection, etc. For example, the feature point detection network is a deep learning network used for feature point detection, and the feature point detection network directly outputs the feature points in the image by learning the local features of the image. For another example, edge detection can calculate the gradient of the image in the horizontal and vertical directions, and then determine the feature points by calculating the gradient amplitude and direction. Edge detection may include but is not limited to: Canny edge detection, Sobel edge detection, etc.

[0070] In some embodiments, feature extraction may be performed on the first image to determine at least one first feature point, and feature extraction may be performed on the second image to determine at least one second feature point. Methods for extracting features from images may include, but are not limited to: extracting features from images through Histogram of Oriented Gradient (HOG), extracting features from images through Local Binary Pattern (LBP), etc. For example, by using HOG, the image is divided into small connected areas, and then the gradient or edge direction histograms of each pixel in the connected area are collected, and finally these histograms are combined to obtain at least one feature point.

[0071] In some embodiments, key points of an image may be identified to obtain feature points. Key point identification methods include, but are not limited to, Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Oriented FAST and Rotated BRIEF (ORB), etc. Among them, SIFT can detect feature points of an image by constructing a Gaussian difference pyramid. SURF can detect feature points by constructing a scale space of an image. ORB removes edge responses by improving FAST and uses Harris corner detection to detect feature points of an image.

[0072] Step 122: perform feature point matching on at least one first feature point and at least one second feature point to determine at least one feature point pair.

[0073] Here, the feature point pair includes feature points in the first image and the second image that have a matching relationship. Since the number of the first feature point and the second feature point is at least one, the number of the feature point pair is at least one.

[0074] In some embodiments, each feature point pair may include a first feature point and a second feature point.

[0075] Feature point matching is an important task in computer vision, which is used to determine the correspondence between two images. Methods for matching the first feature point with the second feature point may include, but are not limited to: matching the first feature point with the second feature point based on Euclidean distance, matching the first feature point with the second feature point based on a deep feature matching network, etc. For example, the Euclidean distance refers to the true distance between two feature points. The Euclidean distance between two feature points is calculated. The smaller the Euclidean distance, the more similar the two feature points are. Therefore, for a first feature point, the second feature point with the smallest Euclidean distance to the first feature point is used as the feature point matched with the first feature point, thereby obtaining a feature point pair. For another example, a deep feature matching network can be used to learn the deep feature representation of the feature points of an image, and the matching relationship between the feature points can be directly obtained through the output of the network to perform feature point matching on the first feature point and the second feature point.

[0076] In some embodiments, the feature points have color information and texture information. When matching the feature points, the color information and texture information of the feature points may be combined for matching, thereby improving the accuracy of the feature point matching.

[0077] Step 123: Align the first image and the second image based on at least one feature point pair to obtain a transition image.

[0078] Here, the transition image refers to an image obtained by aligning the first image and the second image. In some embodiments, in the process of switching the first camera to the second camera, the number of the transition image may be at least one.

[0079] In some embodiments, since a frame of the first image and a frame of the second image can be obtained at each acquisition moment, it is necessary to determine at least one feature point pair of the first image and the second image at each acquisition moment, and align the first image and the second image based on the at least one feature point pair to obtain transition images at multiple acquisition moments.

[0080] The method for aligning the first image and the second image may include, but is not limited to: aligning the first image and the second image through a geometric transformation model, aligning the first image and the second image through a convolutional neural network, etc. For example, the geometric transformation model may include, but is not limited to: at least one of a rigid body transformation model, an affine transformation, a perspective transformation model, etc. The geometric transformation model calculates the pixel values ​​of the transition image based on the feature point pairs, thereby aligning the first image and the second image to obtain the transition image. For another example, at least one feature point pair may be input into a trained convolutional neural network, and the result output by the convolutional neural network is the transition image.

[0081] In some embodiments, the geometric transformation model has corresponding transformation parameters. The transformation parameters of the geometric transformation model can be estimated by using the least squares method, the RANSAC (Random Sample Consensus) algorithm, etc. using the matched feature point pairs, and then the geometric transformation model is adjusted according to the calculated transformation parameters to align the first image and the second image through the adjusted geometric transformation model. For example, a linear equation system is constructed by using the least squares method, and the transformation parameters are obtained by solving the equation system. For another example, the geometric transformation model is estimated by randomly selecting some matching point pairs through the RANSAC algorithm, and then the errors between all matching point pairs and the model are calculated, and the model with the smallest error is selected as the final geometric transformation model.

[0082] In some embodiments, the first image and the second image can be divided respectively based on at least one feature point pair to determine at least one first divided block corresponding to the first image and at least one second divided block corresponding to the second image. Then, based on the matching relationship between the feature points in the first divided block and the feature points in the second divided block, the first divided block and the second divided block are locally aligned to obtain at least one registration block, and then the boundary of the at least one registration block is fused to obtain a transition image.

[0083] Step 124: Generate a transition animation based on the transition image.

[0084] Here, the number of the transition image is at least one, and the at least one transition image can be directly used as the transition animation, so that the transition animation can be generated based on the transition image.

[0085] In some embodiments, since the transition images are generated based on the first image and the second image at different acquisition times, different transition images are generated at different times. The transition images can be sorted from early to late according to the generation times corresponding to the transition images, and then the sorted transition images are used as transition animations, so that transition animations can be generated based on the transition images.

[0086] In an embodiment of the present application, the first image and the second image are aligned to obtain a transition image by using a feature point pair determined by a first feature point of the first image and a second feature point of the second image to generate a transition animation. The obtained transition animation may include features of the first image and the second image, thereby improving the accuracy of the transition animation, thereby enabling smoother camera switching when switching cameras based on the transition animation.

[0087] In another embodiment of the present application, Figure 4 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 3 .like Figure 4 As shown, when the first image and the second image are aligned based on at least one feature point pair to obtain a transition image, the camera zoom method may include the following steps:

[0088] Step 1231: Based on at least one feature point pair, divide the first image and the second image respectively to determine at least one first divided block corresponding to the first image and at least one second divided block corresponding to the second image.

[0089] Here, the first divided block is a partial image of the first image. The number of the first divided blocks may be at least one. The second divided block is a partial image of the second image. The number of the second divided blocks may be at least one. In some embodiments, the number of the first divided blocks may be the same as the number of the second divided blocks.

[0090] The method of dividing an image (including a first image and a second image) may include but is not limited to: dividing an image based on a straight line of a feature point pair, dividing an image based on regional growth of a feature point pair, etc. For example, a pair of matching feature point pairs may be selected, and a straight line may be determined with two feature points in the feature point pair as endpoints, as the image dividing line, thereby determining at least one image dividing line based on at least one feature point pair, so as to divide the image into divided blocks. For another example, a feature point in the matching feature point pair may be used as a seed point, and then, based on a set growth criterion, starting from the seed point, adjacent pixel points may be gradually included in the same area until no further growth is possible, thereby dividing the image into divided blocks.

[0091] In some embodiments, when the image is divided into blocks based on at least one feature point pair, the blocks may be of fixed size or may be adaptively divided according to the feature points of the image. This is to more effectively handle the parallax problem of each local area.

[0092] Step 1232: Based on the matching relationship between the feature points in the first divided block and the feature points in the second divided block, locally align the first divided block with the second divided block to obtain at least one registration block.

[0093] Here, since the alignment result obtained by directly aligning the first image and the second image is only a rough alignment, there will still be large parallax misalignment in the image of the alignment result. Parallax refers to the position difference of the same scene in two images due to different camera shooting angles or different distances between cameras. Therefore, it is necessary to adopt a local alignment method.

[0094] The registration block is a local image obtained by locally aligning the first partition block and the second partition block. The number of the registration blocks can be at least one. In some embodiments, the number of the registration blocks, the number of the first partition blocks, and the number of the second partition blocks can be the same.

[0095] The method of locally aligning the first partition block with the second partition block may include, but is not limited to: locally aligning the first partition block with the second partition block through a geometric transformation model, locally aligning the first partition block with the second partition block through a convolutional neural network, etc. The geometric transformation model may include, but is not limited to: at least one of a rigid body transformation model, an affine transformation, a perspective transformation model, etc.

[0096] In some embodiments, when locally aligning the first partition block and the second partition block, a loss function can be set for the area where the first partition block and the second partition block are located. The loss function can measure the quality of the current registration, and when the loss value of the registration block meets the loss condition, the registration block can be output to obtain at least one registration block. For example, the loss function can be defined using the square error based on the feature point deviation, the photometric consistency error, etc. At the same time, the transformation parameters of each local area are optimized by minimizing the loss function. This is usually an iterative process, which can be solved using optimization algorithms such as gradient descent and Newton's method.

[0097] Step 1233: perform boundary fusion on at least one registration block to obtain a transition image.

[0098] Here, since the registration block is a part of the transition image, it is necessary to perform boundary fusion on at least one registration block to obtain the transition image.

[0099] Boundary fusion is the process of fusing several smaller images to form a larger image. Methods for boundary fusion of registration blocks may include, but are not limited to: boundary fusion of registration blocks by using Python and NumPy, boundary fusion of registration blocks by using OpenCV, etc. For example, when boundary fusion of registration blocks is performed using Python and NumPy, the image size of the transition image is first determined, and then a new blank canvas is created, the size of which is the same as the image size of the transition image, and finally each registration block is looped through, and each registration block is placed in the corresponding position of the canvas according to the position of each registration block in the transition image to fuse the registration blocks to obtain a transition image. For another example, when boundary fusion of registration blocks is performed using OpenCV, concatenated_image() or other functions may be called to boundary fusion of the registration blocks to obtain a transition image.

[0100] In some embodiments, when performing boundary fusion on at least one registration block, in order to ensure the consistency of alignment results between local regions, smoothing processing, transition zone optimization and other technologies may be used to reduce artifacts at region boundaries and more accurately fuse the registration blocks.

[0101] In an embodiment of the present application, the first image and the second image are respectively divided by feature point pairs, and the first divided block and the second divided block obtained by the division are locally aligned to obtain a registration block, and the registration blocks are boundary fused to obtain a transition image, thereby reducing the possibility of large parallax misalignment in the transition image obtained by only using overall alignment; at the same time, by boundary fusion of the registration blocks, artifacts at the boundaries of the registration blocks can also be reduced, thereby improving the accuracy of the transition image.

[0102] In a possible implementation, when determining the second camera to be turned on, the method may further include: determining a zoom mode corresponding to the zoom operation; determining the second camera to be turned on based on the zoom mode; wherein the zoom mode includes: a sliding zoom mode or a click zoom mode.

[0103] Here, since the triggering actions of the zoom operation are different, the zoom modes corresponding to different zoom operations may be different. In some embodiments, the zoom mode may include: a sliding zoom mode or a click zoom mode. Among them, the sliding zoom mode refers to a mode of changing the zoom ratio by sliding. The click zoom mode refers to a mode of changing the zoom ratio by clicking.

[0104] In some embodiments, the zoom mode corresponding to the zoom operation can be determined according to the detected interaction operation. The interaction operation can be any suitable operation, such as sliding, clicking, double-clicking, etc. Exemplarily, when the interaction operation is sliding, the zoom mode corresponding to the zoom operation can be a sliding zoom mode.

[0105] In some embodiments, there is a correspondence between the zoom operation and the zoom mode. A fourth correspondence between the zoom operation and the zoom mode can be established, so that the zoom mode corresponding to the zoom operation can be determined based on the acquired zoom operation according to the fourth correspondence.

[0106] In some embodiments, the second camera may be any camera in the device. In some embodiments, the second cameras corresponding to different zoom modes may be the same or different.

[0107] In some embodiments, there is a correspondence between the zoom mode and the camera. A fifth correspondence between the zoom mode and the camera can be established, so that the second camera to be turned on can be determined based on the zoom mode corresponding to the zoom operation according to the fifth correspondence.

[0108] In some embodiments, when the zoom mode is a sliding zoom mode, a transition camera adjacent to the first camera can be determined based on the sliding zoom mode, and then the transition camera is determined as the second camera. When the zoom mode is a click zoom mode, a target magnification can be determined based on the click zoom mode, and then the target camera corresponding to the target magnification is determined as the second camera.

[0109] In an implementation manner of the present application, the second camera to be turned on is determined by the zoom mode corresponding to the zoom operation, and a zoom mode including a sliding zoom mode or a click zoom mode is proposed. The second camera can be determined under different zoom modes and is compatible with different scenes, thereby improving the flexibility of the camera zoom method.

[0110] In another possible implementation, when determining the second camera to be turned on, the method may further include:

[0111] When the zoom mode is a sliding zoom mode, determining a transition camera adjacent to the first camera based on the sliding zoom mode, and determining the transition camera as the second camera;

[0112] When the zoom mode is a click-to-zoom mode, a target magnification is determined based on the click-to-zoom mode, and a target camera corresponding to the target magnification is determined as the second camera.

[0113] Here, for the target magnification that the user expects to reach, the corresponding camera is called the target camera. Among them, the transition camera can refer to other cameras between the first camera and the target camera. Since the sliding zoom method is a process of gradually changing the focal magnification, a transition camera adjacent to the first camera can be determined based on the sliding zoom method, so that the transition camera is determined as the second camera.

[0114] In an embodiment of the present application, the transition camera may be a camera that is adjacent to the first camera. For example, the transition camera may be the next camera of the first camera, or the previous camera of the first camera. This is because the sliding direction of the sliding zoom mode is different, and the change process of the focal ratio is different. The sliding direction of the sliding zoom mode may include but is not limited to: sliding forward or sliding backward. Exemplarily, the first camera is i, the transition camera is j, and j may be i+1 or i-1.

[0115] In some embodiments, when the sliding direction of the sliding zoom mode is forward sliding, the zoom ratio can change from small to large, and at this time, j can be i + 1. Correspondingly, when the sliding direction of the sliding zoom mode is backward sliding, the zoom ratio can change from large to small, and at this time, j can be i-1.

[0116] In some embodiments, there is a continuous relationship between the zoom magnification interval corresponding to the transition camera and the zoom magnification interval corresponding to the first camera, so the transition camera is adjacent to the first camera. Exemplarily, the zoom magnification interval corresponding to the first camera is [0.6, 1], and the zoom magnification interval corresponding to the transition camera can be [0.1, 0.6] or [1, 1.9].

[0117] In some embodiments, a target camera corresponding to the target magnification can be obtained, and then when there is at least one transition camera between the first camera and the target camera, at least one transition camera is used as the first camera in turn, and based on the current sliding magnification, the transition camera adjacent to the first camera is used as the second camera until the target camera is turned on.

[0118] In some embodiments, the target magnification refers to the magnification that the zoom operation needs to achieve. The target magnification can be any suitable size, for example, 5, 2.3, etc.

[0119] In some embodiments, there is a correspondence between the click zoom mode and the zoom ratio. A sixth correspondence between the zoom mode and the zoom ratio can be established, so that the target ratio can be determined based on the zoom mode corresponding to the zoom operation according to the sixth correspondence.

[0120] In some embodiments, a target camera corresponding to the target magnification may be obtained, and then when there is at least one transition camera between the first camera and the target camera, the at least one transition camera may be skipped and the target camera may be directly opened.

[0121] In an implementation manner of the present application, when the zoom mode is a sliding zoom mode, the transition camera adjacent to the first camera is used as the second camera. When the zoom mode is a click zoom mode, the target camera corresponding to the target magnification is determined as the second camera. Different methods can be used to determine the second camera for different scenes, thereby improving the accuracy of the second camera.

[0122] In a possible implementation, when the zoom mode is a sliding zoom mode, the method may further include: obtaining a target camera corresponding to the target magnification; when there is at least one transition camera between the first camera and the target camera, using at least one transition camera as the first camera in turn, and based on the current sliding magnification, using the transition camera adjacent to the first camera as the second camera until the target camera is turned on.

[0123] Here, the target camera refers to the camera that the zoom operation ultimately opens. The target camera can be any camera in the device. In some embodiments, the target camera and the second camera can be the same or different.

[0124] In some embodiments, there is a correspondence between the zoom ratio and the camera. A seventh correspondence between the zoom ratio and the camera can be established, so that based on the target ratio, the target camera corresponding to the target ratio can be determined according to the seventh correspondence.

[0125] In some embodiments, the target camera corresponding to the target magnification can be determined based on the target magnification and the zoom magnification interval corresponding to the camera. After determining the target magnification, the zoom magnification interval in which the target magnification is located can be determined, and then the camera corresponding to the zoom magnification interval in which the target magnification is located is used as the target camera corresponding to the target magnification. Exemplarily, the target magnification is 1.5, and the zoom magnification interval includes [0.6, 1], [1, 3], and the camera with a zoom magnification interval of [1, 3] can be used as the target camera.

[0126] In some embodiments, since the sliding zoom mode can be a continuous sliding operation, the zoom ratio of the device is in a continuously changing state. When the zoom operation stops, the zoom ratio of the device can be used as the target ratio, and then the target camera corresponding to the target ratio is obtained.

[0127] In some embodiments, since the sliding zoom method can be a continuous sliding operation, when sliding to a transition camera adjacent to the first camera, the camera may be switched continuously by sliding until the target camera is switched. Therefore, there may be at least one transition camera between the first camera and the target camera.

[0128] In some embodiments, the current sliding magnification refers to the zoom magnification detected by the device at the current moment of switching the camera. The current sliding magnification can be any suitable size, for example, 0.6, 1.5, etc.

[0129] In some embodiments, since the sliding zoom method can be a continuous sliding operation, the current sliding ratio continues to change. The change process of the current sliding ratio can be determined based on the sliding direction of the sliding zoom method, and then the next camera or the previous camera of the first camera can be used as the second camera.

[0130] In some embodiments, since at least one transition camera is sequentially used as the first camera, the number of transition cameras adjacent to the first camera is at least one, and correspondingly, the number of second cameras is at least one.

[0131] In some embodiments, when there is no transition camera between the first camera and the target camera, the target camera may be a transition camera adjacent to the first camera. In this case, the transition camera is used as the target camera, and the target camera may be turned on.

[0132] In an embodiment of the present application, when the zoom mode is a sliding zoom mode, by using at least one transition camera as the first camera in turn, and using the transition camera adjacent to the first camera as the second camera until the target camera is turned on, the target camera can be gradually and smoothly transitioned to be turned on, thereby improving the smoothness of the camera's continuous zoom process.

[0133] In another possible implementation, when the zoom mode is a click zoom mode, the method may further include: obtaining a target camera corresponding to the target magnification; and when there is at least one transition camera between the first camera and the target camera, skipping at least one transition camera.

[0134] Here, the target camera can be any camera in the device. In some embodiments, the target camera and the second camera can be the same or different.

[0135] In some embodiments, the zoom magnification interval corresponding to the device can be displayed in the form of a control, so that when the click position of the click zoom mode in the control is detected, the target magnification can be determined from the zoom magnification interval corresponding to the device based on the click position, and then the target camera corresponding to the target magnification is determined based on the target magnification and the zoom magnification interval corresponding to the camera. Exemplarily, the first camera is i, the target camera is l, and l can be i+n or in, where n=1,2,3….

[0136] In some embodiments, there is a correspondence between the zoom ratio and the camera. A seventh correspondence between the zoom ratio and the camera can be established, so that based on the target ratio, the target camera corresponding to the target ratio can be determined according to the seventh correspondence.

[0137] In some embodiments, since the target camera may be a camera that has no adjacent relationship with the first camera, there may be at least one transition camera between the first camera and the target camera.

[0138] In some embodiments, since the duration of the click-to-zoom mode is very short, when there is at least one transition camera between the first camera and the target camera, the at least one transition camera may not be turned on, and the target camera may be directly used as the second camera. Then, the second camera may be turned on to generate a transition animation based on the first image of the first camera and the second image of the second camera, thereby switching the first camera to the second camera based on the transition animation.

[0139] In an embodiment of the present application, when the zoom mode is a click zoom mode, by skipping at least one transition camera to turn on the target camera, it is possible to directly switch from the first camera to the target camera without turning on the transition camera, thereby simplifying the camera zoom process and improving the efficiency of the camera zoom.

[0140] In a possible implementation, when the zoom operation does not meet the camera switching condition, the method may further include the following steps: performing a cropping operation on the first image to obtain a cropped image; performing a zooming operation on the cropped image to obtain a preview image, and displaying the preview image.

[0141] Here, the camera switching condition may include but is not limited to one of the following: the zoom operation is greater than a preset duration, the zoom ratio reaches a switching threshold, etc. Exemplarily, the zoom operation does not meet the camera switching condition may be that the current zoom ratio does not reach the switching threshold.

[0142] The cropping method of the first image may include but is not limited to: cropping the first image through a detection frame, cropping the first image through deep learning, etc. For example, the detection frame can surround the object, so that the first image can be cropped. The first image can be cropped according to the detection frame by calling a preset function in the OpenCV library. The preset function may include but is not limited to: cv2.waitKey(), cv2.imwrite(), etc. OpenCV is an open source computer vision and machine learning software library. It provides the functions of reading, displaying and processing images. For another example, deep learning uses a multi-layer neural network to simulate the complex decision-making ability of the human brain, and can perform foreground extraction on the input target image, thereby realizing the cropping operation on the first image.

[0143] In some embodiments, since the cropped image is obtained by performing a cropping operation on the first image, the cropped image may be a partial image of the first image, and the image size of the cropped image may be smaller than the image size of the first image.

[0144] In some embodiments, the first image may be cropped according to the relationship between the magnification and the focal length by digital zoom to obtain a cropped image, and the number of pixels of the cropped image is reduced.

[0145] In some embodiments, when the cropping operation is performed on the first image, the image in the central area of ​​the first image may be cropped to obtain a cropped image.

[0146] In some embodiments, the preview image refers to an image to be displayed. In some embodiments, since the preview image is obtained by enlarging the cropped image, the display content of the preview image is the same as the display content of the cropped image, and the image size of the preview image is larger than the image size of the cropped image.

[0147] The method of enlarging the cropped image may include but is not limited to: enlarging the cropped image through pixel interpolation, enlarging the cropped image through image editing software, and enlarging the cropped image through a graphics processor (Graphic Process Unit, GPU). For example, pixel interpolation may include but is not limited to: at least one of nearest neighbor interpolation, bilinear interpolation, etc. Among them, nearest neighbor interpolation fills new pixels by copying the nearest pixel value to enlarge the cropped image. Bilinear interpolation calculates new pixel values ​​by averaging the values ​​of four surrounding pixels to enlarge the cropped image. For another example, the image editing software may include but is not limited to: at least one of Photoshop, GIMP, etc. The built-in enlargement tool of the image editing software can enlarge the image, so that the cropped image can be enlarged. For another example, GPU is a coprocessor for processing image and graphic operations, and the cropped image can be enlarged by calling GPU.

[0148] In some embodiments, the cropped image is enlarged, which is equivalent to the central area of ​​the first image being enlarged and displayed. This process is similar to enlarging an area in image processing software, and the resulting preview image can be displayed on the screen.

[0149] In an embodiment of the present application, when the zoom operation does not meet the camera switching condition, the cropped image obtained by performing a cropping operation on the first image is enlarged to obtain a displayed preview image, and the preview image can be changed according to the zoom operation through digital zoom, thereby improving the accuracy of the displayed image, so that the first image can be obtained more accurately when it is necessary to switch the camera.

[0150] In the embodiment of the present application, a camera zoom method is proposed, which can respond to the zoom operation of the first camera and determine the second camera to be turned on; when the zoom operation meets the camera switching condition, the second camera is turned on, and a transition animation is generated based on the first image of the first camera and the second image of the second camera, and the first camera is switched to the second camera based on the transition animation until the second image is displayed. That is to say, the embodiment of the present application can adopt a camera double-opening strategy, and when the zoom operation meets the camera switching condition, the corresponding second camera can be turned on according to the zoom ratio corresponding to the zoom operation, so that the second camera to be switched can be turned on in advance to avoid the problem of image stagnation caused by the untimely opening of the camera when the camera is switched; in addition, during the camera switching process, a transition animation is generated based on the image of the currently turned on first camera and the image of the second camera to be switched, so that the problem of unsmooth zoom caused by the difference in field of view caused by the limit on the number of chip cameras turned on and the physical placement of the camera can also be solved, so that the preview effect can be guaranteed and the smoothness of continuous switching between cameras with different focal lengths is improved.

[0151] The following describes the application of the camera zoom method provided in an embodiment of the present application in actual scenarios.

[0152] Based on the above embodiments, Figure 5 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 4 .like Figure 5 As shown, the camera zoom method may include the following steps:

[0153] Step 201, specifying zoom ratio ranges of different cameras in the device;

[0154] Step 202: Obtain a user zoom instruction;

[0155] Step 203, obtaining a camera dual-opening strategy corresponding to the zoom instruction;

[0156] Step 204: Based on the camera dual-opening strategy, when the zoom operation does not meet the camera switching condition, digital zoom is performed; when the zoom operation meets the camera switching condition, the second camera is turned on;

[0157] Step 205: Acquire a first image from the first camera and a second image from the second camera;

[0158] Step 206: Generate a transition animation based on the first image and the second image;

[0159] Step 207: Switch the first camera to the second camera based on the transition animation until the second image is displayed.

[0160] Each camera has a different zoom ratio range, and a specific preview camera (first camera) is selected according to the ratio input by the user; and according to the camera dual-opening strategy formulated in advance, other specific cameras are opened at the ratio, including the camera corresponding to the zoom end point (second camera); when there is no need to switch cameras, the picture collected by the first camera is cropped and scaled according to the relationship between the ratio and focal length through digital zoom, and sent to the display for preview; when it is necessary to switch cameras, the picture of the first camera and the picture of the second camera are used to make a transition animation through image stitching, fusion and edge elimination; finally, the preview screen is naturally transitioned to the screen of the second camera.

[0161] Specifically, each camera has a different zoom ratio range, from wide angle to telephoto, to meet various shooting needs. For example, the zoom ratio range of camera A is [0.6,1], and the zoom ratio range of camera B is [1,3]. According to the magnification input by the user, the system will automatically select the corresponding preview camera as the first camera according to the magnification range to ensure the best image quality and detail performance. The system's intelligent algorithm can also evaluate factors such as current lighting conditions and the distance of the object being photographed to further optimize the camera selection and settings, so that users can get satisfactory shooting results without manual adjustment.

[0162] At the same time, a dual-camera strategy is formulated in advance. It is agreed that at a specific magnification, in addition to opening the first camera, other specific cameras will be opened, including the target camera corresponding to the end point of the zoom. This can avoid the problem of image stagnation caused by the camera not being opened in time when switching cameras.

[0163] When there is no need to switch cameras, the image captured by the first camera is cropped according to the relationship between the magnification and the focal length through digital zoom, and then the cropped image is interpolated and enlarged, the number of pixels of the cropped part is reduced, and the center area of ​​the image is enlarged and displayed. This is just like enlarging an area in image processing software. The result of digital zoom is sent to the monitor for preview.

[0164] When it is necessary to switch cameras, the system will intelligently use the images from the first camera and the second camera to create a smooth and natural transition animation by stitching and fusion of images and eliminating edges. This process not only ensures seamless image switching, but also greatly enhances the user's visual experience. First, the system will conduct a detailed analysis of the images from the first camera and the second camera to extract the key feature points in the image. Then, the feature points of the two images are aligned through the image stitching algorithm to achieve accurate image superposition. To avoid obvious boundary lines at the image stitching, the system uses advanced edge elimination technology to process the transition area, making the two images more harmonious and consistent in color, brightness and texture.

[0165] The final preview screen naturally transitions to display the image captured by the second camera.

[0166] In the camera zoom method proposed in the embodiment of the present application, the second camera to be turned on can be determined in response to the zoom operation of the first camera; when the zoom operation meets the camera switching condition, the second camera is turned on, and a transition animation is generated based on the first image of the first camera and the second image of the second camera, and the first camera is switched to the second camera based on the transition animation until the second image is displayed. That is to say, the embodiment of the present application can adopt a camera double-opening strategy, and when the zoom operation meets the camera switching condition, the corresponding second camera can be turned on according to the zoom ratio corresponding to the zoom operation, so that the second camera to be switched can be turned on in advance to avoid the problem of image stagnation caused by the untimely opening of the camera when the camera is switched; in addition, during the camera switching process, a transition animation is generated based on the image of the currently turned on first camera and the image of the second camera to be switched, so that the problem of unsmooth zoom caused by the difference in field of view caused by the limitation of the number of chip cameras turned on and the physical placement of the camera can also be solved, so that the preview effect can be guaranteed and the smoothness of continuous switching between cameras with different focal lengths is improved.

[0167] Based on the above embodiments, Figure 6 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 5 .like Figure 6 As shown, the camera zoom method includes a camera double-opening strategy, and the camera zoom method may include the following steps:

[0168] Step 31, obtaining zoom mode and first camera;

[0169] Step 32, determining whether the zoom mode is a sliding zoom mode;

[0170] Here, if yes, go to step 33, if not, the zoom mode is the click zoom mode, then go to step 35.

[0171] Step 33, determining a target magnification corresponding to the sliding zoom mode, and obtaining a target camera corresponding to the target magnification;

[0172] Step 34: based on the current sliding magnification, open at least one transition camera until the target camera is opened;

[0173] Step 35, determining the target magnification corresponding to the click zoom mode, and obtaining the target camera corresponding to the target magnification;

[0174] Step 36: skip at least one transition camera and turn on the target camera;

[0175] Step 37: Generate a transition animation according to the zoom ratio change process, and switch the camera based on the transition animation.

[0176] The specific processing steps are as follows:

[0177] Get the magnification input by the user, and determine the source camera (corresponding to the first camera mentioned above) corresponding to the current input magnification based on the zoom magnification range of each camera. The determination of the target magnification is discussed in two cases:

[0178] When the user is sliding zoom, according to whether the current zoom is zoomin (zoom ratio from small to large) or zoom out (zoom ratio from large to small), the camera j adjacent to the source camera magnification interval is determined as the target camera (corresponding to the aforementioned second camera), that is, j is equal to i+1 or i-1;

[0179] When the user clicks to zoom, the click magnification input by the user and the zoom magnification range of each camera are obtained to determine the target camera corresponding to the current input click magnification. Because click zoom is also divided into zoomin (zoom magnification from small to large) or zoom out (zoom magnification from large to small), but the user's input magnification may not be in the magnification range adjacent to the source camera, so j is equal to i+n or in, (where n=1,2,3...).

[0180] In the embodiment of the present application, the first camera may be a source camera, and accordingly, the first image may be a source picture; the second camera may be a target camera, and accordingly, the second image may be a target picture.

[0181] In the embodiment of the present application, a dual-camera opening strategy is formulated. Here, the dual-camera opening strategy is to open two cameras at the same time within a specific magnification range.

[0182] When sliding zoom and clicking zoom, the zoom ratio changes continuously, so as to ensure the continuous transition of the preview screen. The processing difference of this solution is:

[0183] When sliding zoom, select the transition camera that should be opened in the middle according to the continuously changing magnification, and according to the formulated double-open strategy, open the transition camera in advance before switching cameras until switching to the target camera.

[0184] When you click zoom, you can directly open the target camera, that is, the camera that spans several adjacent magnification ranges in the middle, thus implementing the cross-camera strategy.

[0185] When sliding zoom, if the switching point is not reached, digital zoom processing is performed based on the source camera, and animation transition processing is performed when switching the intermediate transition camera.

[0186] When you click to zoom, if the target camera switching point is not reached, the transition animation is processed based on the source camera until it switches to the target camera, that is, several transition cameras are crossed in the middle.

[0187] In the camera dual-opening strategy proposed in the embodiment of the present application, different processes can be selected to switch cameras under different zoom modes, which is compatible with different scenes and improves the flexibility of the camera zoom method.

[0188] Based on the above embodiments, Figure 7 A schematic diagram of the implementation process of a camera zoom method provided in an embodiment of the present application Figure 6 .like Figure 7 As shown, the camera zoom method includes a processing flow of a transition image, and the camera zoom method may include the following steps:

[0189] Step 41: Match feature points based on the feature points of the source image and the feature points of the target image to determine at least one feature point pair;

[0190] Step 42: Based on at least one feature point pair, the source image and the target image are divided and locally aligned to obtain at least one registration block;

[0191] Step 43: perform boundary fusion on at least one registration block to obtain a transition image.

[0192] The specific processing steps are as follows:

[0193] 1. Perform feature point matching based on the feature points of the source image and the feature points of the target image to determine at least one feature point pair.

[0194] 2. Based on at least one feature point pair, the source image and the target image are divided and locally aligned to obtain at least one registration block:

[0195] Here, due to the influence of parallax, the image alignment result obtained in step 1 is only a rough registration, and there will still be large parallax misalignment in the result image. Parallax refers to the position difference of the same scene in two images due to different camera shooting angles or different distances between cameras. In this case, we adopt a more precise alignment method.

[0196] In order to align the images more accurately, the images are divided into several smaller blocks or grids. These blocks can be of fixed size or adaptively divided according to the characteristics of the image. The purpose of this is to be able to more effectively handle the parallax problem in each local area.

[0197] In each divided area, the feature points of the image are detected and matched. The quality of feature point matching has a great impact on the subsequent alignment accuracy, so it is necessary to select appropriate feature point detection and descriptor matching methods.

[0198] In order to perform local alignment, a loss function needs to be set for each divided area. The loss function can measure the quality of the current registration. For example, the loss function can be defined using the square error based on the feature point deviation, the photometric consistency error, etc.

[0199] The transformation parameters of each local area are optimized by minimizing the loss function. This is usually an iterative process and can be solved using optimization algorithms such as gradient descent and Newton's method.

[0200] 3. Perform boundary fusion on at least one registration block to obtain a transition image:

[0201] The transformation results of each local area are fused to obtain a fine alignment of the entire image. To ensure the consistency of the alignment results between local areas, smoothing processing, transition area optimization and other technologies are used to reduce artifacts at the region boundaries.

[0202] In the processing flow of the transition image proposed in the embodiment of the present application, the source camera image and the target camera image are analyzed in detail to extract the key feature points in the image. Then, the feature points of the two images are aligned through the image stitching algorithm to achieve accurate image superposition. At the same time, in order to avoid obvious boundary lines at the image splicing, the registration block is processed by boundary fusion, which reduces the artifacts at the boundary of the registration block and improves the accuracy of the transition image.

[0203] Based on the above embodiments, a camera zoom device is proposed in an embodiment of the present application.

[0204] In yet another embodiment of the present application, Figure 8 This is a schematic diagram of the structure of a camera zoom device provided in an embodiment of the present application. Figure 8 As shown, the camera zoom device 800 proposed in the embodiment of the present application may include:

[0205] A determining unit 801, configured to determine a second camera to be turned on in response to a zoom operation of the first camera;

[0206] A generating unit 802, configured to enable the second camera when the zoom operation satisfies the camera switching condition, and generate a transition animation based on the first image of the first camera and the second image of the second camera;

[0207] The switching unit 803 is configured to switch the first camera to the second camera based on a transition animation until the second image is displayed.

[0208] In some embodiments, the generation unit 802 is also used to determine at least one first feature point of the first image and at least one second feature point of the second image; perform feature point matching on at least one first feature point and at least one second feature point to determine at least one feature point pair; align the first image and the second image based on at least one feature point pair to obtain a transition image; and generate a transition animation based on the transition image.

[0209] In some embodiments, the generation unit 802 is also used to divide the first image and the second image respectively based on at least one feature point pair, determine at least one first divided block corresponding to the first image and at least one second divided block corresponding to the second image; based on the matching relationship between the feature points in the first divided block and the feature points in the second divided block, locally align the first divided block and the second divided block to obtain at least one registration block; and perform boundary fusion on at least one registration block to obtain a transition image.

[0210] In some embodiments, the determination unit 801 is further used to determine a zoom mode corresponding to the zoom operation; and determine the second camera to be turned on based on the zoom mode; wherein the zoom mode includes: a sliding zoom mode or a click zoom mode.

[0211] In some embodiments, the determination unit 801 is also used to determine a transition camera adjacent to the first camera based on the sliding zoom mode when the zoom mode is a sliding zoom mode, and determine the transition camera as the second camera; when the zoom mode is a click zoom mode, determine the target magnification based on the click zoom mode, and determine the target camera corresponding to the target magnification as the second camera.

[0212] In some embodiments, when the zoom mode is a sliding zoom mode, the determination unit 801 is also used to obtain the target camera corresponding to the target magnification; when there is at least one transition camera between the first camera and the target camera, at least one transition camera is used as the first camera in turn, and based on the current sliding magnification, the transition camera adjacent to the first camera is used as the second camera until the target camera is turned on.

[0213] In some embodiments, when the zoom mode is a click zoom mode, the determination unit 801 is further used to obtain a target camera corresponding to the target magnification; when there is at least one transition camera between the first camera and the target camera, at least one transition camera is skipped.

[0214] In some embodiments, when the zoom operation does not meet the camera switching condition, the camera zoom device 800 may also include an operating unit for cropping the first image to obtain a cropped image; enlarging the cropped image to obtain a preview image, and displaying the preview image.

[0215] In yet another embodiment of the present application, Fig. 9 The following is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 900 proposed in the embodiment of the present application may include a processor 901, a memory 902, a communication interface 903, and a bus 904 for connecting the processor 901, the memory 902, and the communication interface 903, and a camera module 905. Among them, the communication interface 903 is used to receive and send signals during the process of sending and receiving information between other external devices; the memory 902 is used to store a computer program that can be run on the processor 903; the processor 903 is used to execute any of the camera zoom methods in the aforementioned embodiments when running the computer program.

[0216] In the embodiment of the present application, the processor 901 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other, and the embodiment of the present application is not specifically limited. The electronic device 900 can also include a memory 902, which can be connected to the processor 901, wherein the memory 902 is used to store executable program code, the program code includes computer operation instructions, and the memory 902 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0217] In the embodiment of the present application, the bus 904 is used to connect the communication interface 903, the processor 901 and the memory 902, as well as the mutual communication between these devices.

[0218] In practical applications, the memory 902 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor 901.

[0219] In the embodiment of the present application, the camera module 905 includes at least one camera, and the at least one camera may include but is not limited to: at least one of a first camera, a second camera, etc.

[0220] In addition, each functional module in the embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional modules.

[0221] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0222] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the camera zoom method as described above is implemented.

[0223] Specifically, a computer program or instruction corresponding to a camera zoom method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When a computer program or instruction corresponding to a camera zoom method in a storage medium is read or executed by an electronic device, the following steps are included:

[0224] In response to a zoom operation of the first camera, determining a second camera to be turned on;

[0225] When the zoom operation meets the camera switching condition, the second camera is turned on, and a transition animation is generated based on the first image of the first camera and the second image of the second camera;

[0226] The first camera is switched to the second camera based on a transition animation until the second image is displayed.

[0227] The embodiment of the present application also provides a computer program product.

[0228] In some embodiments, the computer program product may include a computer program or instructions.

[0229] In some embodiments, the computer program product can be applied to the electronic device in the embodiments of the present application, and when the computer program or instructions are run on the electronic device, the electronic device executes the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0230] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0231] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0233] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] It should be noted that each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store program code.

[0235] It should also be noted that in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0236] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0237] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0238] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0239] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0240] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A camera zoom method, characterized in that: The method comprises: In response to a zoom operation of the first camera, determining a second camera to be turned on; When the zoom operation satisfies a camera switching condition, turning on the second camera, and generating a transition animation based on the first image of the first camera and the second image of the second camera; The first camera is switched to the second camera based on the transition animation until the second image is displayed.

2. The method according to claim 1, characterized in that The generating a transition animation based on the first image of the first camera and the second image of the second camera includes: determining at least one first feature point of the first image and at least one second feature point of the second image; Performing feature point matching on the at least one first feature point and the at least one second feature point to determine at least one feature point pair; Aligning the first image and the second image based on the at least one feature point pair to obtain a transition image; The transition animation is generated based on the transition image.

3. The method according to claim 2, characterized in that The aligning the first image and the second image based on the at least one feature point pair to obtain a transition image includes: Based on the at least one feature point pair, the first image and the second image are divided respectively to determine at least one first divided block corresponding to the first image and at least one second divided block corresponding to the second image; Based on a matching relationship between feature points in the first divided block and feature points in the second divided block, locally aligning the first divided block with the second divided block to obtain at least one registration block; Boundary fusion is performed on the at least one registration block to obtain the transition image.

4. The method according to claim 1, characterized in that The step of determining, in response to a zoom operation of the first camera, a second camera to be turned on includes: Determining a zoom mode corresponding to the zoom operation; The second camera to be turned on is determined based on the zoom mode; wherein the zoom mode includes: a sliding zoom mode or a click zoom mode.

5. The method according to claim 4, characterized in that The determining the second camera to be turned on based on the zoom mode includes: When the zoom mode is a sliding zoom mode, determining a transition camera adjacent to the first camera based on the sliding zoom mode, and determining the transition camera as the second camera; When the zoom mode is a click-to-zoom mode, a target magnification is determined based on the click-to-zoom mode, and a target camera corresponding to the target magnification is determined as the second camera.

6. The method according to claim 5, characterized in that When the zoom mode is a sliding zoom mode, the method further includes: Obtain a target camera corresponding to the target magnification; When there is at least one transition camera between the first camera and the target camera, the at least one transition camera is used as the first camera in sequence, and based on the current sliding magnification, the transition camera adjacent to the first camera is used as the second camera until the target camera is turned on.

7. The method according to claim 5, characterized in that When the zoom mode is a click zoom mode, the method further includes: Obtain a target camera corresponding to the target magnification; When there is at least one transition camera between the first camera and the target camera, the at least one transition camera is skipped.

8. The method according to any one of claims 1 to 7, characterized in that When the zoom operation does not satisfy the camera switching condition, the method further includes: Performing a cropping operation on the first image to obtain a cropped image; The cropped image is enlarged to obtain a preview image, and the preview image is displayed.

9. A camera zoom device, characterized in that: The camera zoom device comprises: a determining unit, configured to determine a second camera to be turned on in response to a zoom operation of the first camera; a generating unit, configured to, when the zoom operation satisfies a camera switching condition, turn on the second camera, and generate a transition animation based on a first image of the first camera and a second image of the second camera; A switching unit is used to switch the first camera to the second camera based on the transition animation until the second image is displayed.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores a computer program executable on the processor, and the processor is configured to implement the method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images