Image processing method and device, electronic equipment and readable storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a readable storage medium. The method comprises the following steps: in response to zooming operation, obtaining a current zooming magnification and a target zooming magnification; determining a target camera according to the current zoom ratio and the target zoom ratio; according to at least one of the distance between the current camera and the target camera, the object distance or the outflow speed of the target camera, determining a first number of fused zoom magnifications from the current zoom magnification to the target zoom magnification; obtaining a second number of pre-fusion zoom magnifications from the current zoom magnification to the target zoom magnification, obtaining a target number of transition zoom magnifications based on the first number of fusion zoom magnifications and the second number of pre-fusion zoom magnifications, and processing the output image data according to the target number of transition zoom magnifications, and obtaining target image data. By adopting the method, rapid zooming can be carried out while smooth zooming is realized.

Description

Technical Field

[0001] The present application relates to the field of camera technologies, and in particular, to an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the continuous intelligence of electronic devices, the shooting function has become more and more popular. Many electronic devices are equipped with multiple cameras. Different cameras generally correspond to different focal lengths, and the focal length ranges corresponding to each focal length are different, and the focuses presented by the captured images are also different. Therefore, when using an electronic device to take pictures, zooming can be achieved by switching cameras.

[0003] In the traditional technology, during the process of switching cameras during zooming, in order to achieve smooth zooming and improve the alignment effect, as many image data as possible are often designed to be fused, resulting in a slow camera switching speed. Summary of the Invention

[0004] Embodiments of the present application provide an image processing method, apparatus, electronic device, and computer-readable storage medium, which can achieve smooth zooming while performing fast zooming.

[0005] In a first aspect, the present application provides an image processing method, including:

[0006] Responding to a zoom operation, obtaining a current zoom ratio and a target zoom ratio;

[0007] Determining a target camera according to the current zoom ratio and the target zoom ratio;

[0008] Determining a first quantity of the fusion zoom ratio from the current zoom ratio to the target zoom ratio according to at least one of the distance, object distance between the current camera and the target camera, or the outflow speed of the target camera;

[0009] Obtaining a second quantity of the pre-fusion zoom ratio from the current zoom ratio to the target zoom ratio, obtaining a target quantity of transition zoom ratios based on the fusion zoom ratio of the first quantity and the pre-fusion zoom ratio of the second quantity, and processing the output image data according to the target quantity of transition zoom ratios to obtain target image data.

[0010] In a second aspect, the present application further provides an image processing apparatus, including:

[0011] A zoom ratio acquisition module, configured to respond to a zoom operation and obtain a current zoom ratio and a target zoom ratio;

[0012] A target camera determination module, configured to determine a target camera according to the current zoom ratio and the target zoom ratio;

[0013] A first magnification number determination module, configured to determine a first number of fusion zoom magnifications from the current zoom ratio to the target zoom ratio according to at least one of the distance, object distance between the current camera and the target camera, or the out-flow speed of the target camera;

[0014] A second magnification number determination module, configured to obtain a second number of pre-fusion zoom magnifications from the current zoom ratio to the target zoom ratio, obtain a target number of transition zoom magnifications based on the fusion zoom magnifications of the first number and the pre-fusion zoom magnifications of the second number, and process the output image data according to the target number of transition zoom magnifications to obtain target image data.

[0015] In a third aspect, the present application further provides an electronic device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the image processing method provided in the first aspect are implemented.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image processing method provided in the first aspect are implemented.

[0017] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the image processing method provided in the first aspect are implemented.

[0018] For the above image processing method, device, electronic device, computer-readable storage medium, and computer program product, by determining a target camera according to the current zoom ratio and the target zoom ratio corresponding to a zoom operation, determining a first number of fusion zoom magnifications from the current zoom ratio to the target zoom ratio according to at least one of the distance, object distance between the current camera and the target camera, or the out-flow speed of the target camera, obtaining a target number of transition zoom magnifications based on the fusion zoom magnifications of the first number and the pre-fusion zoom magnifications of the second number from the current zoom ratio to the target zoom ratio, and then processing the output image data according to the target number of transition zoom magnifications to obtain target image data, it is possible to accurately determine the number of fusion zoom magnifications based on at least one of the distance, object distance between the current camera and the target camera, or the out-flow speed of the target camera, and while achieving smooth zooming and ensuring an alignment effect, it is also possible to achieve fast zooming, that is, it is possible to balance the zoom effect and the zoom speed and improve the device usage experience. Description of the Drawings

[0019] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a schematic flowchart of an image processing method in some embodiments;

[0021] Figure 2 It is a schematic flowchart of an image processing method in some other embodiments;

[0022] Figure 3 It is a schematic flowchart for determining the number of zoom ratios in some embodiments;

[0023] Figure 4 It is a schematic flowchart of fallback switching in some embodiments;

[0024] Figure 5 It is a structural block diagram of an image processing device in some embodiments;

[0025] Figure 6 It is an internal structure diagram of an electronic device in some embodiments. Detailed implementation manners

[0026] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] In related electronic devices, there may be 3 - 5 different types of lenses. For example, the operation page of a multi-camera mobile phone camera will display focal length selections such as 0.6x, 1x, 2x, or 5x, 10x. The smaller the number, the wider the corresponding viewing angle range and the farther the object being photographed. It roughly corresponds to the focal lengths of an ultra-wide (UW), wide (W), and telephoto (T) lens.

[0028] Normally, the main camera of most mobile phones is a wide-angle lens with a focal length of about 28 millimeters. Because this focal length is close to the viewing range seen by the human eye, the main camera is the most frequently used camera. The operation page of the mobile phone camera will display the value of 1x. 1x is the focal length corresponding to the main camera, and the image quality is relatively clear, suitable for shooting portraits, buildings, landscapes, and documentaries, etc.

[0029] An ultra-wide-angle lens can provide a wider viewing angle range than a wide-angle lens, that is, it can capture a broader picture than a wide-angle lens, allowing more landscape elements to be included in the frame. It is suitable for shooting landscapes, buildings, etc., and the captured images can obtain an impactful sense of picture. The ultra-wide-angle lens has the characteristic of lens distortion. The objects at the edges of the captured images are stretched and enlarged. This distortion characteristic can be fully utilized for shooting from a low angle upward, creating a "near-big-far-small" visual impact. When shooting buildings, the characteristic of lens distortion can make the buildings look more magnificent.

[0030] Above 1x is usually referred to as telephoto. The larger the number in front of x, the narrower the shooting range, but the farther the shooting distance, and the higher the clarity of the objects in the distance. When shooting objects farther away or magnifying the objects in the frame, the image quality will not degrade like digital zoom. The telephoto lens can "bring closer" the distance between the background and the foreground, thus bringing a sense of compression of spatial distance, making the overall picture more substantial. The telephoto lens has little distortion and weak perspective effect, which can shorten the distance between the foreground and the background, enhance the relationship between the foreground and the background, and create a unique visual effect. Using this feature, straight-extending scenes such as roads and railings can be used as leading lines to draw the viewer's attention back to the main body in the depth of the picture.

[0031] Lenses with different focal lengths have different characteristics. In the camera shooting scheme, there is a SAT (Spatial Alignment Transform) scheme to achieve real-time switching of lenses according to user needs in different shooting scenarios.

[0032] For some typical zoom scenarios, such as the point-cut zoom scenario, the change in the zoom ratio has a certain step size, and it is different in different stages. However, the zoom process performance in actual operation is not smooth enough, especially for the point-cut zoom scenario with a large span (a large difference in zoom ratio). The change trend of the zoom ratio included in the zoom curve is the basis for optimizing zoom smoothness. For the optimization of the field of view (FOV) jump during the zoom process, there may be a situation where a single camera switches to a dual camera during the zoom process. After switching to the dual camera, the image data of the dual camera is fused and displayed, and the fused image data (dual open frame) is the basis for performing SAT fusion. The fused image data refers to the image data displayed according to the fused zoom ratio, which is obtained by fusing the image data acquired by at least two cameras. If the number of fused image data decreases and the zoom ratio corresponding to the first-frame fused image data is relatively large, it will further cause a larger jump in the FOV. Therefore, for the 6x (such as 0.6x to 6x or 1x to 6x) point-cut scenario, it is necessary to adjust the zoom curve. For example, the number of transition frames can be increased (from 15 frames to 18 frames), and the zoom ratio of the first-frame fused image data can be decreased. It can be seen from this that there is a strong correlation between the zoom smoothness during the zoom process and the number and timing of the fused image data.

[0033] In the zoom scheme of traditional technologies, for the alignment design in the zoom or fallback (switchback) scenario, the more the number of fused image data is usually better. However, for the actual alignment and lens switching requirements, it is necessary to balance the relationship between the alignment effect and the switching speed. During the zoom process, if the number of fused image data is too large, it will lead to a slower switching speed, and the long-term streaming of multiple cameras will also result in higher power consumption.

[0034] Aiming at the problem of unreasonable design of the number of fused image data in traditional technologies, the embodiments of the present application provide an image processing method, which accurately determines the number of fused zoom ratios according to at least one of the distance between the current camera and the target camera, the object distance, or the streaming speed of the target camera, so as to achieve smooth zooming, ensure the alignment effect, while improving the switching speed of the camera and the zooming effect.

[0035] The image processing method provided by the embodiments of the present application can be applied to an electronic device. Among them, the electronic device can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The electronic device can be a terminal or a server.

[0036] In some exemplary embodiments, as Figure 1 shown, an image processing method is provided, and the method includes the following steps 102 to 108. Among them:

[0037] Step 102, in response to a zoom operation, obtain the current zoom ratio and the target zoom ratio.

[0038] Among them, the zoom operation refers to an operation of changing the focal length. The zoom operation can refer to an uncertain continuous zoom operation, that is, the user has no clear target zoom ratio during the zoom process, and adjusts the zoom ratio and views the result of the zoomed-in picture to achieve a better preview or recording picture. The zoom operation can also be sliding to a certain target zoom ratio. The zoom operation can be implemented by sliding the zoom control on the camera application interface. The zoom control can be a disk, a long bar, or other forms of zoom controls. In some possible implementation manners, the zoom operation can also be implemented by sliding or rotating a hardware device. For example, a rotatable device is configured on the electronic device, and the zoom ratio is adjusted by rotating the rotatable device.

[0039] Exemplarily, the camera application page of the electronic device can be provided with zoom controls. There can be one or more zoom controls, and different zoom controls correspond to different zoom ratios. The corresponding zoom ratio is zoomed by triggering the zoom control. For example, by clicking to trigger the zoom control, the start and stop points of the zoom operation can be determined by the point-cut zoom control. The current zoom ratio is the starting point, and the target zoom ratio is the stopping point. Or, when the zoom ratios corresponding to the point-cuttable zoom controls are limited, the start and stop points of the zoom operation can also be determined by sliding. In other words, the types of zoom operations can include point-cut zoom operations or slide-cut zoom operations. The zoom operation can also be a zoom operation corresponding to a zoom-in scene or a zoom-out scene. Among them, the zoom-in scene refers to a zoom operation scene where the zoom ratio changes from a small value to a larger value, and the zoom-out scene refers to a zoom operation scene where the zoom ratio changes from a large value to a smaller value.

[0040] Understandably, the current zoom ratio is usually for the current image frame, that is, the current zoom ratio corresponding to the current image frame is obtained. The current image frame refers to the image frame that needs to be previewed, photographed, or recorded currently. Correspondingly, the zoom operation can be a zoom operation in the camera preview scene, the photo-taking scene, or the recording scene. The current zoom ratio refers to the zoom ratio of the current image frame. The target zoom ratio can be the zoom ratio currently issued by the camera application. The camera application will obtain the zoom ratio obtained by the zoom operation in real time or at regular intervals and issue the obtained zoom ratio to the hardware abstraction layer as the target zoom ratio. The target zoom ratio currently issued by the camera application refers to the target zoom ratio obtained by the camera application for the current user sliding operation. Exemplarily, in response to a zoom operation generated by the user sliding a disk, or a zoom operation generated by the user rotating a hardware device, or a zoom operation generated by tapping a zoom control, the electronic device can obtain the current zoom ratio of the current image frame and the target zoom ratio currently issued by the camera application. The target zoom ratio is also the zoom ratio to which the zoom operation needs to be changed.

[0041] Step 104, determine the target camera according to the current zoom ratio and the target zoom ratio.

[0042] The target camera refers to the camera other than the current camera among the cameras involved in the zoom operation. The target camera can include one or more. For example, if the current zoom ratio is 3x, the target zoom ratio is 0.5x, the current camera corresponding to the current zoom ratio is the telephoto camera, and the cameras involved in the process of changing the zoom ratio from 3x to 0.5x can include the telephoto camera, the wide-angle camera, and the ultra-wide-angle camera, then, except for the telephoto camera, the wide-angle camera and the ultra-wide-angle camera are both target cameras. The current camera is the camera corresponding to the current zoom ratio.

[0043] Understandably, an electronic device may include one or more cameras. If it includes one camera, its zoom range is limited. When it includes multiple cameras, the zoom range is wider. The electronic device may use one of the cameras as the main camera and the other cameras as secondary cameras. The main camera refers to the primary camera in the electronic device that is used to capture most of the image data. Exemplarily, the cameras of the electronic device may include at least one of an ultra-wide-angle camera, a wide-angle camera, a telephoto camera, and an ultra-telephoto camera (UT). The zoom magnification at which the camera switches is generally the optical zoom point. Different cameras may correspond to different image sensors, that is, different cameras perform imaging through different image sensors. Different cameras have different zoom magnification ranges. To achieve different zoom magnifications, different cameras may be required. When a certain zoom magnification needs to be achieved using a certain camera, the image sensor corresponding to that camera needs to be called, and the image sensor is controlled to turn on and collect image data.

[0044] Exemplarily, the camera to be called may be determined based on the current zoom magnification and the target zoom magnification, and then the camera other than the current camera among the cameras to be called is used as the target camera. Alternatively, candidate optical zoom points between the current zoom magnification and the target zoom magnification may be determined, and the optical zoom points other than the optical zoom point corresponding to the current camera among the candidate optical zoom points are used as the target optical zoom points, and the target camera is determined based on the target optical zoom points.

[0045] Step 106: Determine a first quantity of the fusion zoom magnification from the current zoom magnification to the target zoom magnification based on at least one of the distance between the current camera and the target camera, the object distance, or the out-flow speed of the target camera.

[0046] Among them, the distance between the current camera and the target camera refers to the physical distance between the positions of the current camera and the target camera on the electronic device. Understandably, the distance between the current camera and the target camera is related to the hardware layout of the electronic device. In electronic devices produced by different manufacturers, the distance between the current camera and the target camera is generally different. The distance between the current camera and the target camera can be characterized by the straight-line distance between the current camera and the target camera on the electronic device.

[0047] The object distance refers to the distance between the photographed object and the camera. For example, the object distance can be obtained in real time through the AF (Automatic Focus) module in the electronic device. The AF module is a module used to automatically adjust the focal length. The AF module can determine the object distance by means of phase detection autofocus (PDAF, Phase Detection Auto Focus), laser, or contrast detection autofocus (CDAF, Contrast Detection Auto Focus), etc.

[0048] The output speed of the target camera refers to the speed at which the target camera outputs the image data stream from the start. Understandably, the output speeds of different cameras are generally different. The output speed of the camera is related to, for example, the readout speed of the corresponding image sensor, the sensor resolution, and the software settings.

[0049] The fusion zoom ratio refers to the zoom ratio corresponding to the fused image frame. The fusion zoom ratio is usually between the current zoom ratio and the target zoom ratio. If the target zoom ratio is the optical zoom point, then the target zoom ratio is the fusion zoom ratio. Understandably, during the zooming process, before switching from the current camera to the target camera, the current camera and the target camera will output image data streams simultaneously, and control the image data output by the target camera to be aligned and fused with the image data output by the current camera to obtain the fused image data, and display the fused image data according to the set fusion zoom ratio to reduce the problem of large jumps in the field of view angle during the zooming process. The displayed fused image data is the fused image frame, and the fused image frame usually includes multiple frames.

[0050] In the actual application scenario, the first quantity of the fusion zoom ratio from the current zoom ratio to the target zoom ratio can be determined according to one, two, or three of the distance between the current camera and the target camera, the object distance, or the output speed of the target camera. For example, the first quantity of the fusion zoom ratio from the current camera to the target zoom ratio can be determined according to the distance between the current camera and the target camera, the object distance, or the output speed of the target camera. Or, the first quantity of the fusion zoom ratio from the current camera to the target zoom ratio can be determined according to the distance between the current camera and the target camera and the object distance. Or, the first quantity of the fusion zoom ratio from the current camera to the target zoom ratio can be determined according to the distance between the current camera and the target camera and the output speed of the target camera. Or, the first quantity of the fusion zoom ratio from the current camera to the target zoom ratio can be determined according to the object distance and the output speed of the target camera. Or, the first quantity of the fusion zoom ratio from the current camera to the target zoom ratio can be determined according to the distance between the current camera and the target camera, the object distance, and the output speed of the target camera.

[0051] Exemplarily, the first quantity of the fusion zoom ratio from the current zoom ratio to the target zoom ratio can be determined according to at least one of the distance between the current camera and the target camera, the object distance, the outflow speed of the target camera, the zoom amplitude between the current zoom ratio and the target zoom ratio, or the shooting mode. The first quantity refers to the quantity of the fusion zoom ratio. Wherein, the zoom amplitude is used to characterize the change amplitude between the current zoom ratio and the target zoom ratio. The shooting mode includes the photo mode and the video mode.

[0052] Step 108, obtain the second quantity of the pre-fusion zoom ratio from the current zoom ratio to the target zoom ratio, obtain the target quantity of the transition zoom ratio based on the fusion zoom ratio of the first quantity and the pre-fusion zoom ratio of the second quantity, and process the output image data according to the target quantity of the transition zoom ratio to obtain the target image data.

[0053] Wherein, the pre-fusion zoom ratio refers to the zoom ratio before the first fusion zoom ratio from the current zoom ratio to the target zoom ratio. The transition zoom ratio refers to the zoom ratio that changes from the current zoom ratio to the target zoom ratio. The transition zoom ratio includes the fusion zoom ratio and the pre-fusion zoom ratio.

[0054] In an exemplary embodiment, the fusion zoom ratio of the first quantity and the pre-fusion zoom ratio of the second quantity can be sorted in the direction from the current zoom ratio to the target zoom ratio to obtain the transition zoom ratio. It is easy to understand that if the zoom operation belongs to the operation of magnifying the ratio, the direction from the current zoom ratio to the target zoom ratio is the direction of increasing the ratio, and if the zoom operation belongs to the operation of reducing the ratio, the direction from the current zoom ratio to the target zoom ratio is the direction of reducing the ratio.

[0055] In an exemplary embodiment, if the target zoom ratio does not belong to the optical zoom point, the fusion zoom ratio of the first quantity, the pre-fusion zoom ratio of the second quantity, and the post-fusion zoom ratio of the third quantity can be sorted in the direction from the current zoom ratio to the target zoom ratio to obtain the transition zoom ratio. Wherein, the post-fusion zoom ratio refers to the zoom ratio of the image frame obtained by the target camera displayed after switching to the target camera. For example, assume that the current zoom ratio is 1X and the target zoom ratio is 3.5X. Then, if the target camera is a telephoto camera and the optical zoom point of the telephoto camera is 3X, when the zoom ratio reaches 3X, it switches to the telephoto camera, and the image data displayed after switching to the telephoto camera is the image displayed by the telephoto camera according to the post-fusion zoom ratio.

[0056] It should be noted that, usually, each transitional zoom ratio corresponds to one frame of image data, that is, there is a time duration corresponding to one frame of image data between two adjacent transitional zoom ratios. Then, for the transitional zoom ratios including the target number, the zoom operation is performed through the time duration corresponding to the target number of frames of image data. An electronic device usually obtains the initial image data corresponding to the optical variable point of the camera, and then performs a cropping process on the initial image data based on the zoom ratio to obtain the image data corresponding to the corresponding zoom ratio and display it. After determining the transitional zoom ratio corresponding to each frame of image data, the image data output by the camera can be sequentially cropped according to the transitional zoom ratio to obtain the target image data corresponding to each transitional zoom ratio.

[0057] Exemplarily, when determining the number of transitional zoom ratios, the number of pre-fusion zoom ratios, the number of fusion zoom ratios, or the number of post-fusion zoom ratios, and the current zoom ratio and the target zoom ratio, the magnification value of each transitional zoom ratio can be determined according to the linear distribution or non-linear distribution of the transitional zoom ratios. Among them, the linear distribution can be, for example, evenly spaced between adjacent transitional zoom ratios, and the non-linear distribution can be, for example, logarithmically or exponentially spaced between adjacent transitional zoom ratios. In an exemplary embodiment, the magnification value of the transitional zoom ratio can be determined by the principle of Bezier curve, as shown in the following formula (1).

[0058] Formula (1)

[0059] Wherein, p0 represents the current zoom ratio; p3 represents the target zoom ratio; p1 and p2 represent control points, which are used to determine the shape of the zoom curve formed by the transitional zoom ratios. If p1 and p2 are between p0 and p3, the curve will be relatively smooth. If p1 is close to p0 and p2 is close to p3, the zoom speed will be slow first and then fast. t represents time, and the zoom ratio M changes with the change of time.

[0060] Assume that the number of transitional zoom ratios is N, that is, the transitional zoom ratios are calculated within N steps. Then, the value of t is as shown in the following formula (2).

[0061] Formula (2)

[0062] Substitute each t k into formula (1), and the magnification value M(t k ) of each transitional zoom ratio can be calculated.

[0063] In the above image processing method, by determining a target camera according to the current zoom ratio and the target zoom ratio corresponding to a zoom operation, and determining a first quantity of fusion zoom ratios from the current zoom ratio to the target zoom ratio according to at least one of the distance, object distance, or out-flow speed of the target camera between the current camera and the target camera, obtaining a target quantity of transition zoom ratios according to the first quantity of fusion zoom ratios and a second quantity of pre-fusion zoom ratios from the current zoom ratio to the target zoom ratio, and then processing the output image data according to the target quantity of transition zoom ratios to obtain target image data, it is possible to accurately determine the quantity of fusion zoom ratios based on at least one of the distance, object distance, or out-flow speed of the target camera between the current camera and the target camera, achieve smooth zooming while ensuring an alignment effect, and also achieve fast zooming, that is, it can balance the zoom effect and the zoom speed, improving the device usage experience.

[0064] In some embodiments, the distance between the current camera and the target camera has a positive correlation with the quantity of fusion zoom ratios.

[0065] It is easy to understand that in electronic devices produced by different manufacturers, the positional relationship between the current camera and the target camera is usually different, and correspondingly, the distance between the current camera and the target camera is also different. If the distance between the current camera and the target camera is larger, the more image frames are required to achieve a good alignment effect during the zooming process, and it is more difficult to align, that is, the corresponding quantity of fusion zoom ratios is also larger. If the distance between the current camera and the target camera is smaller, the fewer image frames are required to achieve a good alignment effect during the zooming process, and it is easier to align, that is, the corresponding quantity of fusion zoom ratios is also smaller.

[0066] Exemplarily, the quantity of fusion zoom ratios can be determined according to the distance between the current camera and the target camera. The farther the distance between the current camera and the target camera, the more the quantity of fusion zoom ratios determined; the closer the distance between the current camera and the target camera, the fewer the quantity of fusion zoom ratios determined.

[0067] In this embodiment, the distance between the current camera and the target camera has a positive correlation with the quantity of fusion zoom ratios, and the quantity of fusion zoom ratios can be accurately obtained according to the distance between the current camera and the target camera, enabling the compatibility of zoom smoothness and zoom speed during the zooming process.

[0068] In some embodiments, the object distance has a negative correlation with the quantity of fusion zoom ratios.

[0069] Understandably, the object distance refers to the distance between the camera and the object being photographed. The greater the object distance, the relatively smaller the influence of the distance between the current camera and the target camera, the smaller the difference in the field of view angle caused by the distance between the current camera and the target camera, and the relatively easier it is to align, that is, the fewer the number of corresponding fusion zoom ratios. Conversely, the smaller the object distance, the relatively greater the influence of the distance between the current camera and the target camera, the greater the difference in the field of view angle caused by the distance between the current camera and the target camera, and the relatively more difficult it is to align, that is, the more the number of corresponding fusion zoom ratios.

[0070] Exemplarily, the number of fusion zoom ratios can be determined according to the object distance. The greater the object distance, the fewer the number of fusion zoom ratios; the smaller the object distance, the more the number of fusion zoom ratios.

[0071] In this embodiment, the object distance and the number of fusion zoom ratios are in a negative correlation relationship. According to the object distance, the number of fusion zoom ratios can be accurately determined, enabling the compatibility of zoom smoothness and zoom speed during the zoom process.

[0072] In some embodiments, the output speed of the target camera is in a negative correlation relationship with the number of fusion zoom ratios.

[0073] Understandably, the faster the output speed of the target camera, that is, the faster the image data stream can be output after controlling the target camera to start, the faster the fusion of the image data obtained by the current camera can be achieved to obtain the fused image data. Then, the fewer the number of pre-fusion zoom ratios can be. When the number of transitional zoom ratios is fixed, that is, when the total zoom duration is fixed, more fusion zoom ratios can be set. Conversely, the slower the output speed of the target camera, that is, the slower the speed of outputting the image data stream after controlling the target camera to start, the longer the waiting time is required to fuse the image data obtained by the current camera to obtain the fused image data. Then, the more the number of pre-fusion zoom ratios is. When the number of transitional zoom ratios is fixed, only fewer fusion zoom ratios can be set.

[0074] Exemplarily, according to the output speed of the target camera, the number of fusion zoom ratios can be determined. The faster the output speed of the target camera, the more the corresponding number of fusion zoom ratios can be; the slower the output speed of the target camera, the fewer the corresponding number of fusion zoom ratios can be.

[0075] In this embodiment, the output speed of the target camera is in a negative correlation relationship with the number of fusion zoom ratios. According to the output speed of the target camera, the number of fusion zoom ratios can be accurately determined, enabling the compatibility of zoom smoothness and zoom speed during the zoom process.

[0076] In some embodiments, the above method further includes:

[0077] Determine the zoom range according to the current zoom ratio and the target zoom ratio; determine the number of fusion zoom ratios from the correspondence between the zoom range and the number of fusion zoom ratios; wherein, the zoom range and the number of fusion zoom ratios are in a positive correlation.

[0078] Wherein, the zoom range is used to represent the distance between the current zoom ratio and the target zoom ratio. The zoom range from the current zoom ratio to the target zoom ratio can be represented by the ratio between the larger magnification value and the smaller magnification value of the current zoom ratio and the target zoom ratio.

[0079] Exemplarily, a correspondence between the zoom range and the number of fusion zoom ratios can be preset in advance, and according to this correspondence, determine the number of fusion zoom ratios corresponding to the zoom range from the current zoom ratio to the target zoom ratio. Among them, the larger the zoom range, the larger the number of fusion zoom ratios, and the smaller the zoom range, the smaller the number of fusion zoom ratios. It is easy to understand that the larger the zoom range, the larger the span of the magnification value from the current zoom ratio to the target zoom ratio, and more fusion zoom ratios are required to achieve smooth zoom. And the smaller the zoom range, the smaller the span of the magnification value from the current zoom ratio to the target zoom ratio, and fewer fusion zoom ratios can achieve smooth zoom.

[0080] In an exemplary embodiment, the first number of fusion zoom ratios can be determined according to at least one of the distance between the current camera and the target camera, the object distance, or the out-flow speed of the target camera, and the zoom range from the current zoom ratio to the target zoom ratio.

[0081] In this embodiment, by determining the zoom range from the current zoom ratio to the target zoom ratio, and combining the zoom range, the number of fusion zoom ratios can be further accurately determined, and the requirements of zoom smoothness and zoom speed can be compatible during the zoom process.

[0082] In some embodiments, the above method further includes:

[0083] Obtain the shooting mode; determine the number of fusion zoom ratios from the correspondence between the shooting mode and the number of fusion zoom ratios.

[0084] Among them, the shooting mode refers to the mode in which an electronic device captures images. The shooting mode can include, for example, the photo mode and the video mode. The photo mode can also be called the taking photo mode, which refers to the shooting mode for recording a static image at a certain moment. The video mode can also be called the recording mode, which refers to the shooting mode for continuously recording static images within a certain period of time. The number of hybrid zoom ratios corresponding to different shooting modes is different. The corresponding relationship between the shooting mode and the number of hybrid zoom ratios can be preset. After determining the shooting mode, according to the corresponding relationship between the shooting mode and the number of hybrid zoom ratios, the number of hybrid zoom ratios corresponding to the zoom operation is determined.

[0085] Exemplarily, in the zoom operation with the same zoom range, the number of hybrid zoom ratios in the photo mode is usually more than that in the video mode. For example, if the current zoom ratio is 0.5x and the target zoom ratio is 3x, the zoom range from the current zoom ratio to the target zoom ratio is 6. If the shooting mode is the photo mode, the number of hybrid zoom ratios corresponding to the zoom range can be 12, 13, 14 or 15. If the shooting mode is the video mode, the number of hybrid zoom ratios corresponding to the zoom range can be 7, 8, 9 or 10. Or, in some application scenarios, during the process of camera switching caused by zooming in the video mode, no hybrid processing needs to be performed, and the camera directly switches when it reaches the optical zoom point. This is because in the video mode, the captured images are played at a certain frame rate, and the requirement for zoom smoothness during the zoom process is not as high as that in the photo mode. The change range of the transitional zoom ratios during the zoom process can be greater than that in the photo mode.

[0086] In this embodiment, by determining the number of hybrid zoom ratios from the current zoom ratio to the target zoom ratio according to the corresponding relationship between the shooting mode and the number of hybrid zoom ratios, the accurate determination of the hybrid zoom ratios in different shooting modes can be achieved. While achieving smooth zooming in the corresponding shooting mode, fast zooming can also be realized.

[0087] In some embodiments, determining the target camera according to the current zoom ratio and the target zoom ratio includes:

[0088] Determining the zoom direction and the target optical zoom point according to the current zoom ratio and the target zoom ratio; determining the target camera according to the zoom direction and the target optical zoom point.

[0089] Among them, the target optical zoom point refers to the optical zoom point between the current zoom ratio and the target zoom ratio. The zoom direction and the target optical zoom points to be passed through can be determined according to the current zoom ratio and the target zoom ratio. The zoom direction includes the magnification direction and the reduction direction. If the target zoom ratio is less than the current zoom ratio, the zoom direction is the reduction direction; if the target zoom ratio is greater than the current zoom ratio, the zoom direction is the magnification direction.

[0090] Exemplarily, the magnification range formed by the current zoom ratio and the target zoom ratio can be compared with the optical zoom points in sequence, and the optical zoom points included in the magnification range are used as the target optical zoom points. In the corresponding zoom direction, the camera corresponding to the target optical zoom point is determined as the target camera. For example, if the current zoom ratio is 3.5x, the target zoom ratio is 0.5x, and the zoom direction is the reduction direction, the target optical zoom points to be passed through in the zoom direction include 3x and 1x, then the target cameras include the wide-angle camera corresponding to 3x and the ultra-wide-angle camera corresponding to 1x.

[0091] In this embodiment, by determining the zoom direction and the target optical zoom points according to the current zoom ratio and the target zoom ratio, and determining the target camera according to the zoom direction and the target optical zoom points, a basis is laid for accurately determining the number of fusion zoom ratios.

[0092] In some embodiments, the above method further includes:

[0093] After switching to the first camera corresponding to the target zoom ratio, user operation data is acquired; according to the user operation data, the switch state of the second camera before switching to the first camera is determined; the second camera is controlled according to the switch state of the second camera.

[0094] Among them, the user operation data refers to the data used to characterize the user's zoom operation habits. For example, according to the user operation data, it can be determined whether the user has the habit of continuous zoom operation. If the user has the habit of continuous zoom operation, it can be determined that the switch state of the second camera before switching to the first camera is on. If the user does not have the habit of continuous zoom operation, it can be determined that the switch state of the second camera before switching to the first camera is off.

[0095] In an exemplary embodiment, user operation data is obtained after the user switches to the first camera corresponding to the target zoom ratio, and the user operation data is analyzed. If the user operation data indicates that the user has a habit of continuous zoom operation, it is determined that the switch state of the second camera before switching to the first camera is the on state, and the second camera is controlled to turn on to respond to the continuous zoom operation in a timely manner. If the user operation data indicates that the user does not have a habit of continuous zoom operation, it is determined that the switch state of the second camera before switching to the first camera is the off state, and the second camera is controlled to turn off. It should be noted that multiple pieces of user operation data after the camera switch corresponding to the first zoom operation can be statistically analyzed, and based on the statistically analyzed user operation data, it is determined whether the user has a habit of continuous zoom operation.

[0096] In this embodiment, by determining the switch state of the second camera before switching to the first camera according to the user operation data obtained after switching to the first camera corresponding to the target zoom ratio, and controlling the second camera according to the switch state of the second camera, it is possible to determine whether to turn off the second camera after zooming to the first camera according to the user operation data. If it is determined to turn off the second camera, the device power consumption can be saved. If it is determined to turn on the second camera, subsequent zooming can be responded to in a timely manner, improving the zoom experience.

[0097] In some embodiments, the above method further includes:

[0098] Determine a fallback camera according to the object distance; when the object distance is less than the pre - opening distance threshold, control the fallback camera to turn on; when the object distance is less than or equal to the fusion distance threshold, fuse the first image data obtained by the current camera and the second image data obtained by the fallback camera to obtain first target image data; wherein, the difference between the pre - opening distance threshold and the fusion distance threshold has a negative correlation with the object distance; the pre - opening distance threshold is greater than the fusion distance threshold; after displaying a preset number of first target image data, switch to displaying the second target image data obtained by the fallback camera according to the current zoom ratio; wherein, the preset number has a negative correlation with the object distance.

[0099] Among them, the fallback camera refers to the camera that triggers the fallback scenario. That is, in the case of triggering the fallback scenario, the current camera will be switched to the fallback camera. The fallback scenario is the fallback scenario, and fallback is a mechanism for switching cameras according to the object distance or ambient brightness. For example, if the current camera is a telephoto camera, when the object distance of the shooting scene is less than the distance threshold or the ambient brightness is less than the brightness threshold, in order to obtain a clearer image, the fallback mechanism will be automatically triggered, and thus it will be automatically switched to the wide-angle camera. The wide-angle camera is used to collect image data and display it according to the same field of view (FOV). At this time, the wide-angle camera is the fallback camera. The current zoom ratio refers to the zoom ratio of the current image frame obtained by the current camera. The preset quantity refers to the quantity of the first target image data obtained by fusion, that is, the quantity of the fusion zoom ratio in the corresponding scenario.

[0100] In the actual shooting scenario, the distance between the camera and the shooting object (i.e., the object distance) will constantly change. When the object distance changes from large to small, it is easy to trigger the fallback scenario. When the object distance is less than the pre-opening distance threshold of the fallback camera, the fallback camera is controlled to be turned on. If the object distance continues to change and is less than or equal to the fusion distance threshold, the first image data obtained by the current camera is fused with the second image data obtained by the fallback camera to obtain the first target image data. After obtaining the preset quantity of the first target image data, it is switched to display the second target image data obtained by the fallback camera according to the current zoom ratio. Among them, the pre-opening distance threshold refers to the distance threshold for controlling the opening of the fallback camera, and the fusion distance threshold refers to the distance threshold for fusing the image data obtained by the current camera with the image data obtained by the fallback camera, which can also be called the switching distance threshold. Since the camera needs to be turned on first for fusion, the pre-opening distance threshold is greater than the fusion distance threshold.

[0101] Understandably, the pre - opening distance thresholds and fusion distance thresholds for different cameras corresponding to the trigger fallback scenarios are all different. The trigger fallback scenarios include entering the fallback scenario (fallback in) or exiting the fallback scenario (fallback out). The difference between the pre - opening distance threshold and the fusion distance threshold of the same camera is negatively correlated with the object distance, that is, the larger the object distance, the smaller the difference between the pre - opening distance threshold and the fusion distance threshold; the smaller the object distance, the larger the difference between the pre - opening distance threshold and the fusion distance threshold. In other words, if the object distance is larger, the opening timing of the fallback camera corresponding to the trigger fallback scenario is later; if the object distance is smaller, the opening timing of the fallback camera corresponding to the trigger fallback scenario is earlier. Among them, the larger the object distance, that is, the farther the distance between the camera and the shooting object, the fewer the number of fusion zoom ratios required to achieve alignment, so the fallback camera can be opened later; the smaller the object distance, that is, the closer the distance between the camera and the shooting object, the more the number of fusion zoom ratios required to achieve alignment, so the fallback camera needs to be opened earlier to leave sufficient time for alignment and fusion.

[0102] In this embodiment, by controlling the difference between the pre - opening distance threshold and the fusion distance threshold for opening the fallback camera to be negatively correlated with the object distance in the case of triggering the fallback scenario, it is possible to open the fallback camera later in the scenario with a larger object distance, rather than opening the fallback camera earlier in the scenario with a smaller object distance, which can reduce the opening power consumption of the fallback camera.

[0103] In some embodiments, the zoom curve includes a first curve segment corresponding to the current zoom ratio to the intermediate zoom ratio, and a second curve segment corresponding to the intermediate zoom ratio to the target zoom ratio; the change trend of the second curve segment is flatter than that of the first curve segment, and the intermediate zoom ratio is located between the current zoom ratio and the target zoom ratio; the above - mentioned method further includes:

[0104] Fuse the third image data obtained by the current camera and the fourth image data obtained by the target camera to obtain the third target image data, and sequentially display the third target image data according to the fusion zoom ratios in the first curve segment; after displaying the third target image data according to the fusion zoom ratios in the first curve segment, switch to displaying the fourth target image data obtained by the target camera according to the zoom ratios corresponding to the second curve segment.

[0105] Among them, the zoom curve is used to characterize the change trend of multiple transition zoom ratios from the current zoom ratio to the target zoom ratio. The zoom curve includes a first curve segment and a second curve segment. The first curve segment includes the transition from the current zoom ratio to the intermediate zoom ratio, and the second curve segment includes the transition from the intermediate zoom ratio to the target zoom ratio. The intermediate zoom ratio is a zoom ratio located between the current zoom ratio and the target zoom ratio. Exemplarily, the intermediate zoom ratio may be the optical zoom point of the target camera, that is, the target zoom ratio may not be the optical zoom point. It is easy to understand that the optical zoom point may be the optical zoom point adjusted based on the initial optical zoom point, or it may be the initial optical zoom point. The initial optical zoom point refers to the commonly used optical zoom point, and the adjusted optical zoom point can be obtained by magnifying or reducing based on the initial optical zoom point.

[0106] In an actual application scenario, the first curve segment includes the pre-fusion zoom ratio and the fusion zoom ratio. During the process of triggering zoom, the third image data acquired by the current camera is fused with the fourth image data acquired by the target camera to obtain the third target image data, and the third target image data is sequentially displayed according to the fusion zoom ratio in the first curve segment. After the display of the third target image data is completed, the display is switched to the fourth target image data acquired by the target camera according to the zoom ratio of the second curve segment. Since the change trend of the second curve segment is flatter than that of the first curve segment, it indicates that the second curve segment is the "slow-out" segment of the zoom curve, that is, after switching the current camera to the target camera, a "slow-out" change of the zoom ratio is performed, which can cover the fusion box generated during the fusion process through the field of view jump caused by the camera switch, reduce the fusion trace of the fused image data, and improve the integrity of the fused image data.

[0107] In this embodiment, by fusing the third image data acquired by the current camera and the fourth image data acquired by the target camera to obtain the third target image data, and sequentially displaying the third target image data according to the fusion zoom ratio in the first curve segment, after displaying the third target image data according to the fusion zoom ratio in the first curve segment, the display is switched to the fourth image data acquired by the target camera according to the zoom ratio corresponding to the second curve segment, that is, first fuse and then switch the camera, which can realize displaying the image data acquired by the target camera according to the zoom ratio of the relatively flat second curve segment after the camera switch, and cover the fusion trace through the field of view jump caused by the camera switch, thereby improving the integrity and naturalness of the fused image data and enhancing the zoom experience.

[0108] In some exemplary embodiments, the flowchart of the image processing method is as Figure 2As shown. In an actual shooting scenario, when the camera application is opened, the camera can be used to take pictures, preview, or record the shooting object. It is recognized whether a fallback scenario is triggered, and triggering a fallback scenario includes entering a fallback scenario (fallback in) or exiting a fallback scenario (fallback out). If a fallback scenario is not triggered, it is recognized whether zooming is triggered. If zooming is triggered, the current zoom ratio (i.e., the zoom start point) and the target zoom ratio (the zoom end point) are determined according to the zoom event issued by the camera application (Application, APP). According to the current zoom ratio and the target zoom ratio, the target camera is determined, and according to the zoom range between the current zoom ratio and the target zoom ratio, the distance between the current camera and the target camera (the relative distance of the cameras), the object distance obtained by the autofocus module, and the outflow speed of the target camera, the first quantity of the fusion zoom ratio from the current zoom ratio to the target zoom ratio is determined. The second quantity of the pre-fusion zoom ratio can be determined according to the followability requirement. Among them, the pre-fusion zoom ratio corresponds to the pre-fusion image data (i.e., the pre-taken frame), and the pre-fusion image data can be the image data before cropping and sending for display, and can be obtained from the cache or from the image processing pipeline in the image processing process. The higher the followability requirement, the more the quantity of the pre-fusion zoom ratio; the lower the followability requirement, the less the quantity of the pre-fusion zoom ratio. Among them, followability is used to characterize the timeliness of the zoom response. The faster the zoom response speed and the more timely the response, the better the followability. Conversely, the slower the response speed, the worse the followability. According to the first quantity of the fusion zoom ratio and the second quantity of the pre-fusion zoom ratio, the target quantity of the transition zoom ratio (i.e., corresponding to Figure 2 the transition frame in

[0109] is obtained, and the magnification value of each transition zoom ratio is calculated. The target zoom curve corresponding to the zoom operation can be obtained, and the output image data can be processed according to this target zoom curve to obtain the target image data. The target quantity can be the sum of the first quantity and the second quantity. In some application scenarios, if the requirement for the fusion effect is not high, for example, in the zooming process with a small zoom range, such as switching from the current zoom ratio of 0.6x to the target zoom ratio of 1x or 2x, the quantity of the pre-fusion zoom ratio can be increased. For example, more frames of pre-fusion image data can be obtained from the cache. In other words, when the quantity of the transition zoom ratio is fixed, the followability effect can be improved by a larger quantity of the pre-fusion zoom ratio. Figure 3As shown. Among them, the fusion zoom ratio corresponds to the fusion frame, and one fusion frame image corresponds to one fusion zoom ratio. The transition zoom ratio corresponds to the transition frame, and one transition frame corresponds to one transition zoom ratio. The pre-taken frame corresponds to the pre-fusion zoom ratio, and one pre-taken frame corresponds to one pre-fusion zoom ratio.

[0110] During the process of responding to zooming, that is, according to the transition zoom ratio on the target zoom curve, the field of view of the output image data is cropped in sequence, and the cropped image is displayed. For the fusion frame corresponding to the fusion zoom ratio, it is the fused image data obtained by aligning and fusing the image data acquired by the current camera and the target camera. The fused image data is cropped by the fusion zoom ratio, and the cropped fused image data is displayed. After the image data corresponding to the target zoom ratio is displayed, it indicates that the zooming is completed. After the zooming is completed, for example, after the camera is switched to the target camera, according to the user operation data, it is determined whether the switch state of the camera (sub-camera) before the switch is turned on. At this time, the target camera is the main camera, and the camera before the switch is controlled according to the determined switch state to continue resuming photographing, previewing or recording.

[0111] In addition, during the shooting process, the distance between the electronic device and the shooting object may change continuously, or the brightness of the shooting environment may also change continuously. If the distance between the electronic device and the shooting object or the brightness of the shooting environment reaches the threshold for triggering the fallback scenario, the fallback scenario is triggered. If it is recognized that the fallback scenario is triggered, when the object distance is less than the pre-opening distance threshold, the fallback camera is controlled to be turned on, that is, the pre-pulling of the sub-camera is triggered. When the object distance is less than or equal to the fusion distance threshold, the first image data acquired by the current camera is fused with the second image data acquired by the fallback camera to obtain the first target image data. After the fusion process is completed, the current camera is switched to the target camera to complete the fallback switch. Among them, the difference between the pre-opening distance threshold and the fusion distance threshold has a negative correlation with the object distance, or rather, the difference between the pre-opening distance threshold and the fusion distance threshold has a negative correlation with the focusing distance, that is, the pre-opening distance threshold and the fusion distance threshold can be configured according to the focusing distance of the camera.

[0112] In an actual application scenario, the flow diagram of the fallback switch is as Figure 4As shown. During the process of taking pictures, previewing, or recording, the electronic device displays the image data acquired by the current camera. If the distance between the current camera and the shooting object is less than the pre - opening distance threshold (i.e., the pre - pulling threshold) corresponding to the target camera, the target camera is controlled to be turned on. At this time, the current camera can be called the main camera, and the target camera can be called the secondary camera. It usually takes a duration corresponding to 3 - 4 frames of image data for the target camera to output an image data stream after it is turned on. After the target camera outputs the image data, the image data output by the target camera can be aligned with the image data output by the current camera. Figure 4 As shown in Figure 4 , the alignment is performed at the 6th frame after the target camera is turned on. At this time, both the current camera and the target camera output image data, which is also called double - opening, and the corresponding image data is the double - opening frame. As the object distance changes, if the distance between the current camera and the shooting object is less than or equal to the fusion distance threshold (i.e., the fallback switching threshold) corresponding to the target camera, the image data output by the target camera is aligned and fused with the image data output by the current camera to obtain the first target image data. Figure 4 As shown in Figure 4 , the alignment and fusion are performed at the 9th frame after the target camera is turned on. After displaying a certain number of the first target image data, the display is switched to the second target image data acquired by the target camera, that is, the fallback switching is completed.

[0113] In the above - mentioned embodiments, based on the zoom amplitude between the current zoom ratio and the target zoom ratio, the distance between the current camera and the target camera, the object distance, or the outflow speed of the target camera, the number of fusion zoom ratios can be accurately determined. Based on the determination strategy of the fusion zoom ratio, it is possible to achieve smooth and fast zooming on the basis of timely responding to the zoom, that is, it can balance the zoom effect and the zoom speed. In addition, after the zooming is completed, determining whether to turn off the camera before the switch based on the user operation data can reduce the device power consumption and improve the device zooming experience.

[0114] It should be understood that although the steps in the flowcharts involved in the above - mentioned embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above - mentioned embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0115] Based on the same inventive concept, an embodiment of the present application further provides an image processing apparatus for implementing the above-mentioned image processing method. The solution provided by this apparatus for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following image processing apparatuses can refer to the limitations on the image processing method in the above text, and will not be repeated here.

[0116] In an exemplary embodiment, as Figure 5 shown, an image processing apparatus 500 is provided, including: a zoom ratio acquisition module 502, a target camera determination module 504, a first ratio quantity determination module 506, and a second ratio quantity determination module 508, where:

[0117] The zoom ratio acquisition module 502 is configured to acquire the current zoom ratio and the target zoom ratio in response to a zoom operation;

[0118] The target camera determination module 504 is configured to determine a target camera according to the current zoom ratio and the target zoom ratio;

[0119] The first ratio quantity determination module 506 is configured to determine a first quantity of the fusion zoom ratio from the current zoom ratio to the target zoom ratio according to at least one of the distance between the current camera and the target camera, the object distance, or the outflow speed of the target camera;

[0120] The second ratio quantity determination module 508 is configured to acquire a second quantity of the pre-fusion zoom ratio from the current zoom ratio to the target zoom ratio, obtain a target quantity of the transition zoom ratio based on the fusion zoom ratio of the first quantity and the pre-fusion zoom ratio of the second quantity, and process the output image data according to the target quantity of the transition zoom ratio to obtain target image data.

[0121] In some embodiments, the first ratio quantity determination module 506 is further configured to determine a zoom range according to the current zoom ratio and the target zoom ratio; determine the quantity of the fusion zoom ratio from the corresponding relationship between the zoom range and the quantity of the fusion zoom ratio; the zoom range and the quantity of the fusion zoom ratio have a positive correlation.

[0122] In some embodiments, the first ratio quantity determination module 506 is further configured to acquire a shooting mode; determine the quantity of the fusion zoom ratio from the corresponding relationship between the shooting mode and the quantity of the fusion zoom ratio.

[0123] In some embodiments, the target camera determination module 504 is further configured to determine a zoom direction and a target optical variable point according to the current zoom ratio and the target zoom ratio; determine the target camera according to the zoom direction and the target optical variable point.

[0124] In some embodiments, the image processing apparatus 500 further includes a second camera switch control module, configured to obtain user operation data after switching to the first camera corresponding to the target zoom ratio; determine the switch state of the second camera before switching to the first camera according to the user operation data; and control the second camera according to the switch state of the second camera.

[0125] In some embodiments, the image processing apparatus 500 further includes a fallback switching module, configured to determine a fallback camera according to the object distance; control the fallback camera to be turned on when the object distance is less than a pre - opening distance threshold; when the object distance is less than or equal to a fusion distance threshold, fuse the first image data obtained by the current camera and the second image data obtained by the fallback camera to obtain first target image data; the difference between the pre - opening distance threshold and the fusion distance threshold has a negative correlation with the object distance; the pre - opening distance threshold is greater than the fusion distance threshold; after displaying a preset number of first target image data, switch to displaying second target image data obtained by the fallback camera according to the current zoom ratio; the preset number has a negative correlation with the object distance.

[0126] In some embodiments, the zoom curve includes a first curve segment corresponding to the current zoom ratio to an intermediate zoom ratio, and a second curve segment corresponding to the intermediate zoom ratio to the target zoom ratio; the change trend of the second curve segment is flatter than that of the first curve segment, and the intermediate zoom ratio is between the current zoom ratio and the target zoom ratio; the image processing apparatus 500 further includes a zoom switching module, configured to fuse the third image data obtained by the current camera and the fourth image data obtained by the target camera to obtain third target image data, and sequentially display the third target image data according to the fusion zoom ratio in the first curve segment; after displaying the third target image data according to the fusion zoom ratio in the first curve segment, switch to displaying the fourth target image data obtained by the target camera according to the zoom ratio corresponding to the second curve segment.

[0127] Each module in the above - mentioned image processing apparatus can be implemented in whole or in part by software, hardware, and their combination. Each of the above - mentioned modules can be embedded in the processor in the electronic device in the form of hardware or be independent of the processor, or can be stored in the memory in the electronic device in the form of software, so that the processor can call and execute the operations corresponding to each of the above - mentioned modules.

[0128] In an exemplary embodiment, an electronic device is provided. The electronic device can be a terminal, and its internal structure diagram can be as Figure 6As shown in the figure. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and external devices. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an image processing method. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0129] Those skilled in the art can understand that Figure 6 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0130] In some exemplary embodiments, an electronic device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the steps of the image processing method in the above embodiments.

[0131] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the steps of the image processing method in the above embodiments.

[0132] In some embodiments, a computer program product is provided, including a computer program. When the computer program is executed by the processor, it implements the steps of the image processing method in the above embodiments.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0134] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited thereto.

[0135] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0136] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application should be subject to the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: In response to a zoom operation, obtaining a current zoom ratio and a target zoom ratio; Determining a target camera according to the current zoom ratio and the target zoom ratio; Determining a first number of fused zoom magnifications from the current zoom magnification to the target zoom magnification according to at least one of a distance between a current camera and the target camera, an object distance, or an outbound flow speed of the target camera; Obtain a second number of pre-fusion zoom magnifications from the current zoom magnification to the target zoom magnification, obtain a target number of transitional zoom magnifications based on the first number of fusion zoom magnifications and the second number of pre-fusion zoom magnifications, and process the output image data according to the target number of transitional zoom magnifications to obtain target image data.

2. The method according to claim 1, characterized in that The distance between the current camera and the target camera is positively correlated with the number of fusion zoom ratios.

3. The method according to claim 1, characterized in that: The object distance is negatively correlated with the number of fusion zoom ratios.

4. The method according to claim 1, characterized in that: The outflow speed of the target camera is negatively correlated with the number of the fusion zoom ratios.

5. The method according to claim 1, characterized in that The method further comprises: Determining a zoom range according to the current zoom ratio and the target zoom ratio; According to the zoom amplitude, the number of the fused zoom ratios is determined from the corresponding relationship between the zoom amplitude and the number of fused zoom ratios; the zoom amplitude and the number of fused zoom ratios are positively correlated.

6. The method according to claim 1, characterized in that The method further comprises: Get the shooting mode; According to the shooting mode, the number of the fusion zoom magnifications is determined from the corresponding relationship between the shooting mode and the number of the fusion zoom magnifications.

7. The method according to claim 1, characterized in that The step of determining the target camera according to the current zoom ratio and the target zoom ratio includes: Determining a zoom direction and a target light changing point according to the current zoom magnification and the target zoom magnification; A target camera is determined according to the zoom direction and the target light change point.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: After switching to the first camera corresponding to the target zoom ratio, obtaining user operation data; determining, according to the user operation data, a switch state of the second camera before switching to the first camera; The second camera is controlled according to the switch state of the second camera.

9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Determine the retraction camera according to the object distance; When the object distance is less than the pre-opening distance threshold, controlling to open the retracting camera; When the object distance is less than or equal to the fusion distance threshold, the first image data acquired by the current camera is fused with the second image data acquired by the fallback camera to obtain the first target image data; the difference between the pre-opening distance threshold and the fusion distance threshold is negatively correlated with the object distance; the pre-opening distance threshold is greater than the fusion distance threshold; After displaying a preset number of the first target image data, switch to displaying the second target image data acquired by the retracting camera according to the current zoom ratio; the preset number is negatively correlated with the object distance.

10. The method according to any one of claims 1 to 7, characterized in that: The zoom curve includes a first curve segment corresponding to the current zoom ratio to an intermediate zoom ratio, and a second curve segment corresponding to the intermediate zoom ratio to a target zoom ratio; the change trend of the second curve segment is gentler than that of the first curve segment, and the intermediate zoom ratio is between the current zoom ratio and the target zoom ratio; the method further includes: Fusing the third image data acquired by the current camera and the fourth image data acquired by the target camera to obtain third target image data, and sequentially displaying the third target image data according to the fusion zoom ratio in the first curve segment; After displaying the third target image data according to the fused zoom ratio in the first curve segment, switch to displaying the fourth target image data acquired by the target camera according to the zoom ratio corresponding to the second curve segment.

11. An image processing device, characterized in that: The device comprises: A zoom ratio acquisition module, used for acquiring a current zoom ratio and a target zoom ratio in response to a zoom operation; A target camera determination module, used to determine the target camera according to the current zoom magnification and the target zoom magnification; a first zoom ratio number determination module, configured to determine a first number of fused zoom ratios from the current zoom ratio to the target zoom ratio according to at least one of a distance between a current camera and the target camera, an object distance, or an outflow speed of the target camera; A second magnification quantity determination module is used to obtain a second number of zoom magnifications before fusion from the current zoom magnification to the target zoom magnification, obtain a target number of transitional zoom magnifications based on the first number of fusion zoom magnifications and the second number of pre-fusion zoom magnifications, and process the output image data according to the target number of transitional zoom magnifications to obtain target image data.

12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images