Image processing method and device, electronic equipment and readable storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: in response to a zooming operation, obtaining a current zooming magnification of a current image frame and a target zooming magnification currently issued by a camera application; determining a zoom scene type according to the current zoom ratio and the target zoom ratio; obtaining a corresponding target zooming curve according to the zooming scene type, and obtaining a transition zooming magnification between a current zooming magnification and a target zooming magnification by using the target zooming curve; and processing the acquired image data based on the transition zoom magnification to obtain the image data under the transition zoom magnification. The gradual change of the field angle of the image data in the focus-varying scene is realized, the sudden change of the field angle is avoided, and the picture change is smoother.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, computer-readable storage medium and computer program product. Background Art

[0002] With the development of image processing technology, it is necessary to capture clear images at different distances. For this purpose, the camera can use zoom processing to achieve the image closer processing. The camera zoom methods mainly include optical zoom and digital zoom. Optical zoom is produced by changing the positions of the lens, object and focus. When the imaging surface moves in the horizontal direction, the vision and focal length will change, and the farther the scene will become clearer, making people feel that the object is progressive. Digital zoom changes the angle of the diagonal of the imaging surface to change the viewing angle, thereby producing an effect "equivalent to" the change of the lens focal length.

[0003] Traditional electronic devices are generally equipped with multiple lenses, and electronic devices can use different cameras to relay to achieve zoom. Lens relay refers to achieving a zoom effect by switching between lenses with different focal lengths.

[0004] In traditional technology, when an electronic device is zooming, the field angle of the image may suddenly change. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, electronic device, computer-readable storage medium, and computer program product, which can make the field of view angle of the picture in different zoom scenes change more smoothly and avoid sudden changes.

[0006] In a first aspect, the present application provides an image processing method, comprising:

[0007] In response to the zoom operation, obtaining a current zoom ratio of the current image frame and a target zoom ratio currently issued by the camera application;

[0008] Determining a zoom scene type according to the current zoom magnification and the target zoom magnification;

[0009] Acquire a corresponding target zoom curve according to the zoom scene type, and obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve;

[0010] The collected image data is processed based on the transitional zoom ratio to obtain image data at the transitional zoom ratio.

[0011] In a second aspect, the present application further provides an image processing device, comprising:

[0012] A zoom ratio acquisition module, used to acquire the current zoom ratio of the current image frame and the target zoom ratio currently issued by the camera application in response to the zoom operation;

[0013] A zoom scene type determination module, used to determine the zoom scene type according to the current zoom magnification and the target zoom magnification;

[0014] A zoom parameter determination module, used to obtain a corresponding target zoom curve according to the zoom scene type, and obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio using the target zoom curve;

[0015] The image processing module is used to process the collected image data based on the transition zoom ratio to obtain the image data under the transition zoom ratio.

[0016] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect when executing the computer program.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0018] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0019] The above-mentioned image processing method, device, electronic device, computer-readable storage medium and computer program product obtain the current zoom ratio of the current image frame and the target zoom ratio currently issued by the camera application in response to the zoom operation, determine the zoom scene type according to the current zoom ratio and the target zoom ratio, and then obtain the corresponding target zoom curve according to the zoom scene type, and use the target zoom curve to obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio, and then use the transition zoom ratio to process the collected image data to obtain image data at the transition zoom ratio, so that image data that transitions from the current zoom ratio to the target zoom ratio can be obtained, and the field of view of the image data in the zoom scene is gradually changed to avoid the sudden change of the field of view angle. The picture changes are also smoother. In addition, different zoom curves are used for zoom smoothing processing for different zoom scene types, which adapts to different scenes to balance power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] FIG1 is a schematic diagram of a flow chart of an image processing method in one embodiment;

[0022] FIG2 is a schematic diagram of a camera of an electronic device in one embodiment;

[0023] FIG3 is a schematic diagram of a light changing point in one embodiment;

[0024] FIG4 is a schematic flow chart of an image processing method in another embodiment;

[0025] FIG5 is a schematic diagram of zoom interpolation in one embodiment;

[0026] FIG6 is a block diagram of an image processing device according to an embodiment;

[0027] FIG. 7 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] In an exemplary embodiment, as shown in FIG1 , an image processing method is provided, and the method is applied to an electronic device as an example for explanation. The electronic device may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, smart cars, etc., and the portable wearable devices may be smart watches and smart bracelets, etc. It can be understood that the method can also be applied to a system including an electronic device and a server, and is implemented through the interaction between the electronic device and the server. In this embodiment, the method is applied to an electronic device as an example for explanation, including steps 102 to 106, wherein:

[0030] Step 102: In response to the zoom operation, a current zoom ratio of the current image frame and a target zoom ratio currently issued by the camera application are obtained.

[0031] A zoom operation may refer to an uncertain continuous zoom operation, that is, the user does not have a clear target zoom ratio during the zoom process, and a better preview or recording screen is achieved by adjusting the zoom ratio and viewing the result of the zoom screen. A zoom operation may also be a zoom ratio that is slid to a certain target. The zoom operation may be achieved by sliding a zoom control on the camera application interface, which may be a disk, a strip, or other forms of zoom control. In some possible implementations, the zoom operation may also be achieved by sliding or rotating a hardware device, such as configuring a rotatable device on an electronic device, and adjusting the zoom ratio by rotating the rotatable device.

[0032] The current image frame refers to the image frame that needs to be previewed, captured, or recorded. The current zoom ratio refers to the zoom ratio of the current image frame. The camera application obtains the zoom ratio obtained by the zoom operation in real time or at a fixed time, and sends the obtained zoom ratio as the target zoom ratio to the hardware abstraction layer. The target zoom ratio currently sent by the camera application refers to the target zoom ratio currently obtained by the camera application for the user's sliding operation.

[0033] Optionally, the electronic device obtains the current zoom ratio of the current image frame and the target zoom ratio currently sent by the camera application in response to a zoom operation generated by the user sliding a disc or a zoom operation generated by the user rotating a hardware device.

[0034] Step 104: Determine the zoom scene type according to the current zoom ratio and the target zoom ratio.

[0035] The zoom scene type refers to the zoom scene type generated by the zoom operation, which may include a first zoom scene type, a second zoom scene type, and a zoom scene type without switching the image sensor (fall back). The sliding zoom rate of the first zoom scene type is greater than the sliding zoom rate of the second zoom scene type. In other words, the first zoom scene type is a fast sliding zoom scene, and the second zoom scene type is a slow sliding zoom scene.

[0036] Optionally, the electronic device may determine the zoom scene type according to the difference between the current zoom magnification and the target zoom magnification, or may determine the zoom ratio according to the current zoom magnification and the target zoom magnification, and determine the zoom scene type according to the zoom ratio.

[0037] Optionally, the electronic device can determine the zoom scene type by judging whether to switch the image sensor based on the current zoom ratio and the target zoom ratio. When the image sensor does not need to be switched, the zoom scene type belongs to the fall back type. When the image sensor needs to be switched, it continues to judge whether it is a fast zoom scene or a slow zoom scene based on the current zoom ratio and the target zoom ratio.

[0038] Step 106 , obtaining a corresponding target zoom curve according to the zoom scene type, and using the target zoom curve to obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio.

[0039] The zoom curve is a pre-configured zoom processing method to make the zoom process smoother. It can be expressed by different functions, such as linear function, power function, exponential function, etc. Different zoom curves can be configured for different zoom scene types. Using the target zoom curve, the transition zoom ratio is inserted between the current zoom ratio and the target zoom ratio to obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio.

[0040] Step 108 : Processing the collected image data based on the transitional zoom ratio to obtain image data at the transitional zoom ratio.

[0041] Optionally, the collected image data may be image data collected when the camera is in preview mode, image data collected by pressing a shutter button to take a picture, or image data collected by recording a video.

[0042] Optionally, the electronic device can determine the zoom scene type, obtain the corresponding target zoom curve from the mapping relationship between the preset zoom scene type and the zoom curve, and then use the target zoom curve to obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio. The image data is processed using the transition zoom ratio to obtain image data at the transition zoom ratio, thereby obtaining image data at the current zoom ratio, image data at the transition zoom ratio, and image data at the target zoom ratio, thereby achieving a gradual change in the field of view of the image data, avoiding sudden changes in the field of view (FOV), and making the picture changes smoother.

[0043] In this embodiment, by responding to the zoom operation, the current zoom ratio of the current image frame and the target zoom ratio currently issued by the camera application are obtained, the zoom scene type is determined according to the current zoom ratio and the target zoom ratio, and then the corresponding target zoom curve is obtained according to the zoom scene type, and the target zoom curve is used to obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio, and then the captured image data is processed using the transition zoom ratio to obtain image data at the transition zoom ratio, so that image data that transitions from the current zoom ratio to the target zoom ratio can be obtained, and the field of view of the image data in the zoom scene is gradually changed to avoid sudden changes in the field of view angle, and the picture changes are smoother. In addition, different zoom curves are used for zoom smoothing processing for different zoom scene types, and different scenes are adapted to balance power consumption.

[0044] In some exemplary embodiments, the zoom control method further includes: in the process of calculating a transition zoom ratio using a target zoom curve according to the current zoom ratio and the target zoom ratio, and processing the next adjacent image frame of the current image frame using the transition zoom ratio, when a target zoom ratio newly sent by the camera application is received, the transition zoom ratio is used as the current zoom ratio of the new current image frame, and the new target zoom ratio, and the step of determining the zoom scene type according to the current zoom ratio and the target zoom ratio is re-executed.

[0045] For example, the current zoom ratio is 1X, the target zoom ratio is 3X, and the transition zoom ratio is calculated to be 1.2X using the target zoom curve (based on inserting 3 frames). During the process of processing the next image frame using 1.2X, the target zoom ratio newly sent by the camera application is 3.5X. Then, 1.2X and 3.5X are used to redetermine the zoom scene type, and the corresponding target zoom curve is obtained according to the redetermined zoom scene type. Using the target zoom curve, a transition zoom ratio between the current zoom ratio and the target zoom ratio is obtained.

[0046] When the next image frame is processed using the transition zoom ratio, a new target zoom ratio is received, the transition zoom ratio is used as the new current zoom ratio, and then the zoom scene type is redetermined based on the new current zoom ratio and the new target zoom ratio, and the corresponding target zoom curve is obtained, and the transition zoom ratio is recalculated. This can more accurately reflect the new target zoom ratio generated by the zoom operation, so that the field of view angle change of the image data screen adapts to the new target zoom ratio.

[0047] In some exemplary embodiments, the zoom control method further includes: determining the image sensor to be called according to the current zoom ratio and the target zoom ratio; and obtaining the activation status of the image sensor to be called.

[0048] Different cameras may have different zoom ratio ranges. Different cameras may be required to achieve different zoom ratios. When a certain zoom ratio needs to be achieved using a certain camera, it is necessary to call the image sensor corresponding to the camera and turn on the image sensor. Generally, an electronic device may include one or more cameras. If there is only one camera, its zoom range is limited, and if there are multiple cameras, the zoom range is wider. An electronic device may use one of the cameras as the main camera and the other cameras as secondary cameras. The main camera refers to the main camera used to capture most of the image data in the electronic device. For example, a schematic diagram of the camera of an electronic device is shown in Figure 2. The electronic device includes 4 cameras, namely an ultra-wide-angle UW (Ultra Wide) lens, a wide-angle W (Wide) lens, a telephoto T (Telephoto) lens, and an ultra-telephoto UT (UltraTelephoto). The zoom ratio at which the camera switches is generally the light change point.

[0049] The on-state of the image sensor refers to whether the image sensor is turned on, that is, in an on state or an off state.

[0050] Optionally, the electronic device can determine the activation status of the image sensor by acquiring the status flag of the image sensor. The status flag is a first flag, indicating that the image sensor is activated, and the status flag is a second flag, indicating that the image sensor is not activated, that is, in an activated state. The first flag and the second flag can be configured as required.

[0051] Accordingly, the method acquires a corresponding target zoom curve according to the zoom scene type, and obtains a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve, including:

[0052] According to the activation status of the image sensor and the type of the zoom scene, a corresponding target zoom curve is determined, and a transition zoom ratio between the current zoom ratio and the target zoom ratio is obtained by using the target zoom curve.

[0053] Optionally, the electronic device may pre-configure a mapping relationship between the image sensor's activation status, the zoom scene type, and the zoom curve, and then determine a corresponding target zoom curve from the mapping relationship between the image sensor's activation status, the zoom scene type, and the zoom curve according to the image sensor's activation status and the zoom scene type.

[0054] According to the activation status of the image sensor and the type of zoom scene, the corresponding target zoom curve can be obtained in a more detailed manner, which can more finely control the zoom changes in different zoom scenes, making the picture changes smoother and more fluid.

[0055] In some exemplary embodiments, the target zoom curve may include at least one of a first zoom curve and a second zoom curve. The method determines the corresponding target zoom curve according to the activation status of the image sensor and the zoom scene type, and obtains a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve, including:

[0056] When the image sensor to be called is turned on and the zoom scene type is a first zoom scene type, a corresponding first zoom curve is acquired, and a first number of transition zoom magnifications between the current zoom magnification and the target zoom magnification are obtained by using the first zoom curve;

[0057] When the image sensor to be called is turned on and the zoom scene type is a second zoom scene type, a corresponding second zoom curve is acquired, and a second number of transition zoom magnifications between the current zoom magnification and the target zoom magnification are obtained by using the second zoom curve;

[0058] The zoom rate of the first zoom scene type is greater than the zoom rate of the second zoom scene type.

[0059] Optionally, the first zoom curve is different from the second zoom curve. For example, the first zoom curve is a curve represented by a linear function, and the second zoom curve is a curve represented by an exponential function, but the present invention is not limited thereto.

[0060] The zoom rate of the first zoom scene type is greater than the zoom rate of the second zoom scene type, that is, the first zoom scene type is a fast zoom scene, and the second zoom scene type is a slow zoom scene. The zoom rate refers to the zoom rate generated by the zoom operation, which can be represented by the difference between the target zoom ratio and the current zoom ratio. If the difference is greater than the difference threshold, it indicates a fast zoom scene. If the difference is less than or equal to the difference threshold, it indicates a slow zoom scene. It can also be represented by the ratio of the difference between the target zoom ratio and the current zoom ratio to the current zoom ratio. If the ratio is greater than the ratio threshold, it indicates a fast zoom scene. If the ratio is less than or equal to the ratio threshold, it indicates a slow zoom scene.

[0061] In some exemplary embodiments, the zoom scene type is determined based on the current zoom magnification and the target zoom magnification, including: if the difference between the target zoom magnification and the current zoom magnification is greater than a difference threshold, determining the zoom scene type to be the first zoom scene type; or, if the ratio of the difference between the target zoom magnification and the current zoom magnification to the current zoom magnification is greater than a ratio threshold, determining the zoom scene type to be the first zoom scene type.

[0062] In some exemplary embodiments, the zoom scene type is determined based on the current zoom magnification and the target zoom magnification, including: if the difference between the target zoom magnification and the current zoom magnification is less than or equal to a difference threshold, then the zoom scene type is determined to be the second zoom scene type; or, if the ratio of the difference between the target zoom magnification and the current zoom magnification and the current zoom magnification is less than or equal to a ratio threshold, then the zoom scene type is determined to be the second zoom scene type.

[0063] The difference threshold and the ratio threshold can be configured as needed. The zoom scene is judged by the difference, which is simple and fast. The zoom scene is judged by the ratio, which is more accurate.

[0064] Optionally, the first number and the second number can be determined according to the opening and closing angle of the disk. Taking the sliding zoom operation on the disk as an example, the basic number of transitional zoom ratios can be determined according to the opening and closing angle of the disk, and then whether to increase the number of transitional zoom ratios is determined according to whether the current zoom ratio and the target zoom ratio pass through the light change point, thereby obtaining the first number and the second number.

[0065] Optionally, a first number of transitional zoom ratios is determined according to the current zoom ratio, the target zoom ratio, the first zoom curve and the first number; and a second number of transitional zoom ratios is determined according to the current zoom ratio, the target zoom ratio, the second zoom curve and the second number.

[0066] If the image sensor to be called is already turned on, the image sensor can be used directly to achieve zooming, without the need to prepare for the image sensor to be turned on, and without the need to insert multiple transitional zoom ratios for the image sensor to be turned on. For different zoom scenes, different zoom curves are configured to obtain corresponding transitional zoom ratios, so that the image data can gradually change with the zoom ratio, and the field of view angle can gradually change to avoid sudden changes.

[0067] If the image sensor is not turned on, it is necessary to detect whether the image signal processor resources can support the image sensor that needs to be called to be turned on. In some electronic devices, the image signal processor resources are less than the number of cameras and cannot support multiple cameras to be turned on at the same time. When the image signal processor resources cannot support multiple cameras to be turned on at the same time, it is necessary to turn off some unnecessary cameras to release the occupied image signal processor resources, and then use the released image signal processor resources to support the camera that needs to be called to be turned on. When the image signal processor resources cannot support the image sensor to be turned on, more time is required to wait for the unnecessary image sensors to be turned off and the required image sensors to be turned on. To this end, more transition image frames are required, and more transition zoom ratios need to be added. Using more transition zoom ratios to process the image data can avoid sudden changes in the field of view of the image screen, and also avoid errors or freezes when the image sensor cannot be turned on.

[0068] In some exemplary embodiments, the target zoom curve may include at least one of a third zoom curve, a fourth zoom curve, a fifth zoom curve, and a sixth zoom curve. The determining of the corresponding target zoom curve according to the activation status of the image sensor and the zoom scene type, and obtaining a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve includes:

[0069] When the image sensor to be called is not turned on, detecting whether the image signal processor resources support turning on the image sensor to be called;

[0070] If the image signal processor resources can support the opening, and the zoom scene type is the first zoom scene type, then obtain a corresponding third zoom curve, and use the third zoom curve to obtain a third number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification;

[0071] If the image signal processor resources cannot support turning on, and the zoom scene type is the first zoom scene type, then obtain the corresponding fourth zoom curve, use the fourth zoom curve, and obtain a fourth number of transitional zoom ratios between the current zoom ratio and the target zoom ratio; the fourth number is greater than the third number.

[0072] In some exemplary embodiments, the target zoom curve may include at least one of a third zoom curve, a fourth zoom curve, a fifth zoom curve, and a sixth zoom curve. The method determines the corresponding target zoom curve according to the activation status of the image sensor and the zoom scene type, and obtains a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve, including:

[0073] If the image signal processor resources can support the opening, and the zoom scene type is the second zoom scene type, then obtain the corresponding fifth zoom curve, adopt the fifth zoom curve, and obtain a fifth number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification;

[0074] If the image signal processor resources cannot support the opening, and the zoom scene type is the second zoom scene type, obtaining a corresponding sixth zoom curve, using the sixth zoom curve, and obtaining a sixth transition zoom ratio between the current zoom ratio and the target zoom ratio;

[0075] The zoom rate of the first zoom scene type is greater than the zoom rate of the second zoom scene type, and the sixth number is greater than the fifth number.

[0076] Optionally, a transitional zoom ratio of a third number is determined based on the current zoom ratio, the target zoom ratio, the third zoom curve and the third number; a transitional zoom ratio of a fourth number is determined based on the current zoom ratio, the target zoom ratio, the fourth zoom curve and the fourth number; a transitional zoom ratio of a fifth number is determined based on the current zoom ratio, the target zoom ratio, the fifth zoom curve and the fifth number; a transitional zoom ratio of a sixth number is determined based on the current zoom ratio, the target zoom ratio, the sixth zoom curve and the sixth number.

[0077] In this embodiment, for the same zoom scene, if the image signal processor resources cannot support the activation of the image sensor, more transitional image frames can be obtained by increasing the transitional zoom ratio, so that the field of view angle of the displayed image changes more smoothly during the zoom process, and problems such as screen freeze or freezing caused by the image sensor not being activated in time can be avoided.

[0078] In some exemplary embodiments, the image sensor that needs to be called is determined based on the current zoom ratio and the target zoom ratio, including: determining the zoom direction and the light change point that needs to be passed based on the current zoom ratio and the target zoom ratio; determining the image sensor that needs to be called based on the zoom direction and the light change point that needs to be passed.

[0079] The light change point refers to the zoom magnification threshold for switching or opening a new camera, which can be understood as the zoom magnification threshold for switching from one camera to another, or adding a zoom magnification threshold for a new camera based on the current camera. For example, a schematic diagram of the light change point is shown in Figure 3. The cameras include UW (Ultra Wide), W (Wide), T (Telephoto), and UT (Ultra Telephoto). In a stable state, the light change point corresponding to the ultra-wide-angle lens is 0.6, the light change point corresponding to the wide-angle lens is 1.0, the light change point corresponding to the telephoto lens is 3.0, and the light change point corresponding to the ultra-telephoto lens is 6.0.

[0080] The zoom direction and the light change points to be passed can be determined based on the current zoom ratio and the target zoom ratio. The zoom direction can be a zoom in direction (zoom in) or a zoom out direction (zoom out). By comparing the interval between the current zoom ratio and the target zoom ratio with the light change points, it can be determined which light change points to pass through in the interval between the current zoom ratio and the target zoom ratio.

[0081] In some exemplary embodiments, the image signal processing resources are less than the image sensors and cannot support multiple image sensors being turned on at the same time. When a new image sensor needs to be turned on, the unnecessary image sensors need to be turned off first. The above method also includes: determining the image sensor to be turned off according to the zoom direction and the light change point to be passed.

[0082] According to the zoom direction and the light change point to be passed, the image sensors that need to be called and the image sensors that need to be turned off can be determined, and the image sensors that need to be called can be pulled up in time, and the image sensors that are not needed can be turned off in time. In order to ensure that the availability of image signal processor resources can be known in time when the image signal processor resources are limited, the shutdown status of the image sensors that need to be turned off can be detected frame by frame, so as to facilitate the precise control of the timing of pulling up the image sensors.

[0083] For example, if the current zoom ratio is 0.8X and the target zoom ratio is 3X, the zoom direction is the zoom-in direction, and the light change points passed include 1X and 3X. The image sensors that need to be called include the image sensor corresponding to the wide-angle camera at 1X and the image sensor corresponding to the telephoto camera at 3X. Assuming that the image signal processor resources are insufficient, the image sensor corresponding to the ultra-wide-angle camera can be turned off. For example, if the current zoom ratio is 10X and the target zoom ratio is 2X, the zoom direction is the zoom-out direction, and the light change points passed include 6X and 3X. The image sensors that need to be called include the image sensor corresponding to the telephoto camera at 6X and the image sensor corresponding to the wide-angle camera at 3X. Assuming that the image signal processor resources are insufficient, the image sensor corresponding to the ultra-telephoto camera can be turned off.

[0084] In some exemplary embodiments, the zoom scene type is a first zoom scene type, and obtaining a corresponding target zoom curve according to the zoom scene type includes: dividing a ratio interval between a current zoom ratio and a target zoom ratio into multiple sub-intervals to obtain a corresponding multiple sub-focal segments; obtaining a zoom curve corresponding to each sub-focal segment; wherein a rate of change of a transition zoom ratio obtained by the zoom curves of the multiple sub-focal segments is from fast to slow; and obtaining a target zoom curve corresponding to the first zoom scene type according to the zoom curves corresponding to the multiple sub-focal segments.

[0085] When the zoom scene type is the first zoom scene type, regardless of whether the image sensor is turned on or not, the ratio interval between the current zoom ratio and the target zoom ratio can be divided into multiple sub-intervals to obtain multiple corresponding sub-focal segments, and a zoom curve corresponding to each sub-focal segment is obtained. According to the zoom curves corresponding to the multiple sub-focal segments, a target zoom curve corresponding to the first zoom scene type is obtained. The change rate of the transition zoom ratio obtained by the zoom curves corresponding to the multiple sub-focal segments is from fast to slow, that is, from the current zoom ratio to the target zoom ratio, the zoom ratio changes faster at the beginning, and then changes slowly and steadily.

[0086] For example, if the current zoom ratio is 7X and the target zoom ratio is 0.6X, it can be divided into multiple sub-focal lengths: 7X to 3X, 3X to 0.8X, and 0.8X to 0.6X. 7X to 3X uses the average interpolation method, such as 6X, 5X, and 4X; 3X to 0.8X uses the average interpolation method, 2.45X, 1.9X, and 1.35X; 0.8X to 0.6X uses the exponential curve interpolation method, such as 0.8, 0.75, 0.7, 0.65, and 0.6.

[0087] For scenes with a large zoom span, the zoom ratio range is divided into multiple sub-ranges, and each sub-range uses a corresponding zoom curve to make the zoom process from fast to slow smoother, and the field of view angle of the picture changes more smoothly.

[0088] In some exemplary embodiments, the target zoom curve is used to obtain a transition zoom ratio between a current zoom ratio and a target zoom ratio, including: obtaining the opening and closing angle of the disk acted upon by the zoom operation, and determining a basic number of the transition zoom ratio based on the opening and closing angle; determining whether a light change point is passed based on the current zoom ratio and the target zoom ratio; if the light change point is passed, obtaining an increase in the transition zoom ratio that needs to be increased before passing the light change point, and obtaining a transition zoom ratio with a target number between the current zoom ratio and the target zoom ratio based on the basic number and the increase; if the light change point is not passed, obtaining a transition zoom ratio with a target number between the current zoom ratio and the target zoom ratio based on the basic number.

[0089] Optionally, the disc refers to a circular control component in the camera operation interface. The zoom ratio is adjusted by sliding the disc. When the disc of the camera application is slid toward the telephoto camera, the focal length of the lens becomes longer and the zoom ratio increases, i.e. zoom in; conversely, when slid toward the wide-angle camera, the focal length of the lens becomes shorter and the zoom ratio decreases, i.e. zoom out. The opening and closing angle of the disc refers to the angle at which the disc slides or rotates when the sliding operation acts on the disc. The mapping relationship between the opening and closing angle of the disc and the basic number of transitional zoom ratios can be pre-configured. For example, the opening and closing angle is 30°, and the corresponding basic number is 3.

[0090] If the light changing point is passed, the corresponding increase number is configured, that is, more transition zoom ratios are added before the light changing point, and the more transition zoom ratios are used to process the image data to obtain more image frame data, leaving enough time for the image sensor to switch at the light changing point. If the light changing point is not passed, there is no need to add more transition zoom ratios.

[0091] Optionally, the transition zoom ratio of the target number can be determined according to the current zoom ratio, the target zoom ratio, the target zoom curve and the target number. Optionally, if the target zoom curve is a linear zoom curve, the zoom step is determined according to the current zoom ratio, the target zoom ratio and the first number, and each transition zoom ratio is calculated according to the current zoom curve, the zoom step and the target number. The target number can be any value of the first number, the second number, the third number, the fourth number, the fifth number and the sixth number.

[0092] For example, the current zoom ratio Z1=10, the target zoom ratio Z2=20, the number of targets to be inserted n is 3, the target zoom curve is a linear zoom curve, calculate the zoom step length , the first transition zoom ratio , the second transition zoom ratio , the third transition zoom ratio .

[0093] In some exemplary embodiments, the zoom control method further includes: if the hardware abstraction layer receives a zoom tail insertion flag sent by the camera application, stopping the insertion and adjusting the image data according to the zoom ratio sent by the camera application.

[0094] Among them, the Hardware Abstractio Layer (HAL) refers to the software layer located between the camera application and the image sensor, which realizes the interaction between the camera application and the image sensor through a unified interface. It can be understood that the camera application sends the zoom tail insertion flag to the hardware abstraction layer, and the hardware abstraction layer controls the image sensor according to the control request information.

[0095] The zoom tail interpolation flag is used to indicate that the camera application side processes the animation at the end of the zoom, and the HAL layer does not need to perform interpolation processing. Interpolation refers to the mapping of the zoom ratio of the intermediate transition image frame. The interpolation does not change the interval and frame rate of the image frame. The intermediate transition image frame refers to the image frame between the current image frame and the target image frame corresponding to the target zoom ratio. At the end of the zoom, that is, the final stage of the zoom, the camera application has recognized the stop sliding operation, but the HAL layer has not yet recognized the stop sliding operation. The camera application will send the zoom tail interpolation flag, indicating that the zoom tail animation is processed by the camera application side. When the HAL layer receives the zoom tail interpolation flag, it stops interpolation and adjusts the image data according to the zoom ratio sent by the camera application. Adjusting the image data means cropping the field of view of the image data according to the corresponding zoom ratio. The camera application sends the zoom tail interpolation flag and zoom ratio. When the HAL layer receives the zoom tail interpolation flag, it stops interpolation. The tail interpolation processing of the sliding zoom is generated by the zoom ratio interpolation animation sent by the camera application, avoiding the zoom ratio mapping of the camera application and HAL when the sliding cut stops, resulting in a long stop time. The stop of the sliding cut can follow the sliding operation faster, the optimization independence of the tail curve is better, and a more reasonable tail curve can be used to further improve the fluency of the zoom experience.

[0096] In some exemplary embodiments, the zoom control method further includes: obtaining an activation ratio range of the image sensor; if it is detected that the current zoom ratio is within the activation ratio range of the image sensor, controlling the image sensor to be activated; if it is detected that the current zoom ratio is outside the activation ratio range of the image sensor, controlling the image sensor to be activated.

[0097] The opening magnification range refers to the zoom magnification range of the image sensor. The opening magnification range of the image sensor can be configured according to different needs. For example, in order to prevent the change of the zoom magnification from exceeding the capability of the super telephoto camera and the image sensor corresponding to the telephoto camera has not been turned on in the subsequent large-span rapid zoom out sliding zoom operation, in order to avoid the situation where the field of view of several frames remains unchanged or even twitches, the opening magnification range of the image sensor corresponding to the telephoto camera can be extended. As shown in Figure 3, in the case of zoom in, the opening magnification range of the image sensor corresponding to the super wide-angle camera is 0.6X to 1.5X, the opening magnification range of the image sensor corresponding to the wide-angle camera is 0.6X to 4.8X, the opening magnification range of the image sensor corresponding to the telephoto camera is 1.5X to 10.0X, and the opening magnification range of the image sensor corresponding to the super telephoto camera is greater than or equal to 4.8X.

[0098] Optionally, the electronic device obtains the activation ratio range of each image sensor, and then determines frame by frame whether the current zoom ratio has reached the activation ratio range of the image sensor. If it is detected that the current zoom ratio is within the activation ratio range of the image sensor, the image sensor is controlled to be turned on. If it is detected that the current zoom ratio is outside the activation ratio range of the image sensor and the image sensor is already turned on, the image sensor is controlled to be turned off.

[0099] By detecting frame by frame whether the zoom ratio reaches the opening ratio range of the image sensor, the image sensor can be pulled up or closed in time, thereby realizing precise control of the pulling up timing and closing timing of the image sensor.

[0100] In some exemplary embodiments, the zoom control method further includes: determining the zoom direction and the light change points to be passed according to the current zoom ratio and the target zoom ratio, then determining the image sensors to be called and the image sensors to be turned off according to the zoom direction and the light change points to be passed, obtaining the turn-on magnification range of each image sensor to be called and the turn-on magnification range of the image sensors to be turned off, obtaining the new current zoom ratio frame by frame, detecting whether the new current zoom ratio is within the turn-on magnification range of the image sensor, and if it is detected that the current zoom ratio is within the turn-on magnification range of the image sensor, controlling the image sensor to be turned on; if it is detected that the current zoom ratio is outside the turn-on magnification range of the image sensor and the image sensor is already turned on, controlling the image sensor to be turned off.

[0101] The image sensors that need to be called and those that need to be turned off are determined by the zoom direction and the light change points that need to be passed. Then, the current zoom ratio is detected frame by frame in combination with the opening ratio range of each image sensor and compared with the opening ratio range of the image sensor. The required image sensors can be turned on in time, and the turning on of the image sensors that need to be called can be ensured before the light change point. When the light change point is reached, the corresponding image sensors can be switched to adjust the corresponding field of view angle of the image data. Unnecessary image sensors can also be turned off in time to reserve image signal processor resources for other image sensors that need to be turned on.

[0102] In some exemplary embodiments, the start-up magnification range of the image sensor is greater than the switching magnification range of the image sensor, and the switching magnification range is a magnification range used by the image sensor based on the light changing point switching.

[0103] The opening magnification range of the image sensor can be configured as needed. For example, the opening magnification range of the image sensor corresponding to the telephoto lens is 1.5X to 10.0X. Correspondingly, the switching magnification range corresponding to the telephoto lens can be 3.0 to 6.0, that is, according to the light change point, in the 3X to 6X zoom magnification range, it is necessary to switch to use the telephoto lens to achieve it.

[0104] Setting the image sensor's on-rate range to be larger than its switching rate range actually prolongs the image sensor's on-time, ensuring that the corresponding camera can be switched in time when the camera needs to be switched at the light change point. This also avoids the situation where the field of view of some cameras is very small and exceeds the zoom range supported by some cameras, resulting in the field of view remaining unchanged for several frames or even twitching, which causes unsmooth zooming or a poor experience.

[0105] In some exemplary embodiments, a corresponding target zoom curve is obtained according to the zoom scene type, and a transition zoom ratio between the current zoom ratio and the target zoom ratio is obtained using the target zoom curve, including: when the zoom scene type is a non-switching image sensor type, obtaining a seventh zoom curve corresponding to the non-switching image sensor type, and using the seventh zoom curve to obtain a seventh transition zoom ratio between the current zoom ratio and the target zoom ratio.

[0106] Not switching the image sensor type means that there is no need to switch the camera during the zoom process, such as the fallback scenario. During the zoom process, there is no need to switch the image sensor, or only a small number of image sensor switches are required, that is, the image sensor needs to be turned on less frequently. For the zoom curve in this scenario, since the magnification change is between a single camera, the interpolation curve can be adjusted based on the zoom effect of a single camera. Optionally, the electronic device determines that the zoom scene type does not require switching the image sensor type based on the current zoom magnification and the target zoom magnification, then obtains the corresponding seventh zoom curve, and uses the seventh zoom curve to obtain a seventh number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification. The seventh number may be less than the third number or the fifth number.

[0107] For example, an electronic device includes 4 cameras, 3 image signal processor (ISP) resources, and the 3 ISP resources can support 3 cameras to be turned on at the same time. As shown in FIG4 , a zoom control method includes:

[0108] (1) The electronic device obtains information that a user opens a camera application and starts previewing or recording an image;

[0109] (2) Triggering a sliding zoom operation: The electronic device responds to a zoom operation of a user sliding a disc on a camera application interface. The sliding curve control module of the hardware abstraction layer obtains the current zoom ratio of the current image frame and the target zoom ratio sent by the camera application, and determines the zoom method and the light change point to be passed according to the current zoom ratio of the current image frame and the target zoom ratio sent by the camera application. It also determines the image sensor to be called and the image sensor to be turned off according to the zoom direction and the light change point to be passed.

[0110] The control decision performs image sensor detection control and interpolation control, and performs interpolation processing, including (3) (4) (5), as follows:

[0111] (3) The sliding curve control module can determine the corresponding target zoom curve according to the current zoom ratio and the target zoom ratio, as well as the activation status of the image sensor to be called, and use the target zoom curve to calculate a transition zoom ratio between the current zoom ratio and the target zoom ratio, and the transition zoom ratio may include the zoom ratio of the next image frame adjacent to the current image frame;

[0112] (4) If the image sensor to be called is turned on, determine the zoom scene type and the opening and closing angle of the disk according to the current zoom ratio and the target zoom ratio to perform interpolation calculation;

[0113] If the difference between the target zoom ratio and the current zoom ratio is greater than the difference threshold, the zoom scene type is determined to be the first zoom scene type; or if the ratio of the difference between the target zoom ratio and the current zoom ratio to the current zoom ratio is greater than the ratio threshold, the zoom scene type is determined to be the first zoom scene type. The first zoom scene type is a fast sliding zoom scene.

[0114] The method comprises: obtaining the opening and closing angle of the disk on which the zoom operation acts, and determining a basic number of transitional zoom ratios according to the opening and closing angles; determining whether a light change point is passed according to the current zoom ratio and the target zoom ratio; if the light change point is passed, obtaining an increase number of the transitional zoom ratio that needs to be increased before passing the light change point, and obtaining a transitional zoom ratio whose target number is between the current zoom ratio and the target zoom ratio according to the basic number and the increase number; and if the light change point is not passed, obtaining a transitional zoom ratio whose target number is between the current zoom ratio and the target zoom ratio according to the basic number.

[0115] Optionally, judging the zoom scene type and the opening and closing angle of the disk according to the current zoom ratio and the target zoom ratio to perform interpolation calculation includes: when the image sensor to be called is turned on and the zoom scene type is a first zoom scene type, obtaining a corresponding first zoom curve, using the first zoom curve, and obtaining a first number of transition zoom ratios between the current zoom ratio and the target zoom ratio; when the image sensor to be called is turned on and the zoom scene type is a second zoom scene type, obtaining a corresponding second zoom curve, using the second zoom curve, and obtaining a second number of transition zoom ratios between the current zoom ratio and the target zoom ratio.

[0116] (5) If the image sensor to be called is not turned on, check whether the image signal processor resources can support the opening of the image sensor to be called. If the image signal processor resources can support the opening of the image sensor, insert N more frames to ensure the timely opening of the image sensor to be called to avoid FOV stopping or pulling. If the image signal processor resources cannot support the opening of the image sensor, more frames need to be inserted to wait for the ISP resources to be freed up and then turn on the image sensor to be called. N can be a positive integer, for example, N is 2, 3, 4, 5, etc.

[0117] Optionally, if the image signal processor resources cannot support the opening, and the zoom scene type is the first zoom scene type, obtaining a corresponding fourth zoom curve, using the fourth zoom curve to obtain a fourth number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification;

[0118] If the image signal processor resources can support the opening, and the zoom scene type is the second zoom scene type, then obtaining a corresponding fifth zoom curve, using the fifth zoom curve to obtain a fifth number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification;

[0119] If the image signal processor resources cannot support the opening, and the zoom scene type is the second zoom scene type, obtaining a corresponding sixth zoom curve, using the sixth zoom curve, and obtaining a sixth transition zoom ratio between the current zoom ratio and the target zoom ratio;

[0120] Among them, the zoom rate of the first zoom scene type is greater than the zoom rate of the second zoom scene type, the fourth number is greater than the third number, and the sixth number is greater than the fifth number.

[0121] Optionally, during the sliding zoom operation, the camera application side will continuously send multiple target zoom ratios, and the difference between two adjacent target zoom ratios and the number of sent ratios will be related to the user's sliding speed. For example, the sent target zoom ratios can be 1X, 2X, 6X, 19X, 50X, etc.

[0122] Obtaining a difference between two adjacent target zoom magnifications, and if the difference between the two adjacent target zoom magnifications is greater than a difference threshold, determining that the zoom scene type is a first zoom scene type;

[0123] Alternatively, if the ratio of the difference between the latter target zoom magnification and the former target zoom magnification of two adjacent target zoom magnifications to the former target zoom magnification is greater than the ratio threshold, the zoom scene type is determined to be the first zoom scene type.

[0124] The interpolation method used is to calculate the zoom ratio of the next image frame based on interpolating 3 frames. The current zoom ratio is 1X, the target zoom ratio is 3X, and the transition zoom ratio is 1.2X calculated using the target zoom curve (based on interpolating 3 frames). In the process of processing the next image frame using 1.2X, the target zoom ratio newly sent by the camera application is 3.5X, and then the new transition zoom ratio corresponding to the next image frame is recalculated based on 1.2X and 3.5X according to the interpolation method of 3 frames. Although the calculation is based on interpolating 3 frames, the actual operation is to insert one or two more frames. As shown in Figure 5, before the modification, the original interpolation method has 3 image frames, and the zoom ratios are 1.0, 2.0 and 3.0. After the modification, the interpolation method of this case has 5 image frames, and the zoom ratios are 1.0, 1.5, 2.0, 2.5, 2.5, 3.0. Three new frames are inserted, and the zoom ratio changes more smoothly.

[0125] (6) Slide cut tail mark detection: When the zoom tail insertion mark sent by the camera application lock is received, the HAL layer stops inserting frames and only crops the FOV according to the zoom ratio below the camera application. The camera application obtains the current zoom ratio and target zoom ratio corresponding to the end of the sliding zoom operation. According to the current zoom ratio and target zoom ratio at the end of the sliding zoom operation, the image sensor to be called and the activation status of the image sensor to be called are determined. According to the activation status of the image sensor and the zoom scene type, the corresponding target zoom curve is determined. Using the target zoom curve, the transition zoom ratio between the current zoom ratio and the target zoom ratio at the end of the sliding zoom operation is obtained.

[0126] (7) Zoom is complete, continue previewing or recording.

[0127] For sliding zoom operations or zooming scenes through hardware devices, the zoom curve is adjusted by mapping the zoom ratio of the transition image frame, especially by performing differentiated interpolation processing on the recognition of different stages of the zoom process such as fast sliding, slow sliding, and tail, so as to better balance the zoom effect and power consumption; when sliding zoom, differentiated zoom is used according to the characteristics of the zoom scene and the degree of opening and closing of the disc Ratio mapping strategy is used to ensure the smoothness of zooming during the sliding zoom process and the timely start of the image sensor that needs to be called. In order to ensure that there is no resource competition when the image sensor is switched, a protection mechanism is designed in the switching timing, that is, first detect whether the ISP resources can support the start of the image sensor to ensure the successful start of the new image sensor, and avoid the stability problem caused by competition during switching; for different sliding zoom scenarios, differentiated zoom mapping processing is performed according to the corresponding zoom curve. For example, the FOV twitching encountered in fast sliding zoom is mostly related to the untimely pull-up of the sensor. For this scenario, differentiated multi-intermediate frames are selected (that is, more transition zoom ratios are increased) to buy more time for the image sensor that needs to be called (such as the secondary camera) to be pulled up, and balance power consumption on the basis of ensuring the zoom switching effect. Avoid directly lengthening the opening magnification range of the image sensor that needs to be called; in order to ensure optimal power consumption, the opening magnification range of different image sensors can be controlled. Therefore, some image sensors are turned off after sliding to certain focal lengths. If the sliding direction changes suddenly next time, they need to be pulled up again; the use of this solution can not only ensure the sliding process experience of the Sanshe platform, including fast sliding and slow sliding scenes, but also better optimize the four-camera fast sliding zoom scene. When there is a need to pull up a new image sensor, the availability of ISP resources must be detected to avoid stability problems caused by pull-up failure, so that it is no longer limited by the platform ISP resources, and a zoom effect close to or consistent with the three-camera platform is achieved, so that the zoom experience of the sliding scene is better, and the relationship between tracking hand, zoom duration, zoom experience and power consumption and memory is better balanced.

[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0129] The technical features or steps of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features or steps in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features or steps, they should be considered to be within the scope of the present application.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a working mode control device for implementing the working mode control method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more working mode control device embodiments provided below can refer to the limitations of the working mode control method above, and will not be repeated here.

[0131] In an exemplary embodiment, as shown in FIG6 , a zoom control device is provided, including a zoom ratio acquisition module 602, a zoom scene type determination module 604, a zoom parameter determination module 606 and an image processing module 608. Among them:

[0132] A zoom ratio acquisition module 602 is used to acquire the current zoom ratio of the current image frame and the target zoom ratio currently issued by the camera application in response to the zoom operation;

[0133] A zoom scene type determination module 604, configured to determine a zoom scene type according to the current zoom magnification and the target zoom magnification;

[0134] A zoom parameter determination module 606 is used to obtain a corresponding target zoom curve according to the zoom scene type, and obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio using the target zoom curve;

[0135] The image processing module 608 is used to process the image data based on the transition zoom ratio to obtain the image data at the transition zoom ratio.

[0136] In some exemplary embodiments, the above-mentioned apparatus further includes: an image sensor determination module and an opening condition acquisition module.

[0137] The image sensor determination module is used to determine the image sensor to be called according to the current zoom magnification and the target zoom magnification;

[0138] The opening status acquisition module is used to obtain the opening status of the image sensor that needs to be called;

[0139] The zoom parameter determination module 606 is further used to determine a corresponding target zoom curve according to the activation status of the image sensor and the zoom scene type, and obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio using the target zoom curve.

[0140] In some exemplary embodiments, the zoom parameter determination module 606 is further configured to:

[0141] When the image sensor to be called is turned on and the zoom scene type is a first zoom scene type, a corresponding first zoom curve is acquired, and a first number of transition zoom magnifications between the current zoom magnification and the target zoom magnification are obtained by using the first zoom curve;

[0142] When the image sensor to be called is turned on and the zoom scene type is the second zoom scene type, a corresponding second zoom curve is obtained, and the second zoom curve is used to obtain a second number of transition zoom ratios between the current zoom ratio and the target zoom ratio; the zoom rate of the first zoom scene type is greater than the zoom rate of the second zoom scene type.

[0143] In some exemplary embodiments, the zoom parameter determination module 606 is further configured to detect whether the image signal processor resources support the activation of the image sensor to be invoked when the image sensor to be invoked is not activated;

[0144] If the image signal processor resources can support the opening, and the zoom scene type is the first zoom scene type, then obtain a corresponding third zoom curve, and use the third zoom curve to obtain a third number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification;

[0145] If the image signal processor resources cannot support turning on, and the zoom scene type is the first zoom scene type, then obtain the corresponding fourth zoom curve, use the fourth zoom curve, and obtain a fourth number of transitional zoom ratios between the current zoom ratio and the target zoom ratio; the fourth number is greater than the third number.

[0146] In some exemplary embodiments, the zoom parameter determination module 606 is further configured to, if the image signal processor resources can support the opening and the zoom scene type is the second zoom scene type, obtain a corresponding fifth zoom curve, and use the fifth zoom curve to obtain a fifth number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification;

[0147] If the image signal processor resources cannot support the opening, and the zoom scene type is the second zoom scene type, obtaining a corresponding sixth zoom curve, using the sixth zoom curve, and obtaining a sixth transition zoom ratio between the current zoom ratio and the target zoom ratio;

[0148] The sixth number is greater than the fifth number. It can be understood that the zoom rate of the first zoom scene type is greater than the zoom rate of the second zoom scene type.

[0149] In some exemplary embodiments, the image sensor determination module is further used to determine the zoom direction and the light change point to be passed according to the current zoom ratio and the target zoom ratio; and determine the image sensor to be called according to the zoom direction and the light change point to be passed.

[0150] In some exemplary embodiments, the zoom scene type determination module 604 is also used to determine that the zoom scene type is the first zoom scene type if the difference between the target zoom ratio and the current zoom ratio is greater than a difference threshold; or, if the ratio of the difference between the target zoom ratio and the current zoom ratio and the current zoom ratio is greater than a ratio threshold, determine that the zoom scene type is the first zoom scene type.

[0151] In some exemplary embodiments, the zoom scene type is a first zoom scene type, and the zoom parameter determination module 606 is further used to divide the ratio interval between the current zoom ratio and the target zoom ratio into multiple sub-intervals to obtain corresponding multiple sub-focal segments; obtain a zoom curve corresponding to each sub-focal segment; wherein the rate of change of the transition zoom ratio obtained by the zoom curves of the multiple sub-focal segments is from fast to slow; and obtain a target zoom curve corresponding to the first zoom scene type based on the zoom curves corresponding to the multiple sub-focal segments.

[0152] In some exemplary embodiments, the zoom parameter determination module 606 is also used to obtain the opening and closing angle of the disk acted on by the zoom operation, and determine the basic number of the transition zoom ratio according to the opening and closing angle; determine whether the light change point is passed according to the current zoom ratio and the target zoom ratio; if the light change point is passed, obtain the increase number of the transition zoom ratio that needs to be increased before passing the light change point, and obtain the transition zoom ratio with a target number between the current zoom ratio and the target zoom ratio according to the basic number and the increase number; if the light change point is not passed, obtain the transition zoom ratio with a target number between the current zoom ratio and the target zoom ratio according to the basic number.

[0153] In some exemplary embodiments, the image processing module 608 is further configured to stop frame insertion if the hardware abstraction layer receives a zoom tail frame insertion flag sent by the camera application, and adjust the image data according to the zoom ratio sent by the camera application.

[0154] In some exemplary embodiments, the above-mentioned device also includes: an image sensor switch module, which is used to obtain the activation ratio range of the image sensor; if it is detected that the current zoom ratio is within the activation ratio range of the image sensor, the image sensor is controlled to be turned on; if it is detected that the current zoom ratio is outside the activation ratio range of the image sensor, the image sensor is controlled to be turned off.

[0155] In some exemplary embodiments, the start-up magnification range of the image sensor is greater than the switching magnification range of the image sensor, and the switching magnification range is a magnification range used by the image sensor based on the light changing point switching.

[0156] In some exemplary embodiments, the zoom parameter determination module 606 is further used to obtain a seventh zoom curve corresponding to the non-switching image sensor type when the zoom scene type is a non-switching image sensor type, and use the seventh zoom curve to obtain a seventh number of transition zoom ratios between the current zoom ratio and the target zoom ratio.

[0157] Each module in the above device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in an electronic device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0158] In an exemplary embodiment, an electronic device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG7. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a working mode control method is implemented. The display unit of the electronic device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0159] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0160] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0161] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0162] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0164] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0165] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0166] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: include: In response to the zoom operation, obtaining a current zoom ratio of the current image frame and a target zoom ratio currently issued by the camera application; Determining a zoom scene type according to the current zoom magnification and the target zoom magnification; Acquire a corresponding target zoom curve according to the zoom scene type, and obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve; The collected image data is processed based on the transitional zoom ratio to obtain image data at the transitional zoom ratio.

2. The method according to claim 1, characterized in that The method further comprises: Determining an image sensor to be called according to the current zoom magnification and the target zoom magnification; Get the activation status of the image sensor to be called; The step of acquiring a corresponding target zoom curve according to the zoom scene type, and obtaining a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve, comprises: According to the activation status of the image sensor and the zoom scene type, a corresponding target zoom curve is determined, and a transition zoom ratio between the current zoom ratio and the target zoom ratio is obtained by using the target zoom curve.

3. The method according to claim 2, characterized in that The step of determining a corresponding target zoom curve according to the activation status of the image sensor and the zoom scene type, and obtaining a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve, comprises: When the image sensor to be called is turned on and the zoom scene type is a first zoom scene type, obtaining a corresponding first zoom curve, and using the first zoom curve to obtain a first number of transition zoom magnifications between the current zoom magnification and the target zoom magnification; When the image sensor to be called is turned on and the zoom scene type is a second zoom scene type, a corresponding second zoom curve is obtained, and the second zoom curve is used to obtain a second number of transition zoom ratios between the current zoom ratio and the target zoom ratio; the zoom rate of the first zoom scene type is greater than the zoom rate of the second zoom scene type.

4. The method according to claim 2, characterized in that: The step of determining a corresponding target zoom curve according to the activation status of the image sensor and the zoom scene type, and obtaining a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve comprises: When the image sensor to be called is not turned on, detecting whether image signal processor resources support turning on the image sensor to be called; If the image signal processor resources can support the opening, and the zoom scene type is the first zoom scene type, then obtain a corresponding third zoom curve, and use the third zoom curve to obtain a third number of transition zoom magnifications between the current zoom magnification and the target zoom magnification; If the image signal processor resources cannot support being turned on, and the zoom scene type is the first zoom scene type, then obtain the corresponding fourth zoom curve, and use the fourth zoom curve to obtain a fourth number of transitional zoom ratios between the current zoom ratio and the target zoom ratio; the fourth number is greater than the third number.

5. The method according to claim 2, characterized in that: The step of determining a corresponding target zoom curve according to the activation status of the image sensor and the zoom scene type, and obtaining a transition zoom ratio between a current zoom ratio and a target zoom ratio by using the target zoom curve, comprises: If the image signal processor resources can support the opening, and the zoom scene type is the second zoom scene type, then obtaining a corresponding fifth zoom curve, using the fifth zoom curve to obtain a fifth number of transitional zoom magnifications between the current zoom magnification and the target zoom magnification; If the image signal processor resources cannot support the opening, and the zoom scene type is the second zoom scene type, obtaining a corresponding sixth zoom curve, using the sixth zoom curve, and obtaining a sixth transition zoom ratio between the current zoom ratio and the target zoom ratio; Wherein, the sixth number is greater than the fifth number.

6. The method according to claim 2, characterized in that The step of determining the image sensor to be called according to the current zoom magnification and the target zoom magnification includes: Determining a zoom direction and a light change point to be passed according to the current zoom magnification and the target zoom magnification; The image sensor to be called is determined according to the zoom direction and the light changing point to be passed.

7. The method according to claim 1, characterized in that The determining the zoom scene type according to the current zoom magnification and the target zoom magnification includes: If the difference between the target zoom ratio and the current zoom ratio is greater than the difference threshold, determining the zoom scene type to be the first zoom scene type; Alternatively, if the ratio of the difference between the target zoom magnification and the current zoom magnification to the current zoom magnification is greater than the ratio threshold, the zoom scene type is determined to be the first zoom scene type.

8. The method according to claim 7, characterized in that The zoom scene type is a first zoom scene type, and acquiring a corresponding target zoom curve according to the zoom scene type includes: Divide the zoom magnification interval between the current zoom magnification and the target zoom magnification into a plurality of sub-intervals to obtain a plurality of corresponding sub-focal segments; Obtaining a zoom curve corresponding to each sub-focal segment; wherein the change rate of the transition zoom ratio obtained by the zoom curves of the multiple sub-focal segments is from fast to slow; According to the zoom curves corresponding to the multiple sub-focal segments, a target zoom curve corresponding to the first zoom scene type is obtained.

9. The method according to claim 1, characterized in that: The step of obtaining a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve includes: Obtaining the opening and closing angle of the disk affected by the zoom operation, and determining the basic number of transitional zoom ratios according to the opening and closing angle; Determining whether a light changing point has been passed according to the current zoom magnification and the target zoom magnification; If the light changing point is passed, the increase amount of the transition zoom factor that needs to be increased before passing the light changing point is obtained, and a transition zoom factor whose target amount is between the current zoom factor and the target zoom factor is obtained according to the basic amount and the increase amount; If the light changing point is not passed, a transition zoom ratio whose target number is between the current zoom ratio and the target zoom ratio is obtained according to the basic number.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: If the hardware abstraction layer receives the zoom tail insertion flag sent by the camera application, the frame insertion is stopped, and the image data is adjusted according to the zoom ratio sent by the camera application.

11. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Get the opening magnification range of the image sensor; If it is detected that the current zoom ratio is within the opening ratio interval of the image sensor, controlling the image sensor to be opened; If it is detected that the current zoom ratio is outside the on-ratio range of the image sensor, the image sensor is controlled to be turned off.

12. The method according to claim 11, characterized in that The start-up magnification range of the image sensor is greater than the switching magnification range of the image sensor, and the switching magnification range is a magnification range used by the image sensor based on the light changing point switching.

13. The method according to claim 1, characterized in that The step of acquiring a corresponding target zoom curve according to the zoom scene type, and obtaining a transition zoom ratio between the current zoom ratio and the target zoom ratio by using the target zoom curve, comprises: When the zoom scene type is a non-switching image sensor type, a seventh zoom curve corresponding to the non-switching image sensor type is obtained, and the seventh zoom curve is used to obtain a seventh number of transition zoom magnifications between the current zoom magnification and the target zoom magnification.

14. An image processing device, characterized in that: include: A zoom ratio acquisition module, used to acquire the current zoom ratio of the current image frame and the target zoom ratio currently issued by the camera application in response to a zoom operation; A zoom scene type determination module, used to determine the zoom scene type according to the current zoom magnification and the target zoom magnification; A zoom parameter determination module, used to obtain a corresponding target zoom curve according to the zoom scene type, and obtain a transition zoom ratio between the current zoom ratio and the target zoom ratio using the target zoom curve; The image processing module is used to process the collected image data based on the transition zoom ratio to obtain the image data under the transition zoom ratio.

15. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images