Photographic parameter adjustment method, electronic device, and storage medium

Abstract

Provided in the present application are a photographic parameter adjustment method, an electronic device, and a storage medium. The method comprises: an electronic device receiving from a user a first operation regarding a camera application, and responding to same, so as to acquire a target photographic parameter; a photographic parameter generation module determining a first transitional photographic parameter on the basis of a frame return moment of a first image frame and the target photographic parameter; an image cropping module processing the first image frame on the basis of the first transitional photographic parameter and photographic parameters of the first image frame, so as to obtain a first transitional image frame, wherein photographic parameters of the first transitional image frame comprise the first transitional photographic parameter; and during a rendering process, rendering a first transitional image on the basis of a vertical synchronization signal cycle of the rendering process, and the electronic device displaying the rendered first transitional image in the camera application. The method ensures that a photographic parameter can smoothly transition to a target photographic parameter, and also enables the electronic device to quickly and uniformly display an image frame in response to a user operation for changing the photographic parameter, thereby avoiding the problem of a delayed response.

Description

Shooting parameter adjustment method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application with application number 202311692270.9 filed with the State Intellectual Property Office of China on December 8, 2023, and priority to the Chinese patent application with the invention name “A shooting parameter adjustment method, electronic device and storage medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a shooting parameter adjustment method, an electronic device, and a storage medium. Background Art

[0003] With the development of science and technology, the shooting effects of electronic devices (such as digital cameras, or mobile phones and tablet computers with cameras, etc.) are getting better and better, and using electronic devices to shoot has become a common choice for people.

[0004] When users use electronic devices to shoot, they can adjust shooting parameters to achieve the desired effect. For example, users can adjust the shooting range by zooming in or out, or adjust shooting parameters such as exposure. Currently, when users change camera shooting parameters, the electronic device will experience a delay in responding to the user's operation, resulting in image jitter or freezes, and a poor user experience. Summary of the Invention

[0005] The present application provides a shooting parameter adjustment method, an electronic device, and a storage medium, which not only ensure that the shooting parameters can smoothly transition to the target shooting parameters, but also enable the electronic device to quickly and evenly display image frames in response to user operations to change the shooting parameters without the problem of delayed response.

[0006] In the first aspect, the present application provides a shooting parameter generation method, characterized in that the electronic device includes a shooting parameter generation module, an image cropping module, a display thread and a rendering process, and the method includes: the electronic device receives and responds to the user's first operation on the camera application, obtains the target shooting parameters, wherein the target shooting parameters are the shooting parameters set for the first operation; the shooting parameter generation module determines the first transition shooting parameters according to the return frame time of the first image frame and the target shooting parameters, and the first image frame is the image frame captured after the electronic device receives the first operation; the image cropping module processes the first image frame according to the first transition shooting parameters and the shooting parameters of the first image frame to obtain the first transition image frame, and the shooting parameters of the first transition image frame include the first transition shooting parameters; the rendering process renders the first transition image according to the vertical synchronization signal period of the rendering process; the electronic device displays the rendered first transition image in the camera application.

[0007] Through this method, on the one hand, the electronic device can calculate one or more transition shooting parameters in real time, so that the electronic device can collect and display image frames in accordance with one or more transition shooting parameters in a timely manner, so that the change of the shooting parameters conforms to the preset smooth adjustment curve, and the shooting parameters can be smoothly transitioned to the target shooting parameters, avoiding the situation of picture jitter or freeze, thereby improving the user's visual experience. On the other hand, by copying and cropping image frames, the electronic device can quickly display image frames in response to user operations to change shooting parameters without the problem of delayed response. On the other hand, in the process of changing shooting parameters, the electronic device 100 can control and change the vsync signal period, and associate the display timing of the transition image frame with the vsync signal period to ensure that the electronic device 100 displays the image frame evenly, thereby improving the user's visual experience.

[0008] In combination with the first aspect, in a possible implementation method, the shooting parameter generation module determines the first transition shooting parameter based on the return frame time of the first image frame and the target shooting parameter, specifically including: the shooting parameter generation module determines the first transition shooting parameter based on the initial shooting parameter, the return frame time of the first image frame and the target shooting parameter, wherein the initial shooting parameter is the shooting parameter set before the electronic device receives the first operation.

[0009] Through this method, the electronic device can calculate one or more transition shooting parameters in real time based on the return frame moment of the image frame, and smoothly switch from the initial shooting parameters to the target shooting parameters according to one or more transition shooting parameters in turn, avoiding picture jitter or freeze during the switching of shooting parameters, thereby improving the user's visual experience.

[0010] In combination with the first aspect, in a possible implementation method, the shooting parameter generation module determines the first transition shooting parameter based on the initial shooting parameter, the return frame time of the first image frame and the target shooting parameter, specifically including: the shooting parameter generation module obtains a smooth adjustment curve based on the initial shooting parameter and the target shooting parameter, the starting point of the smooth adjustment curve is the initial shooting parameter, and the end point of the smooth adjustment curve is the target shooting parameter; the shooting parameter generation module determines the first transition shooting parameter from the smooth adjustment curve based on the return frame time of the first image frame.

[0011] In some embodiments, the smooth adjustment curve may be a Bezier curve. The total duration of the smooth adjustment curve may be fixed or determined by initial shooting parameters and target shooting parameters. The total duration of the smooth adjustment curve may also be different depending on the initial shooting parameters and the target shooting parameters.

[0012] The starting point of the smooth adjustment curve may correspond to the time when the electronic device receives the first operation for the camera application. The ending point of the smooth adjustment curve may correspond to the time determined by adding the starting point of the smooth adjustment curve to the total duration of the smooth adjustment curve.

[0013] The electronic device can obtain a smooth adjustment curve based on the initial shooting parameters and the target shooting parameters, and determine one or more transition shooting parameters from the smooth adjustment curve through the return frame moment of the image frame, so that the camera application can smoothly switch from the initial shooting parameters to the target shooting parameters based on the one or more transition shooting parameters. The change in the shooting parameters of the image frame finally obtained by the electronic device conforms to the change law of the smooth adjustment curve, so that the shooting parameters can be smoothly transitioned to the target shooting parameters, avoiding picture jitter or freeze, and improving the user's visual experience.

[0014] In combination with the first aspect, in a possible implementation method, the rendering process renders the first transition image according to the vertical synchronization signal period of the rendering process, specifically including: when the time interval between the first time and the display time of the previous image frame is greater than or equal to m first vertical synchronization signal periods, the rendering process renders the first transition image at the first time.

[0015] In this way, the electronic device can temporarily store the image frame first, and then send the next image frame for rendering after the electronic device finishes rendering the previous image frame, thereby avoiding the speed of sending the image frame exceeding the speed of the electronic device rendering the image frame.

[0016] In combination with the first aspect, in a possible implementation, after the shooting parameter generation module determines the first transition shooting parameter based on the return frame time of the first image frame and the target shooting parameter, the method also includes: the shooting parameter generation module determines the second transition shooting parameter based on the return frame time of the first image frame, the preset duration and the target shooting parameter; the image cropping module processes the first image frame according to the second transition shooting parameter and the shooting parameter of the first image frame to obtain a second transition image frame, and the shooting parameter of the second transition image frame includes the second transition shooting parameter; the rendering process renders the second transition image according to the vertical synchronization signal period of the rendering process; the electronic device displays the rendered second transition image in the camera application.

[0017] Optionally, the electronic device is not limited to inserting one image frame, and can also insert more image frames, which is not limited in this application.

[0018] Optionally, the preset duration is related to the number of inserted frames and the output frame rate of the HAL layer. For example, if 1 frame is inserted and the output frame rate is 30PFS (33.3), the preset duration is 16.7ms.

[0019] In this way, the electronic device can increase the number of image frames in the process of adjusting the shooting parameters by inserting frames, so as to reduce the shooting parameter change value between two adjacent image frames, making the shooting parameter adjustment process smoother.

[0020] In combination with the first aspect, in a possible implementation method, the rendering process renders the second transition image according to the vertical synchronization signal period of the rendering process, specifically including: when the time interval between the second time and the display time of the previous image frame is greater than or equal to m second vertical synchronization signal periods, the rendering process renders the second transition image at the second time.

[0021] Optionally, after the vertical synchronization signal period is changed, the display frame rate of the electronic device also needs to be changed.

[0022] Thus, in the case of interpolation, it is necessary to change the vertical synchronization signal period so that the electronic device can timely render the multiple image frames obtained after interpolation.

[0023] In combination with the first aspect, in a possible implementation, the second vertical synchronization signal period is smaller than the first vertical synchronization signal period.

[0024] Thus, due to the interpolation, in order to render multiple transition image frames in time, the vertical synchronization signal period needs to be shortened.

[0025] In combination with the first aspect, in a possible implementation, after the shooting parameter generation module determines the second transition shooting parameter based on the return frame time of the first image frame, the preset duration and the target shooting parameter, the method also includes: the shooting parameter generation module obtains the return frame time of the second image frame, and the second image frame is an image frame collected after the electronic device receives the first operation; the shooting parameter generation module determines whether the return frame time of the second image frame is later than the time corresponding to the target shooting parameter in the smooth adjustment curve; in a case where the return frame time of the second image frame is later than the time corresponding to the target shooting parameter in the smooth adjustment curve, the electronic device switches the vertical synchronization signal period from the second vertical synchronization signal period to the first vertical synchronization signal period.

[0026] In this way, after the shooting parameter adjustment is completed, that is, the returning frame time of the image frame is later than the time corresponding to the target shooting parameter, the frame insertion is stopped and the vertical synchronization signal cycle needs to be restored to the previous cycle.

[0027] In combination with the first aspect, in one possible implementation, after the electronic device receives and responds to the user's first operation on the camera application, the method also includes: the electronic device switches the vertical synchronization signal period from the first vertical synchronization signal period to the second vertical synchronization signal period.

[0028] In this way, when the electronic device receives a user operation to change a shooting parameter, the electronic device needs to shorten the vertical synchronization signal period due to frame insertion.

[0029] In combination with the first aspect, in a possible implementation, the electronic device also includes a camera module. Before the electronic device receives and responds to the user's first operation on the camera application, the method also includes: the electronic device receives and responds to the second operation of starting the camera application to obtain initial shooting parameters; the camera module captures image frames based on the initial shooting parameters.

[0030] In this way, before the electronic device receives a user operation to change the shooting parameters, the electronic device captures and displays image frames with the initial shooting parameters.

[0031] In conjunction with the first aspect, in one possible implementation, the shooting parameter generation module determines the first transition shooting parameter from the smooth adjustment curve based on the frame return time of the first image frame, specifically including:

[0032] The shooting parameter generation module determines whether the return frame time of the first image frame is earlier than the time corresponding to the target shooting parameter in the smooth adjustment curve; when the return frame time of the first image frame is earlier than the time corresponding to the target shooting parameter, the shooting parameter generation module determines the first transition shooting parameter from the smooth adjustment curve based on the return frame time of the first image frame.

[0033] In a possible implementation, the method further includes: when the return frame time of the first image frame is later than the time corresponding to the target shooting parameter, the camera module captures the image frame based on the target shooting parameter.

[0034] In this way, the electronic device can determine whether to determine the transition shooting parameters based on the image frame's frame return time and the time corresponding to the target shooting parameters in the smooth adjustment curve. In the case that the image frame's frame return time is earlier than the time corresponding to the target shooting parameters in the smooth adjustment curve, the electronic device can determine one or more transition shooting parameters based on the image frame's frame return time. In the case that the image frame's frame return time is later than the time corresponding to the target shooting parameters in the smooth adjustment curve, the electronic device does not need to determine other shooting parameters, and the shooting parameter adjustment is completed.

[0035] In combination with the first aspect, in one possible implementation, when the initial shooting parameter is greater than the target shooting parameter, the first transition shooting parameter is less than the initial shooting parameter and greater than the target shooting parameter; when the initial shooting parameter is less than the target shooting parameter, the first transition shooting parameter is greater than the initial shooting parameter and less than the target shooting parameter.

[0036] In combination with the first aspect, in a possible implementation, the electronic device includes an application framework layer and a hardware abstraction layer; the first image frame is an image frame sent by the hardware abstraction layer to the application framework layer after the electronic device receives the first operation, and the return frame moment of the first image frame is the moment when the hardware abstraction layer sends the first image frame to the application framework layer.

[0037] In a possible implementation, the photography parameter generation module, the image cropping module, and the rendering process are functional modules in the application framework layer.

[0038] In this way, first, the calculation process of the transition shooting parameters is completed in the application architecture layer, which reduces the interaction steps between the application layer and the application architecture layer, saves the time of the application layer sending the transition shooting parameters to the application architecture layer, and after the HAL layer returns the image frame, the application framework layer can promptly send the camera request carrying the transition shooting parameters to the HAL layer. Secondly, when the HAL layer returns the image frame to the application architecture layer, the application architecture layer can determine the transition shooting parameters in real time based on the return frame time of the image frame returned by the HAL layer, so that the smooth adjustment curve corresponding to the image frame returned by the HAL layer during the shooting parameter adjustment process conforms to the change law of the preset smooth adjustment curve, making the shooting parameter adjustment process smoother, without image jitter or freeze, and improving the user's visual experience.

[0039] In combination with the first aspect, in a possible implementation, the electronic device includes an application framework layer and a hardware abstraction layer; the camera module acquires an image frame based on a first transition shooting parameter, specifically including: in response to the return frame moment of the first image frame being earlier than the moment corresponding to the target shooting parameter, the application framework layer sends a first request to the HAL layer, the first request carries the first transition shooting parameter; the HAL layer sends the first request to the camera module; the first transition shooting parameter acquires a second image frame, specifically including: in response to the first request, the camera module acquires the second image frame based on the first transition shooting parameter

[0040] In this way, the application framework layer, HAL layer and hardware layer can cooperate with each other to complete the shooting parameter adjustment process.

[0041] In combination with the first aspect, in a possible implementation method, when the return frame moment of the first image frame is later than the moment corresponding to the target shooting parameters, the method also includes: the image cropping module determines whether the shooting parameters of the first image frame are the same as the target shooting parameters; when the shooting parameters of the first image frame are different from the target shooting parameters, the image cropping module processes the first image frame according to the shooting parameters of the first image frame and the target shooting parameters to obtain a third transition image frame; the rendering process renders the third transition image according to the vertical synchronization signal period of the rendering process; the electronic device displays the rendered third transition image in the camera application.

[0042] In this way, when the shooting parameters of the first image frame are inconsistent with the target shooting parameters, it means that the first image frame was not captured with the target shooting parameters. The electronic device can process the first image frame captured by the camera into a third transition image. The camera parameters of the third transition image are the same as the target shooting parameters. In this way, since the return time of the first image frame is later than the time corresponding to the target shooting parameters in the shooting parameter adjustment curve, the shooting parameter adjustment is completed, and the image frame corresponding to the shooting parameters of the first image frame has been displayed in advance. The electronic device can adjust the first image frame to the image frame corresponding to the target shooting parameters.

[0043] In combination with the first aspect, in a possible implementation, the method also includes: when the shooting parameters of the first image frame are the same as the target shooting parameters, the rendering process renders the first image frame according to the vertical synchronization signal period of the rendering process; the electronic device displays the rendered first image frame in the camera application.

[0044] In this way, when the shooting parameters of the first image frame are consistent with the target shooting parameters, it means that the first image frame is captured with the target shooting parameters, and the electronic device can directly render and display the first image frame.

[0045] In combination with the first aspect, in a possible implementation method, the image cropping module processes the first image frame according to the first transition shooting parameters and the shooting parameters of the first image frame to obtain the first transition image frame, specifically including the image cropping module changing the field of view FOV in the first image frame to obtain the first transition image frame.

[0046] In combination with the first aspect, in a possible implementation method, the image cropping module changes the field of view FOV in the first image frame to obtain a first transition image frame, specifically including: the image cropping module multiplies the field of view FOV of the first image frame by a first proportional coefficient to obtain the first transition image frame, wherein the first proportional coefficient is the ratio of the first transition shooting parameter to the shooting parameter of the first image frame.

[0047] In combination with the first aspect, in a possible implementation, the FOV of the first image frame is a first FOV, the FOV of the first transition image frame is a second FOV, and the second FOV is a product of the first FOV and a first proportional coefficient.

[0048] In this way, when the FOV of the first image frame and the first proportional coefficient are determined, the FOV of the second image frame can be obtained.

[0049] In combination with the first aspect, in a possible implementation, the size of the first image frame is a first size, and the image cropping module changes the field of view FOV in the first image frame to obtain a first transition image frame, specifically including: the image cropping module crops the first image frame from the first size to the second size to obtain a first image; the second size is the product of the first size and the second proportional coefficient, and the second proportional coefficient is the reciprocal of the first proportional coefficient; the image cropping module changes the first image at the second size from the first FOV to the second FOV; the image cropping module scales the first image from the second size to the first size to obtain a first transition image frame.

[0050] In this way, the first image frame can be processed into the second image frame through cropping and scaling.

[0051] With reference to the first aspect, in a possible implementation, the initial shooting parameter includes a first zoom ratio, and the target shooting parameter includes a second zoom ratio.

[0052] In combination with the first aspect, in a possible implementation, before the electronic device receives and responds to the user's first operation on the camera application, the method also includes: the camera application displays a first preview interface, the first preview interface includes a zoom ratio control, and the zoom ratio control indicates that the zoom ratio value of the image frame displayed in the first preview interface is a first zoom ratio; after the electronic device receives and responds to the user's first operation on the camera application, the method also includes: the camera application displays a second preview interface, the second preview interface includes a zoom ratio control, and the zoom ratio control indicates that the zoom ratio value of the image frame displayed in the second preview interface is a second zoom ratio.

[0053] In this way, the user can know the current zoom ratio from the zoom ratio control on the camera preview interface.

[0054] In combination with the first aspect, in a possible implementation, the method also includes: when the time interval between the first time and the display time of the previous image frame is less than m first vertical synchronization signal cycles, the electronic device stores the first image frame in a cache buffer queue of image frames to be displayed.

[0055] In this way, the display thread module can determine whether to send the first image frame to the rendering process module based on the vsync signal. If the time interval between the first time and the display time of the previous image frame is less than m first vertical synchronization signal periods, the electronic device does not render the first image frame first, thereby avoiding frame loss due to delayed rendering.

[0056] In combination with the first aspect, in a possible implementation, the electronic device also includes a display sending thread and a buffer manager, the buffer manager is used to temporarily store image frames, and after the image cropping module processes the first image frame according to the first transition shooting parameters and the shooting parameters of the first image frame to obtain the first transition image frame, the method also includes: when the number of image frames in the buffer manager is greater than or equal to a first threshold, the display sending thread sends the first transition image to the rendering process for rendering; when the number of image frames in the buffer manager is less than the first threshold, the buffer manager temporarily stores the first transition image frame.

[0057] In this way, when the number of image frames temporarily stored in the buffer manager is relatively small, the buffer manager can continue to temporarily store the first transitional image frame in the buffer queue. When the number of image frames temporarily stored in the buffer manager is large, the electronic device can directly render and display the first transitional image frame. This can prevent queue congestion caused by too many temporarily stored image frames, which could affect the smoothness of the electronic device's display of image frames.

[0058] In combination with the first aspect, in a possible implementation, after the buffer manager temporarily stores the first transition image frame, the method further includes: the display thread obtains the first transition image frame from the buffer manager.

[0059] In this way, the display sending process can obtain the first transition image frame in the buffer manager.

[0060] In combination with the first aspect, in a possible implementation, the electronic device further includes a vsync signal monitoring module, which is used to monitor the vsync signal of the rendering process module.

[0061] In this way, the vsync signal monitoring module can determine whether the rendering process module has completed rendering the image frame by monitoring the rendering process module.

[0062] In combination with the first aspect, in a possible implementation method, the method also includes: the vsync signal monitoring module sends the vsync signal to the display process module; when the time interval between the first time and the display time of the previous image frame is greater than or equal to m first vertical synchronization signal cycles, the rendering process renders the first transition image at the first time, specifically including: when the display process module determines based on the vsync signal that the time interval between the first time and the display time of the previous image frame is greater than or equal to m vertical synchronization vsync signal cycles, the rendering process renders the first transition image at the first time.

[0063] In this way, the display thread module can determine the timing of sending the first image frame to the rendering process module based on the vsync signal. The electronic device can process the rendering and display the first image frame through the rendering process module.

[0064] In combination with the first aspect, in a possible implementation, the value of m is determined by the frame rate of the electronic device and the periodic interval of the vsync signal.

[0065] In combination with the first aspect, in a possible implementation, the value of m is equal to the first value divided by the frame rate, and then divided by the vsync signal period interval.

[0066] In this way, the value of K can be determined.

[0067] In combination with the first aspect, in a possible implementation, the electronic device also includes a timer module, and the method also includes: the timer sets a first timestamp for the first transition image; when the display thread module determines that the time interval between the first timestamp and the timestamp of the previous image frame sent by the display process module to the rendering process is greater than a second threshold, the rendering process renders the first transition image according to the vertical synchronization signal period of the rendering process.

[0068] In this way, the display thread module can determine the timing of sending the image frame to the rendering process module based on the timestamp set by the timer, thereby keeping the time interval for sending the image frame to the rendering process module stable.

[0069] In combination with the first aspect, in a possible implementation, the shooting parameter includes any one of the following: zoom ratio, exposure, aperture, white balance, sharpness, contrast, and saturation.

[0070] In a second aspect, an electronic device is provided, which may include one or more cameras, a display, one or more processors and one or more memories; wherein the one or more cameras, the display, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method involved in any possible implementation method of the first aspect.

[0071] According to a third aspect, an electronic device is provided. The electronic device may include one or more functional modules, and the one or more functional modules are used for the method involved in any possible implementation of the first aspect.

[0072] In a fourth aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method involved in any possible implementation of the first aspect.

[0073] In a fifth aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the method involved in any possible implementation of the first aspect.

[0074] In a sixth aspect, a computer program product is provided. When the program product is run on an electronic device, the electronic device executes the method involved in any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] FIG1 is a schematic diagram showing a zoom smoothing adjustment curve;

[0076] FIG2 is a schematic diagram showing the interaction between the software and hardware architecture of a camera application for switching an initial zoom ratio to a target zoom ratio based on a smooth adjustment curve;

[0077] FIG3 is a schematic diagram showing a zoom curve of an image frame captured by the electronic device 100 during the zooming process;

[0078] FIG4 is a schematic diagram showing an electronic device 100 processing a camera request;

[0079] FIG5 shows a schematic diagram of an electronic device 100 rendering an image frame;

[0080] FIG6 shows a schematic structural diagram of the electronic device 100;

[0081] FIG7 shows a software structure block diagram of the electronic device 100;

[0082] 8A-8I are schematic diagrams showing the electronic device 100 adjusting the zoom ratio of a camera application based on user operations;

[0083] FIG9A shows a software architecture diagram of an electronic device 100 for capturing images at an initial zoom ratio according to an embodiment of the present application;

[0084] FIG9B shows a schematic flow chart of a photographing method provided by an embodiment of the present application;

[0085] FIG10 shows a timing diagram of an electronic device 100 rendering and displaying an image frame using a vsync signal cycle;

[0086] FIG11 shows a timing diagram of an electronic device 100 rendering and displaying image frames with different vsync signal periods;

[0087] FIG12A shows a software architecture diagram of another electronic device 100 for capturing images at an initial zoom ratio provided by an embodiment of the present application;

[0088] FIG12B shows a schematic diagram of a method flow of another shooting method provided by an embodiment of the present application;

[0089] FIG12C shows a schematic diagram of a method flow of another shooting method provided in an embodiment of the present application;

[0090] 13A-13D are schematic diagrams showing another zoom smooth adjustment curve;

[0091] FIG14A shows a schematic diagram of the electronic device 100 cropping an image;

[0092] FIG14B shows a schematic diagram of how another electronic device 100 processes a camera request and a zoom request;

[0093] FIG14C shows a timing diagram of another electronic device 100 rendering and displaying an image frame using a vsync signal cycle;

[0094] FIG15A shows a schematic diagram of a method flow of another shooting method provided in an embodiment of the present application;

[0095] FIG15B shows a schematic diagram of a method flow of another shooting method provided in an embodiment of the present application;

[0096] 16A-16B are schematic diagrams showing another zoom smoothing adjustment curve;

[0097] FIG17A shows a schematic diagram of how yet another electronic device 100 processes a camera request and a zoom request;

[0098] FIG17B shows a timing diagram of another electronic device 100 rendering and displaying an image frame using a vsync signal cycle;

[0099] FIG18 is a flow chart of a shooting parameter adjustment method provided in this application. DETAILED DESCRIPTION

[0100] The following is a clear and detailed description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship between related objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0101] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0102] The term "user interface (UI)" in the following embodiments of this application refers to the media interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of a wearable device.

[0103] When an electronic device displays the photo preview screen or video preview screen of a camera application, the user can change shooting parameters. Shooting parameters may include, but are not limited to, any of the following: focal length, exposure, aperture, white balance, sharpness, contrast, saturation, etc.

[0104] If a user operation to adjust a shooting parameter is detected, the electronic device may generate a smooth adjustment curve based on the initial shooting parameter and the target shooting parameter, and smoothly adjust the parameter to be adjusted from the initial shooting parameter to the target shooting parameter based on the smooth adjustment curve. The initial shooting parameter value refers to the shooting parameter value before the current shooting parameter adjustment operation is detected, and the target shooting parameter value refers to the shooting parameter value after the adjustment trigger operation is detected.

[0105] It is understood that the smooth adjustment curve may include initial shooting parameters, one or more transition shooting parameters, and target shooting parameters. The electronic device can sequentially obtain a preview image frame using the initial shooting parameters, one or more transition shooting parameters, and the target shooting parameters. This avoids the situation where the electronic device directly adjusts the shooting parameters to the target shooting parameters, resulting in a sudden change in the image.

[0106] Illustratively, the shooting parameter may be a focal length, the initial shooting parameter may be an initial zoom ratio, and the target shooting parameter may be a target zoom ratio.

[0107] FIG1 shows a schematic diagram of a zoom smoothing adjustment curve.

[0108] As shown in Figure 1, upon detecting a user operation to adjust the focal length, the electronic device can generate a zoom smoothing adjustment curve as shown in Figure 1 based on the initial zoom ratio and the target zoom ratio. The Bezier curve shown in Figure 1 is a curve obtained by starting with the initial zoom ratio and ending with the target zoom ratio. The Bezier curve includes one or more transition zoom ratios. For example, the Bezier curve shown in Figure 1 includes transition zoom ratio 1, transition zoom ratio 2, and transition zoom ratio 3.

[0109] In response to detecting a user operation of adjusting the focal length, the electronic device can obtain a preview image frame through transition zoom ratio 1, transition zoom ratio 2, transition zoom ratio 3 and target zoom ratio in sequence, thereby achieving a smooth transition from the initial zoom ratio to the target zoom ratio.

[0110] Currently, electronic devices use camera applications to smoothly adjust parameters to be adjusted from initial shooting parameters to target shooting parameters.

[0111] FIG2 shows a schematic diagram of the software and hardware architecture interaction of a camera application switching an initial zoom ratio to a target zoom ratio based on a smooth adjustment curve.

[0112] As shown in Figure 2, a layered architecture divides the system into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the system is divided into four layers: application layer, application framework layer, hardware abstraction layer, and hardware layer, from top to bottom.

[0113] The application layer may include a series of application packages, which may include a camera application, etc.

[0114] The camera application integrates a camera request sending module and a timing module.

[0115] The camera request sending module is used to send camera requests and focus adjustment requests to the application framework layer. Camera requests may include but are not limited to preview requests, photo taking requests, etc.

[0116] The preview request may be sent after the camera application is started. The photo request may be sent after the camera application detects a photo operation. The focus adjustment request may be sent after the camera application detects a focus adjustment operation.

[0117] The timing module is used to obtain the frame interval and, after detecting the user operation of adjusting the focus, determine one or more transition zoom ratios based on the frame interval and the Bezier curve, so that the camera request sending module can determine the number of times to send the focus adjustment request to the program framework layer based on the one or more transition zoom ratios.

[0118] The application framework layer (Frameworks, FWK) provides the application programming interface (API) and programming framework for the application packages in the application layer. The application framework layer also includes some predefined functions.

[0119] In some embodiments, the application framework layer may further include a request queue (RequestQueue) module, and the request queue module may include camera requests and focus adjustment requests sent by the camera application.

[0120] In some embodiments, the application framework layer may further include a request queue processing (RequestThread) module. The request queue processing module may include a wait for request (WaitForRequest) module, a send batch request (SendRequestsBatch) module, and a prepare hardware abstraction layer request (PrepareHardwareAbstractLayerRequest, which may be referred to as PrepareHALRequest) module.

[0121] Among them, the WaitForRequest module can be used to keep the camera request to be executed in the request queue module waiting when there is no empty buffer in the N cache buffers allocated to the camera request of the camera application in the buffer area of ​​the electronic device. The PrepareHALRequest module can be used to construct the Capture request of the HAL layer and the output buffer outputBuffers. When there is an empty buffer in the N cache buffers allocated to the camera request of the camera application, the PrepareHALRequest module can obtain the buffer from the Allocate Buffer interface for the camera request sent by the camera application. The Send batch request module can be used to send the Capture request of the HAL layer constructed by the PrepareHALRequest module to the HAL layer.

[0122] For example, N can be 8, meaning that the number of buffers allocated by the electronic device to the camera application for camera requests can be 8. N can also be other values, such as 7 or 9, and the present application does not limit the value of N. The following description uses the example of N being 8 as an example.

[0123] In some embodiments, the application framework layer may further include an Allocate Buffer interface for allocating a buffer for a camera request issued by a camera application. For example, the Allocate Buffer interface may specify the address of a buffer for storing the camera request.

[0124] In some embodiments, the application framework layer may further include a result processing module configured to receive an image frame sent by the HAL layer, or a buffer address storing an image frame. The result processing module may also send the image frame or the buffer address storing the image frame to the camera application.

[0125] In some embodiments, the application framework layer may further include a rendering process module. The rendering process module may be used to render the image frames sent by the camera application and send the rendered image frames to the camera application, which may then display the rendered results.

[0126] In some embodiments, the application framework layer may further include a frame return data processing module. The result processing module is used to receive the image frame sent by the HAL layer, or after storing the buffer address of the image frame, send the frame return time of the image frame to the frame return data processing module. The frame return data processing module can determine the frame interval based on the frame return time of the image frame. The frame return time of the image frame can refer to the moment when the application framework layer or the result processing module in the application framework layer receives the image frame uploaded by the HAL layer, and the frame interval can refer to the time difference between the frame return times of two adjacent image frames.

[0127] In some embodiments, the application framework layer may also be referred to as the application framework layer.

[0128] The hardware abstraction HAL layer is an interface layer located between the application framework layer and the hardware layer, providing a virtual hardware platform for the operating system.

[0129] The Hardware Abstraction Layer (HAL) receives camera requests from the application framework layer and sends the camera parameters and buffer address in the camera request to the camera module. It also calls the result-returning interface (for example, ProcessCaptureResult) to send image frames captured by the camera module according to the camera parameters to the result processing module in the application framework layer.

[0130] The hardware layer may include a camera module, an image signal processor, a display, and the like. The camera module may include one or more camera image sensors (e.g., image sensor 1, image sensor 2, etc.). Optionally, the camera module may also include a time of flight (TOF) sensor, a multispectral sensor, and the like. The image signal processor may be used to process image frames captured by the camera module.

[0131] In combination with the above system structure, the following describes in detail the processing mechanism of the electronic device 100 for a camera request generated based on a user's operation for changing the focus, taking zoom as an example.

[0132] 1. When the electronic device 100 receives an operation from the user to start the camera application, the electronic device 100 starts the camera application. The camera application can issue multiple camera requests for obtaining preview images. The first camera request for obtaining preview images issued by the camera application can fill up the N buffers reserved for camera requests in the cache area.

[0133] Optionally, after the camera application is started, if the shooting parameters have not changed, the camera application may only send a camera request to the application framework layer once.

[0134] For the process of how the electronic device 100 processes the camera request sent by the camera application and sends the image frame for display, please refer to the process description of how the electronic device 100 processes the focus adjustment request and sends the image frame for display in steps 7 to 16. This application will not go into details here.

[0135] 2. The application framework layer of the electronic device 100 obtains the return frame time of the image frame.

[0136] When the application framework layer of the electronic device 100 obtains the image frame uploaded by the HAL layer, it can determine the frame return time of the image frame based on the time when the image frame was uploaded by the HAL layer.

[0137] The application framework layer of the electronic device 100 can determine the frame interval based on the frame return time of two adjacent image frames.

[0138] 3. The camera application obtains the frame interval.

[0139] In some embodiments, the frame interval may refer to an average frame interval, i.e., an average of multiple frame intervals. In other embodiments, the frame interval may refer to a real-time frame interval, which may refer to the time difference between the return time of the most recently returned image frame and the return time of the immediately preceding returned image frame.

[0140] 4. The camera app receives the zoom operation.

[0141] The camera application can receive and respond to the user's zoom operation, determine the initial zoom ratio and the target zoom ratio, and then determine a zoom smoothing adjustment curve based on the initial zoom ratio and the target zoom ratio. The zoom smoothing adjustment curve shows the corresponding relationship between time and zoom ratio.

[0142] 5. The camera application determines one or more transition zoom ratios based on the frame interval and the zoom smoothing curve.

[0143] After obtaining the frame interval and the zoom smoothing adjustment curve, the camera application can determine one or more transition zoom ratios based on the frame interval and the zoom smoothing adjustment curve. This allows the camera application to determine the number of focus adjustment requests to be sent to the application framework layer based on the one or more transition zoom ratios.

[0144] For example, as shown in FIG1 , time t1 may be the time when the camera application receives a zoom operation, and time t2 may be the end time of the zoom smoothing adjustment curve. After time t1, the camera application may determine time t3, time t4, and time t5 based on the first frame interval. Transitional zoom magnifications 1, 2, and 3 may be determined based on time t3, time t4, and time t5 and the zoom smoothing adjustment curve, respectively.

[0145] The camera application can sequentially send focus adjustment request 1 including transition zoom ratio 1, focus adjustment request 2 including transition zoom ratio 2, focus adjustment request 3 including transition zoom ratio 3, and focus adjustment request 4 including the target zoom ratio to the application framework layer, so that the camera module can capture and display image frames based on the transition zoom ratios.

[0146] 6. The camera application sends a focus adjustment request 1 to the application framework layer.

[0147] The electronic device 100 may receive a user's operation to change the focus of the camera application. Based on this operation, after the application framework layer receives the image frame returned by the HAL layer, the camera application in the electronic device 100 may send a focus adjustment request 1 to the application framework layer.

[0148] 7. The request queue processing module obtains the cache get buffer.

[0149] In some embodiments, the application framework layer may include an Allocate Buffer interface. In response to the focus adjustment request 1 sent by the camera application, the HAL request preparation module may send a buffer acquisition request to the Allocate Buffer interface to obtain a free buffer and allocate the free buffer for the focus adjustment request 1.

[0150] 8. The application framework layer allocates a buffer for focus adjustment request 1.

[0151] When there is an empty buffer among the N buffers reserved for camera requests in the cache area of ​​electronic device 100, electronic device 100 can send focus adjustment request 1 to the preparation HAL request module in the request queue processing module. After obtaining the Allocate Buffer interface to allocate a buffer address for focus adjustment request 1, the preparation HAL request module can store focus adjustment request 1 in the buffer.

[0152] 9. The application framework layer sends a focus adjustment request to the HAL layer.

[0153] After the application framework layer allocates a buffer for the focus adjustment request 1, the application framework layer sends the focus adjustment request 1 to the HAL layer, and the focus adjustment request 1 carries a transition zoom ratio 1. The focus adjustment request 1 may also carry other data, which is not limited in this application.

[0154] Specifically, the HAL request preparation module can construct the focus adjustment request 1 into a HAL layer Capture request and send it to the HAL layer through the batch request sending module. The HAL layer then sends the camera parameters in the Capture request and the buffer address of the Capture request to the camera module. The camera parameters in the Capture request can include, but are not limited to, the transition zoom factor 1.

[0155] 10. The HAL layer sends a focus adjustment request to the camera module.

[0156] In response to the focus adjustment request sent by the application framework layer, the HAL layer then sends the focus adjustment request to the camera module.

[0157] 11. The electronic device 100 turns on the camera, obtains the image frame captured by the camera based on the focus adjustment request, and sends the image frame to the HAL layer.

[0158] The camera module in the electronic device 100 can receive the focus adjustment request and the buffer address of the focus adjustment request from the HAL layer. The camera module can then acquire image frames according to the camera parameters (e.g., a transition zoom factor of 1) and store the acquired image frames in the buffer corresponding to the buffer address carried in the focus adjustment request.

[0159] In some examples, the image frames captured by the camera module can be transmitted to the image signal processor, which can pre-process the image frames and upload them to the HAL layer through the camera driver or the image processor driver.

[0160] 12. The HAL layer uploads the image frame to the result processing module in the application framework layer.

[0161] After receiving the image frame uploaded by the image signal processor, the HAL layer uploads the image frame to the result processing module in the application framework layer.

[0162] 13. The result processing module uploads the image frame to the camera application.

[0163] In response to the image frame sent by the HAL layer, the result processing module uploads the image frame and / or the buffer address of the image frame to the camera application.

[0164] 14. The camera application sends a display instruction to the rendering process Surface flinger.

[0165] 15. The rendering process sends the rendered image frames to the camera application.

[0166] 16. The camera application sends the image frame for display.

[0167] After obtaining the image frame and / or its buffer address, the camera application can send the image frame to the rendering process module for rendering. The rendering process module can obtain the rendered image frame and then send the rendered image frame to the camera application. The camera application then sends the rendered image frame to the display for display.

[0168] 17. The electronic device 100 clears the buffer storing the focus adjustment request.

[0169] In some embodiments, after the display successfully displays the image frame, it notifies the result processing module that the image frame has been successfully displayed. It will be appreciated that after the image frame is displayed, the buffer in the electronic device's cache storing the image frame and the focus adjustment request used to obtain the image frame is cleared. The result processing module can send the address of the cleared buffer to the Allocate Buffer interface. The Allocate Buffer interface can subsequently assign the cleared buffer address to the next focus adjustment request.

[0170] In some embodiments, the result processing module will also send the return frame time of the image frame to the return frame data processing module, so that the return frame data processing module can statistically obtain the frame interval of multiple image frames displayed by the electronic device 100.

[0171] After the HAL layer sends the image frame acquired in response to focus adjustment request 1 to the application framework layer, the application layer can then send focus adjustment request 2 to the application framework layer, and then follow steps 7-16 to acquire the image frame corresponding to focus adjustment request 2. This process continues until the application framework layer acquires the image frame corresponding to focus adjustment request 4, at which point the application layer stops sending focus adjustment requests to the application framework layer. Afterwards, electronic device 100 continues to capture and display image frames at the target zoom factor.

[0172] However, the processing process of FIG1 has the following three defects.

[0173] Defect 1: The actual zoom curve deviates from the ideal zoom curve, and the zoom process is not smooth.

[0174] As shown in Figure 1, the electronic device 100 smoothly adjusts the zoom magnification from the initial zoom magnification to the target zoom magnification through the camera application. The camera application determines one or more transitional zoom magnifications based on the frame interval and the zoom smoothing adjustment curve. For example, the camera application can determine time t10, time t11, and time t12 based on the frame interval, and determine transitional zoom magnification 1, transitional zoom magnification 2, and transitional zoom magnification 3 based on time t10, time t11, and time t12, respectively. During the zooming process, the camera module can capture image frames based on the one or more transitional zoom magnifications issued by the camera application and return them to the HAL layer, which then returns the image frames to the application framework layer.

[0175] During the zoom process, based on the zoom magnification and return time of the image frame returned by the HAL layer to the application framework layer, the HAL layer can obtain the zoom curve shown in Figure 3. The zoom curve shown in Figure 3 reflects the zoom process of the image frame collected by the electronic device 100 during the zoom process.

[0176] Based on the above analysis, it can be seen that the camera application determines one or more transitional zoom ratios based on the frame interval and the zoom smoothing curve. However, due to hardware performance limitations and unstable image acquisition and processing algorithms, the camera application can no longer adjust the transitional zoom ratio based on the real-time image frame return time. Furthermore, it takes time for the application layer to send a zoom request to the application framework layer. After the HAL layer returns the image frame, the application framework layer cannot promptly send the zoom request to the HAL layer. Furthermore, the time required for the application layer to send a zoom request to the application framework layer and the time required for the application framework layer to send a zoom request to the HAL layer are both unstable. Ultimately, the time when the HAL layer returns the image frame in response to the zoom request and the corresponding transitional zoom ratio will deviate from the zoom smoothing curve. The HAL layer ultimately obtains the zoom curve shown in Figure 3. This deviates from the zoom smoothing curve, which manifests as image delay and jitter on the display, affecting the user's visual experience.

[0177] Defect 2: There is a delay in the electronic device responding to the user's zoom operation. It will respond to the user's zoom operation after a certain period of time after receiving the user's zoom operation.

[0178] As shown in Figure 2, the application framework periodically sends camera requests to the HAL layer. Before the HAL layer responds to the zoom request corresponding to the user's zoom operation, it must process other pending camera requests. The HAL layer only processes the zoom request after all previous camera requests have been processed.

[0179] For example, as shown in Figure 4, before the HAL layer processes the zoom request, the HAL layer has six other camera requests to process: camera request 1, camera request 2, camera request 3, camera request 4, camera request 5, and camera request 6. If the HAL layer receives zoom request 1 at time tms, it places zoom request 1 in the pending request queue in chronological order. According to the first-in-first-out principle, at time tms, the HAL layer removes camera request 1 from the queue and processes it. In response to processing camera request 1, the camera module can capture image frame A based on the initial zoom factor and send it to the application framework layer through the HAL layer. The application framework layer then sends image frame A to the camera application, which can then display image frame A.

[0180] If the HAL layer takes 33.3ms to process a camera request, the HAL layer receives zoom request 1 at (t+33.3)ms and places zoom request 2 in the pending request queue in chronological order. Following the first-in-first-out principle, at (t+33.3)ms, the HAL layer removes camera request 2 from the queue and processes it. In response to processing camera request 2, the camera module captures image frame B based on the initial zoom factor and sends it to the application framework layer through the HAL layer. The application framework layer then sends image frame B to the camera application, which then displays image frame B.

[0181] Similarly, at (t+66.6) ms, the HAL layer receives zoom request 3 and places it in the pending request queue in chronological order. Following the first-in-first-out principle, at (t+66.6) ms, the HAL layer removes camera request 3 from the queue and processes it. In response to processing camera request 3, the camera module captures image frame C based on the initial zoom factor and sends it to the application framework layer through the HAL layer. The application framework layer then sends image frame C to the camera application, which then displays it.

[0182] Similarly, at (t+99.9) ms, the HAL layer receives zoom request 4 and places it in the pending request queue in chronological order. Following the first-in-first-out principle, at (t+99.9) ms, the HAL layer removes camera request 4 from the queue and processes it. In response to processing camera request 4, the camera module captures image frame D based on the initial zoom factor and sends it to the application framework layer via the HAL layer. The application framework layer then sends image frame D to the camera application, which then displays it.

[0183] Similarly, at (t+133.2) ms, the HAL layer receives zoom request 5 and places it in the pending request queue in chronological order. Following the first-in-first-out principle, at (t+133.2) ms, the HAL layer removes camera request 5 from the queue and processes it. In response to processing camera request 5, the camera module captures image frame E based on the initial zoom factor and sends it to the application framework layer via the HAL layer. The application framework layer then sends image frame E to the camera application, which then displays it.

[0184] Similarly, at (t+166.5) ms, the HAL layer receives zoom request 6 and places it in the pending request queue in chronological order. Following the first-in, first-out principle, at (t+166.5) ms, the HAL layer removes camera request 6 from the queue and processes it. In response to processing camera request 6, the camera module captures image frame F based on the initial zoom factor and sends it to the application framework layer via the HAL layer. The application framework layer then sends image frame F to the camera application, which then displays it.

[0185] Similarly, at (t+199.8) ms, the HAL layer receives Zoom Request 7 and places it in the pending request queue in chronological order. Following the first-in-first-out principle, at (t+199.8) ms, the HAL layer removes Zoom Request 1 from the queue and processes it. In response to processing Zoom Request 7, the camera module captures image frame G based on the initial zoom factor and sends it to the application framework layer via the HAL layer. The application framework layer then sends image frame G to the camera application, which then displays it.

[0186] From the analysis of Figure 4, it can be seen that the HAL layer will respond to the zoom request corresponding to the user's zoom operation 199.8ms after receiving the user's zoom operation, and zoom lag will occur, which should be experienced by the user.

[0187] Defect 3. During the zoom process, from the time the electronic device 100 sends a zoom request from the camera application to the time it obtains the image frame and sends the image frame to the rendering process module for rendering, the processing time for each zoom request in the electronic device 100 may be different, and frame loss may occur, resulting in uneven time for the electronic device 100 to display the picture frame, which will cause the zoom preview picture displayed by the electronic device to be stuck during the zoom process, thereby affecting the user experience.

[0188] For example, as shown in Figure 5, taking the rendering cycle of the rendering process module as 33.3ms as an example, it shows a scenario in which the rendering process module discards the image frames when the speed at which the electronic device 100 continuously sends image frames to the rendering process module exceeds the rendering capability of the electronic device 100.

[0189] As shown in Figure 5, image frame A is sent to the rendering process module. After image frame A is rendered, the electronic device can display image frame A. Then, image frame B can be sent to the rendering process module. After image frame B is rendered, the electronic device can display image frame B. Then, image frame C can be sent to the rendering process module. After image frame C is rendered, the electronic device can display C. Then, image frame D can be sent to the rendering process module. After image frame D is rendered, the electronic device can display image frame D. When the rendering process module is still processing image frame D, image frame E has already been sent to the rendering process module, and the rendering process module will discard image frame E. After image frame D is successfully displayed, the electronic device 100 can wait for image frame F. After image frame F is sent to the rendering process module, the rendering process module can render image frame F and send image frame F for display.

[0190] 5 , when the electronic device 100 displays image frame D, since image frame E is discarded, the electronic device 100 may freeze while waiting for image frame F to be displayed, resulting in uneven display time for each image frame.

[0191] Based on the above analysis, this application provides a method for adjusting shooting parameters. The method includes the following steps:

[0192] Step 1: The electronic device 100 receives a user operation to change shooting parameters and determines the initial shooting parameters and the target shooting parameters.

[0193] When the electronic device 100 displays the camera application's photo preview interface or video preview interface, the user can change shooting parameters from initial shooting parameters to target shooting parameters. Shooting parameters may include, but are not limited to, any of the following: focal length, exposure, aperture, white balance, sharpness, contrast, saturation, etc.

[0194] The initial shooting parameter value refers to the shooting parameter value before the current shooting parameter adjustment operation is detected, and the target shooting parameter value refers to the shooting parameter value after the adjustment trigger operation is detected.

[0195] Exemplarily, the shooting parameter may be focal length, and the electronic device 100 may increase the zoom ratio of the camera application based on user operation. For example, if the electronic device 100 increases the zoom ratio from 1X to 5X based on user operation, the initial zoom ratio is 1X and the target zoom ratio is 5X.

[0196] The electronic device 100 may also reduce the zoom ratio of the camera application based on user operation. For example, if the electronic device 100 reduces the zoom ratio from 1X to 0.5X based on user operation, the initial zoom ratio is 1X and the target zoom ratio is 0.5X.

[0197] Step 2: The electronic device 100 obtains a smooth adjustment curve based on the initial shooting parameters and the target shooting parameters.

[0198] It is understood that the smooth adjustment curve may include initial shooting parameters, one or more transition shooting parameters, and target shooting parameters. The electronic device can sequentially obtain a preview image frame using the initial shooting parameters, one or more transition shooting parameters, and the target shooting parameters. This avoids the situation where the electronic device directly adjusts the shooting parameters to the target shooting parameters, resulting in a sudden change in the image.

[0199] Exemplarily, when the shooting parameter is a zoom ratio, the electronic device 100 can obtain a focus smoothing adjustment curve based on the initial zoom ratio and the target zoom ratio. The focus smoothing adjustment curve is a curve starting from the initial zoom ratio and ending at the target zoom ratio. The initial zoom ratio corresponds to time t1, and the target zoom ratio corresponds to time t2, t2 is later than t1, where t1 is the moment when the user chooses to switch to the target zoom ratio. The focus smoothing adjustment curve is used for the electronic device 100 to obtain multiple image frames through the initial zoom ratio, one or more zoom ratios and the target zoom ratio in sequence, so as to achieve smooth switching from the initial zoom ratio to the target zoom ratio.

[0200] Step 3: In response to a user operation to change shooting parameters, the electronic device 100 obtains a return frame time of a first image frame, where the first image frame is an image captured by the camera, and determines transition shooting parameter 1 from the smooth adjustment curve based on the return frame time of the first image frame. The electronic device 100 determines a cropping ratio 1 based on the shooting parameters of the first image frame (e.g., initial shooting parameters) and transition shooting parameter 1, crops the first image frame based on the cropping ratio 1 to obtain transition image frame 1, and displays transition image frame 1.

[0201] Afterwards, the electronic device 100 can obtain the return frame time of the second image frame, which is the image captured by the camera, and then process the second image frame according to the processing method of the first image frame. This application will not go into details here.

[0202] In some embodiments, the electronic device 100 may also determine transition shooting parameters 2 from the smooth adjustment curve based on the return frame time and the preset duration of the first image frame. The electronic device 100 determines cropping ratio 2 based on the shooting parameters of the first image frame (e.g., initial shooting parameters) and transition shooting parameters 2, crops the first image frame based on the cropping ratio 2 to obtain transition image frame 2, and displays transition image frame 2.

[0203] Optionally, the preset duration may be related to the image output frame rate of the camera application and the number of interpolated frames a, for example, the preset duration = (a+1) / (1 / (image output frame rate). For example, if the image output frame rate of the camera application is 30 pfs, that is, the HAL layer returns an image frame to the application framework layer every 33.3 ms, when a is 1, the preset duration may be 16.7 ms.

[0204] In response to determining the transition shooting parameter 2, the electronic device 100 can modify the vsync signal period from the first value to the second value, where the second value is greater than the first value, thereby increasing the display frame rate of the electronic device 100, so that the electronic device 100 can synthesize and display multiple transition image frames in a timely manner.

[0205] In some embodiments, after the transition image frame is displayed, the electronic device 100 changes the period of the vsync signal from the second value to the first value.

[0206] In some embodiments, the electronic device 100 can calculate the rendering timing of each transition image frame based on the vsync signal period, so that the rendering duration of each transition image frame is fixed and uniform, so that the frame rate of the picture seen by the user is stable.

[0207] Through the above method, on the one hand, the electronic device 100 can calculate one or more transition shooting parameters in real time, so that the electronic device 100 can promptly capture and display image frames according to one or more transition shooting parameters, so that the change of shooting parameters conforms to the preset smooth adjustment curve, and the shooting parameters can smoothly transition to the target shooting parameters, avoiding picture jitter or freeze, thereby improving the user's visual experience.

[0208] On the other hand, by copying and cropping image frames, the electronic device 100 can quickly display image frames in response to user operations to change shooting parameters without the problem of delayed response.

[0209] On the other hand, in the process of changing the shooting parameters, the electronic device 100 can control and change the vsync signal period, and associate the display timing of the transition image frame with the vsync signal period, ensuring that the electronic device 100 displays the image frame evenly, thereby improving the user's visual experience.

[0210] The following describes the hardware structure of an electronic device 100 provided in an embodiment of the present application.

[0211] FIG6 shows a schematic structural diagram of the electronic device 100 .

[0212] The following embodiments are described in detail using electronic device 100 as an example. It should be understood that the electronic device 100 shown in FIG6 is merely an example, and that electronic device 100 may have more or fewer components than those shown in FIG6 , may combine two or more components, or may have a different component configuration. The various components shown in FIG6 may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0213] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0214] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0215] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0216] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0217] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0218] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0219] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby enabling the touch function of the electronic device 100.

[0220] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0221] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0222] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0223] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0224] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0225] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0226] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0227] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0228] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0229] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0230] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0231] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0232] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0233] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0234] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0235] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0236] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display screen panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniLED, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0237] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0238] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0239] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0240] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0241] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0242] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0243] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0244] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0245] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0246] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0247] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0248] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0249] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0250] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0251] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0252] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0253] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0254] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0255] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0256] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0257] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0258] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0259] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0260] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0261] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0262] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0263] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0264] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0265] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0266] The SIM card interface 195 is used to connect a SIM card.

[0267] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0268] FIG7 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present invention.

[0269] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, from top to bottom, these layers are the application layer, application framework layer, HAL layer, driver layer, and hardware layer.

[0270] The application layer can include a series of application packages.

[0271] As shown in FIG7 , the application package may include a camera application, and the application package may also include gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications, which are not shown in FIG7 .

[0272] The camera application includes, but is not limited to, a camera request sending module. This module can be used to send camera requests and focus adjustment requests to the application framework layer. Camera requests can include, but are not limited to, preview requests and photo requests.

[0273] The preview request may be sent after the camera application is started. The photo request may be sent after the camera application detects a photo operation. The focus adjustment request may be sent after the camera application detects a focus adjustment operation.

[0274] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes a number of predefined functions. The application framework layer may include a window manager, content provider, view system, telephony manager, resource manager, and notification manager. The window manager manages window programs. It can obtain the display screen size, determine whether a status bar is present, lock the screen, and take screenshots. The content provider stores and retrieves data and makes it accessible to applications. This data may include video, images, audio, incoming and outgoing calls, browsing history and bookmarks, and the phone book. The view system includes visual controls, such as those that display text and images. The view system can be used to build applications. A display interface may consist of one or more views. For example, a display interface containing a text message notification icon may include a view that displays text and a view that displays images. The telephony manager provides communication functions for electronic device 100, such as managing call status (including connected and ended calls). The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, and video files. The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0275] As shown in Figure 7, the application framework layer also includes a shooting parameter generation module, a request queue processing module, an allocation buffer interface, a result processing module, an image cropping module, a vsync signal monitoring module, a display thread, a timer, a buffer manager, and a rendering process surface flinger.

[0276] The shooting parameter generation module is configured to generate a smooth adjustment curve based on the initial shooting parameters and the target shooting parameters sent by the camera application when detecting that the user has adjusted the shooting parameters. The module also determines one or more transition shooting parameters based on the image frame return time and the smooth adjustment curve.

[0277] The shooting parameter generation module is also used to send a transition shooting parameter to the HAL layer when it detects that the HAL layer uploads an image frame to the result processing module. The HAL layer then sends the transition shooting parameter to the camera module, so that the camera module can obtain the image frame based on the transition shooting parameter.

[0278] In some embodiments, the shooting parameter generation module may also be located in the HAL layer. This application does not limit the location of the shooting parameter generation module.

[0279] For the functions of the request queue processing module and the allocation buffer area interface, please refer to the description in the embodiment of Figure 2, and this application will not go into details here.

[0280] The result processing module is used to receive the image frame and the shooting parameters of the image frame sent by the HAL layer, or store the buffer address of the image frame and the shooting parameters of the image frame.

[0281] The result processing module is further used to send the image frame and the shooting parameters of the image frame, or the buffer address storing the image frame and the shooting parameters of image frame A to the image cropping module.

[0282] In some embodiments, the result processing module is further configured to store the image frame and the shooting parameters of the image frame, or the buffer address storing the image frame and the shooting parameters of the image frame in the buffer manager.

[0283] The image cropping module is configured to obtain the image frame's return time and send it to the shooting parameter generation module. The shooting parameter generation module can obtain transition shooting parameter 1 based on the image frame's return time and the smoothing adjustment curve. The image cropping module then determines a cropping ratio based on the image frame's shooting parameters and transition shooting parameter 1. Based on the cropping ratio, the image frame received by the result processing module is cropped to obtain transition image frame 1. The transition image frame 1, or the buffer address storing the transition image frame 1, is then sent to the buffer manager.

[0284] In some embodiments, the image cropping module is further configured to determine transition shooting parameter 2 from the smooth adjustment curve based on the image frame's return frame time and a preset duration. The image cropping module determines cropping ratio 2 based on the image frame's shooting parameters and transition shooting parameter 2, and crops the image frame based on cropping ratio 2 to obtain transition image frame 2. The image cropping module then sends transition image frame 2 or the buffer address storing transition image frame A2 to the buffer manager.

[0285] The preset duration may be related to the output frame rate of the camera application and the value of the number of interpolated frames a, for example, the preset duration = (a+1) / (1 / (output frame rate). For example, if the output frame rate of the camera application is 30 pfs, that is, the HAL layer returns an image frame to the application framework layer every 33.3 ms, when a is 1, the preset duration may be 16.7 ms.

[0286] For another example, if the output frame rate of the camera application is 60pfs, that is, the HAL layer returns an image frame to the application framework layer every 16.7ms, when a is 1, the preset duration can be 8.35ms.

[0287] The buffer manager is used to store image frames to be displayed or to store the buffer address of image frames to be displayed.

[0288] The buffer manager is further configured to send the stored image frame or the buffer address storing the image frame to the display sending thread when the display sending thread obtains the image frame.

[0289] The display sending thread is used to determine the timing of obtaining the next image frame to be sent for display from the buffer manager based on the vsync signal period or the timer period.

[0290] Specifically, in one possible implementation, the display thread can be configured to obtain the next image frame to be displayed from the buffer manager after m vsync signal cycles have elapsed since the previous image frame was displayed, and send the obtained image frame to the camera application. The camera application can then send the obtained image frame to the rendering process module.

[0291] Optionally, in one possible implementation, the display thread may also obtain the next image frame to be displayed from the buffer manager after a timer display period has elapsed since the previous image frame was displayed, and send the obtained image frame to the camera application. The camera application may send the obtained image frame to the rendering process module.

[0292] The vsync signal monitoring module is used to monitor the vsync signal cycle in the rendering process and send the vsync signal cycle to the display process.

[0293] The timer can be used to send a timer period signal to the display thread, wherein the period of the timer can be determined by the screen refresh frame rate of the electronic device.

[0294] The rendering process module is used to render the image frames sent by the camera application and send the rendering results (i.e., the rendered image frames) to the camera application. The camera application can display the rendering results.

[0295] In some embodiments, the application framework layer may also be referred to as the application framework layer.

[0296] The hardware abstraction HAL layer is an interface layer located between the application framework layer and the hardware layer, providing a virtual hardware platform for the operating system.

[0297] The HAL layer receives camera requests from the application framework layer and sends the camera parameters and buffer address in the camera request to the camera module. It also calls the result-returning interface (for example, ProcessCaptureResult) to send image frames captured by the camera module according to the camera parameters to the result processing module in the application framework layer.

[0298] Optionally, the HAL layer may store the image frame acquired by the camera module into the buffer address of the corresponding camera request, and send the image frame or the buffer address of the image frame to the result processing module in the application framework layer.

[0299] In some embodiments, the HAL layer is further configured to send the camera parameters of the image frame to the result processing module. For example, the camera parameters may be shooting parameters corresponding to the image frame, such as initial shooting parameters, transition shooting parameters, or target shooting parameters.

[0300] In some embodiments, the HAL layer is further configured to send a frame return time of the image frame to the result processing module, wherein the frame return time of the image frame is the time when the HAL layer sends the image frame to the result processing module.

[0301] The driver layer includes but is not limited to the camera driver and the display driver. The camera driver is used to drive the camera module to capture image frames, and the display driver is used to drive the display to display image frames.

[0302] The hardware layer may include a camera module, an image signal processor, a display, and the like. The camera module may include one or more camera image sensors (e.g., image sensor 1, image sensor 2, etc.). Optionally, the camera module may also include a time of flight (TOF) sensor, a multispectral sensor, and the like. The image signal processor may be used to process image frames captured by the camera module. The display may be used to display image frames sent by the camera application.

[0303] Next, the electronic device 100 provided in an embodiment of the present application will be described with reference to a UI diagram to adjust shooting parameters based on user operations.

[0304] The embodiment of the present application is described by taking the zoom ratio as an example of a shooting parameter.

[0305] 8A-8I are schematic diagrams showing the electronic device 100 adjusting the zoom ratio of a camera application based on user operations.

[0306] In some implementations, the electronic device 100 may receive a user operation to increase the zoom magnification of the camera application. In other implementations, the electronic device 100 may receive a user operation to decrease the zoom magnification of the camera application.

[0307] Increase the zoom ratio of the camera app

[0308] 8A-8E are schematic diagrams showing the electronic device 100 increasing the zoom ratio of a camera application based on a user operation.

[0309] As shown in Figure 8A, the electronic device 100 can display a desktop, in which a page with application icons is displayed, and the page includes multiple application icons (for example, a settings application icon, an application market application icon, a gallery application icon, a browser application icon, etc.). A page indicator is also displayed below the multiple application icons to indicate the positional relationship between the currently displayed page and other pages. A tray area is displayed below the page indicator. Among them, the tray area includes multiple tray icons, for example, a camera application icon, an address book application icon, a phone application icon, and a message application icon. The tray area remains displayed when the page switches. In some embodiments, the above-mentioned page may also include multiple application icons and a page indicator. The page indicator may not be part of the page and may exist separately. The above-mentioned tray icon is also optional, and the embodiments of the present application are not limited to this.

[0310] The electronic device 100 may receive user input (e.g., a single click) on the camera application icon. In response to the input operation, the electronic device 100 may display a user interface as shown in FIG8B . FIG8B is a user interface for a capture and display service provided by the electronic device 100 according to an embodiment of the present application, which may also be referred to as a preview interface.

[0311] As shown in FIG. 8B , the preview interface may include a mode bar 501 , a shooting control 502 , a preview window 503 , a review control 504 , a quick function area 505 , and a focus adjustment option 506 .

[0312] The mode bar 501 may include multiple shooting mode options, such as "night scene", "portrait", "photo", "video recording", etc. Different shooting modes can provide users with shooting services with different effects. Users can select any shooting mode from multiple shooting modes to shoot according to different needs. For example, "photo recording" may be the default shooting mode for taking photos. "Video recording" is used to record videos. The "night scene" mode is suitable for shooting scenes with dim light, such as at night. The "portrait" mode is suitable for shooting scenes where the subject is a person. The electronic device 100 can also provide more shooting modes, such as "large aperture", "movie", "professional", etc., which are not listed here one by one.

[0313] The electronic device 100 can detect user operations on the shooting mode options in the mode bar 501 and change the currently used shooting mode according to the above user operations. The above user operations are, for example, left / right swipe operations. For example, when it is detected that the mode bar 501 is dragged and slid to the left (left swipe operation) and the float stops at the "Portrait" option, the electronic device 100 can switch to the "Portrait" mode. By default, the electronic device 100 first uses the "Photograph" mode.

[0314] The capture control 502 is used to trigger a photo. The electronic device 100 can detect whether a user action, such as a click, is applied to the capture control 502. Upon detecting a user action on the capture control 502, the electronic device 100 can generate a capture instruction. Based on the capture instruction, the electronic device 100 can retrieve the image reported by the camera at the corresponding timestamp and save it as a photo.

[0315] The preview window 503 can be used to display the image reported by the camera in real time. In different shooting modes, the electronic device 100 can process the image reported by the camera to improve the image display effect. For example, in "Portrait" mode, the electronic device 100 can blur the background of the image reported by the camera to highlight the portrait. Here, the preview window 503 can display the image processed by the image processing algorithm corresponding to each shooting mode in real time, so that the user can perceive the shooting effect corresponding to each shooting mode in real time.

[0316] The review control 504 can be used to browse thumbnails of photos / videos that have been taken. When a user operation acting on the review control 504 is detected, the electronic device 100 can also display the best photo corresponding to the thumbnail.

[0317] The quick function area 505 may include a control 505A for the protagonist recording mode, an AI scene recognition control 505B, a flash control 505C, a color mode control 505D, a setting control 505E, and the like. The control 505A for the protagonist recording mode can be used to trigger the electronic device 100 to identify the protagonist among multiple characters in the preview screen when it is turned on. The AI ​​scene recognition control 505B can be used to trigger the electronic device 100 to identify the shooting scene in the preview screen when it is turned on. The current AI scene recognition control 505B is in the off state. The flash control 505C can be used to trigger the electronic device 100 to turn on or off the flash. The color mode control 505D can be used to trigger the electronic device 100 to use a color filter to process the image captured by the camera. The setting control 505E can be used to set the shooting parameters of the electronic device 100 (for example, image size, image storage format, etc.), turn on the "auto-snap" function of the electronic device 100, and the like.

[0318] The focus adjustment option 506 shows a plurality of zoom magnification options, such as a 0.5x zoom magnification option, a 1x zoom magnification option, a 2.5x zoom magnification option, and a 10x zoom magnification option.

[0319] The current zoom ratio is 1x. The preview window 503 displays an image frame captured by the electronic device 100 at a zoom ratio of 1x.

[0320] As shown in FIG8B , the electronic device 100 may receive a user input operation for the 2.5x zoom ratio option in the focus adjustment option 506 . In response to the user input operation, the electronic device 100 may switch the zoom ratio from 1x to 2.5x.

[0321] In response to the electronic device 100 switching the zoom magnification from 1x to 2.5x, the electronic device 100 may determine multiple transition zoom magnifications based on the initial zoom magnification 1x and the target zoom magnification 2.5x. For example, the multiple transition zoom magnifications may be 1.4x and 1.9x.

[0322] Optionally, the electronic device 100 is not limited to determining two transition zoom ratios. It can also determine more or fewer transition zoom ratios, which is not limited in this application. The values ​​of the initial zoom ratio, target zoom ratio, and multiple transition zoom ratios can also be other values. This application is only used as an example to explain this application and does not constitute a limitation.

[0323] In response to the electronic device 100 determining a plurality of transition zoom magnifications, the electronic device 100 may sequentially capture image frames based on the plurality of transition zoom magnifications and display the image frames.

[0324] As shown in FIG. 8C and FIG. 8D , FIG. 8C shows an image frame captured by the electronic device 100 at a zoom ratio of 1.4×, and FIG. 8D shows an image frame captured by the electronic device 100 at a zoom ratio of 1.9×.

[0325] Afterwards, the electronic device 100 continuously captures and displays image frames at the target zoom magnification of 2.5x. FIG8E shows image frames captured by the electronic device 100 at a zoom magnification of 2.5x.

[0326] As can be seen from Figures 8A to 8E, when the electronic device 100 increases the zoom ratio based on user operation, the electronic device 100 can smoothly switch to the target zoom ratio through multiple transition zoom ratios in sequence, thereby achieving smooth switching from the initial zoom ratio to the target zoom ratio.

[0327] Reduce the zoom level of the camera app

[0328] 8F-8I are schematic diagrams showing the electronic device 100 reducing the zoom magnification of the camera application based on a user operation.

[0329] The electronic device 100 may display a preview interface of the camera application as shown in FIG8F . The current zoom ratio is 2.5x. The preview window 503 displays an image frame captured by the electronic device 100 at a zoom ratio of 2.5x.

[0330] As shown in FIG. 8F , the electronic device 100 may receive a user input operation for the 1x zoom ratio option in the focus adjustment option 506 . In response to the user input operation, the electronic device 100 may switch the zoom ratio from 2.5x to 1x.

[0331] In response to the electronic device 100 switching the zoom magnification from 2.5x to 1x, the electronic device 100 may determine multiple transition zoom magnifications based on the initial zoom magnification 2.5x and the target zoom magnification 1x. For example, the multiple transition zoom magnifications may be 1.9x and 1.4x.

[0332] Optionally, the electronic device 100 is not limited to determining two transition zoom ratios. It can also determine more or fewer transition zoom ratios, which is not limited in this application. The values ​​of the initial zoom ratio, target zoom ratio, and multiple transition zoom ratios can also be other values. This application is only used as an example to explain this application and does not constitute a limitation.

[0333] In response to the electronic device 100 determining a plurality of transition zoom magnifications, the electronic device 100 may sequentially capture image frames based on the plurality of transition zoom magnifications and display the image frames.

[0334] As shown in FIG. 8G-FIG . 8H , FIG. 8G shows an image frame captured by the electronic device 100 at a zoom ratio of 1.9×, and FIG. 8H shows an image frame captured by the electronic device 100 at a zoom ratio of 1.4×.

[0335] Afterwards, the electronic device 100 continuously captures and displays image frames at the target zoom magnification of 1x. FIG8I shows image frames captured by the electronic device 100 at a zoom magnification of 1x.

[0336] As can be seen from Figures 8F to 8I, when the electronic device 100 reduces the zoom ratio based on user operation, the electronic device 100 can smoothly switch to the target zoom ratio through multiple transition zoom ratios in sequence, thereby achieving smooth switching from the initial zoom ratio to the target zoom ratio.

[0337] It should be noted that Figures 8A-8I above illustrate the example of a user changing the zoom ratio in the camera app's photo preview interface. In other embodiments, the user can also change the zoom ratio in the camera app's video preview interface. This application does not limit the application scenario for changing the zoom ratio.

[0338] FIG9A shows a software architecture diagram of an electronic device 100 for capturing images at an initial zoom ratio according to an embodiment of the present application.

[0339] As shown in Figure 9A, the software architecture of electronic device 100 includes, but is not limited to, the interaction and collaboration between the application layer (such as the camera application), the application framework layer (framework), the hardware abstraction layer (HAL), and the hardware. The application layer mainly involves the camera request sending module in the camera application. The application framework layer mainly involves the request queue processing module, the allocation buffer interface, the result processing module, the rendering process surface flinger, the timer, the vsync signal monitoring module, the display sending thread, and the buffer manager. The hardware mainly includes the camera module and the display.

[0340] Before the electronic device 100 receives a user operation to change the zoom magnification, the electronic device 100 may capture and display an image at an initial zoom magnification.

[0341] Including but not limited to the following steps:

[0342] 1. The vsync signal monitoring module monitors the vsync signal cycle from the rendering process.

[0343] 2. The vsync signal monitoring module sends the vsync signal periodically to the display thread.

[0344] When the electronic device 100 receives the user's operation to start the camera application, the electronic device 100 starts the camera application. The vsync signal monitoring module can monitor the vsync signal cycle in the rendering process and send the vsync signal cycle to the display process.

[0345] 3. The timer sends the timing information to the display thread.

[0346] The timer can be used to send a timer period signal to the display thread. The timer period can be determined by the screen refresh rate of the electronic device 100. For example, if the screen refresh rate of the electronic device 100 is 30 Hz, the rendering process renders an image frame every 33.3 ms. Therefore, the timing information can be 33.3 ms.

[0347] It should be noted that steps 1 to 3 are executed continuously / irregularly / periodically after the camera application is opened.

[0348] 4. When the electronic device 100 receives the user's operation to start the camera application, the electronic device 100 starts the camera application. The camera application can issue multiple camera requests for obtaining preview images. The first camera request for obtaining preview images issued by the camera application can occupy the N buffers reserved for camera requests in the cache area.

[0349] Optionally, after the camera application is started, if the shooting parameters have not changed, the camera application may only send a camera request to the application framework layer once.

[0350] For the process of how the electronic device 100 processes the camera request sent by the camera application and sends the image frame for display, please refer to the process description of how the electronic device 100 processes the zoom ratio adjustment request and sends the image frame in steps 6 to 14. This application will not go into details here.

[0351] The camera application sends a camera request to the request queue processing module in the application framework layer.

[0352] The electronic device 100 may receive an operation from a user to start a camera application. Based on this operation, the camera application may send a camera request to the application framework layer, where the camera request carries an initial zoom factor.

[0353] 5. The request queue processing module obtains the cache get buffer.

[0354] In some embodiments, the application framework layer may include an Allocate Buffer interface. In response to a camera request sent by a camera application, the HAL request preparation module may send a get buffer request to the Allocate Buffer interface to obtain a free buffer and allocate the free buffer for the camera request.

[0355] 6. The request queue processing module allocates buffer for camera requests.

[0356] The electronic device 100 may send the camera request to the prepare HAL request module in the request queue processing module. When there is an empty buffer among the N buffers in the cache area of ​​the electronic device 100, the prepare HAL request module obtains the Allocate Buffer interface to allocate a buffer address for the camera request and then stores the camera request in the buffer.

[0357] 7. The request queue processing module sends camera requests to the HAL layer.

[0358] After the application framework layer allocates a buffer for the camera request, the application framework layer sends a camera request to the HAL layer. The camera request carries the initial zoom factor. The camera request can also carry other data, which is not limited in this application.

[0359] Specifically, the HAL request preparation module constructs the camera request into a HAL-layer Capture request and sends it to the HAL layer via the batch request processing module. The HAL layer then sends the camera parameters in the Capture request and the buffer address of the Capture request to the camera module. The camera parameters in the Capture request may include, but are not limited to, the initial zoom factor.

[0360] 8. The HAL layer sends a camera request to the camera module.

[0361] In response to the camera request sent by the application framework layer, the HAL layer then sends the camera request to the camera module.

[0362] 9. The electronic device 100 turns on the camera, obtains the image frame captured by the camera based on the initial zoom ratio, and sends the image frame to the HAL layer.

[0363] The camera module in the electronic device 100 can receive the camera request and the buffer address of the camera from the HAL layer. Then, the camera module can obtain image frames according to the camera parameters (such as the initial zoom factor) and store the obtained image frames in the buffer corresponding to the buffer address carried in the zoom factor adjustment request.

[0364] In some examples, the image frames captured by the camera module can be transmitted to the image signal processor, which can pre-process the image frames and upload them to the HAL layer through the camera driver or the image processor driver.

[0365] 10. The HAL layer uploads the image frame to the result processing module in the application framework layer.

[0366] After receiving the image frame uploaded by the image signal processor, the HAL layer uploads the image frame to the result processing module in the application framework layer.

[0367] In some examples, the HAL layer may upload the image frame to the camera application through the application framework layer, and the camera application may process the image frame before sending it to the result processing module.

[0368] 11. The result processing module sends the return frame time of the image frame to the image cropping module.

[0369] 12. The image cropping module sends the return frame time of the image frame to the shooting parameter generation module.

[0370] 13. The result processing module sends the image frame to the buffer manager.

[0371] After receiving the image frame sent by the HAL layer, the result processing module stores the image frame in the buffer manager so that the buffer manager can cache the image frame and send the cached image frame to the display thread at the appropriate time.

[0372] The result processing module also needs to send the image frame return time to the image cropping module, and the image cropping module then sends the image frame return time to the shooting parameter generation module, so that the subsequent shooting parameter generation module can determine one or more transition zoom ratios based on the image frame return time.

[0373] Optionally, the result processing module may also directly send the return frame time of the image frame to the shooting parameter generation module.

[0374] 14. The display thread may determine the timing to obtain the next image frame to be displayed from the buffer manager based on the vsync signal period or the timer period, and send a request to the buffer manager to obtain the image frame based on the timing of the next image frame to be displayed.

[0375] 15. The display thread obtains the image frame from the buffer manager.

[0376] In a possible implementation, the display thread may be configured to obtain the next image frame to be displayed from the buffer manager after an interval of m vsync signal cycles from the time when the previous image frame was displayed.

[0377] In a possible implementation, the display thread may also obtain the next image frame to be displayed from the buffer manager after an interval of a display period of the timer between the display time and the display time of the previous image frame.

[0378] 16. The display thread sends the acquired image frame to the surface flinger.

[0379] 17. The rendering process surface flinger renders the image frame and sends the rendered image frame to the camera application.

[0380] 18. The camera application sends the rendered image frame to the display.

[0381] In some embodiments, after the display sending thread obtains an image frame from the buffer manager, it may first send the image frame to the camera application. The camera application then sends the image frame to the rendering process (surface flinger). The rendering process (surface flinger) renders the image frame and sends the rendered image frame to the camera application. The camera application then sends the rendered image frame to the display.

[0382] FIG9B shows a schematic flow chart of a photographing method provided in an embodiment of the present application.

[0383] This method is implemented through the interaction between the application layer (e.g., camera application), the application framework layer, the hardware abstraction layer (HAL), and the hardware within the electronic device. The application layer primarily involves the camera request sending module within the camera application. The application framework layer primarily involves the request queue processing module, the buffer allocation interface, the result processing module, the rendering process (surface flinger), the timer, the display sending thread, and the buffer manager. The hardware primarily includes the camera module and display.

[0384] S901-S918 illustrate a process in which the electronic device 100 captures and displays image frames at an initial zoom ratio before the user changes the zoom ratio.

[0385] S901. The user clicks the “Camera” desktop icon.

[0386] The electronic device 100 may detect a user's operation of opening a camera application, for example, the user clicks a desktop icon of "camera".

[0387] S902. The vsync signal monitoring module monitors and obtains the vsync signal period from the rendering process.

[0388] The vsync signal period is determined by the screen refresh frame rate, and the vsync signal period is the inverse of the screen refresh frame rate. For example, when the screen refresh frame rate is 120Hz, the vsync signal period may be 8.3ms; when the screen refresh frame rate is 60Hz, the vsync signal period may be 16.6ms; when the screen refresh frame rate is 30Hz, the vsync signal period may be 33.3ms.

[0389] The screen refresh rate (refresh rate) indicates the number of times the display screen on the electronic device refreshes the display frame within 1 second. Generally, the refresh rate of the display screen of the electronic device is related to the fixed parameters of the display screen and is a fixed value.

[0390] The electronic device may support multiple optional screen refresh frame rates, and the multiple optional screen refresh frame rates may include at least a minimum screen refresh frame rate and a maximum screen refresh frame rate.

[0391] Optionally, the multiple selectable screen refresh frame rates may have only two selectable screen refresh frame rates, one being the highest screen refresh frame rate and the other being the lowest screen refresh frame rate. For example, the two selectable screen refresh frame rates are 60 Hz and 90 Hz. 60 Hz is the lowest screen refresh frame rate among the multiple selectable screen refresh frame rates, and 90 Hz is the highest screen refresh frame rate among the multiple selectable screen refresh frame rates.

[0392] Optionally, the multiple selectable screen refresh frame rates may further include more selectable screen refresh frame rates. For example, three selectable screen refresh frame rates are included: 60 Hz, 90 Hz, and 120 Hz. 60 Hz is the lowest screen refresh frame rate among the multiple selectable screen refresh frame rates, and 120 Hz is the highest screen refresh frame rate among the multiple selectable screen refresh frame rates.

[0393] It is understandable that, according to actual needs, the multiple optional screen refresh frame rates can also include more screen refresh frame rates. The optional screen refresh frame rates can also have more different options, which are not limited here.

[0394] The vsync monitoring module can monitor the vsync signal in the rendering process module.

[0395] It is understandable that after the camera application is started, the vsync monitoring module can start to periodically / irregularly monitor the vsync signal in the rendering process module. That is, S901 is continuously executed during the entire process of the electronic device 100 processing the camera request.

[0396] Exemplarily, the vsync signal period may be 33.3 ms, that is, the rendering process sends a vsync signal every 33.3 ms.

[0397] The vsync signal cycle is used to control the rendering process's progress in rendering image frames. Between two vsync signals (or one vsync signal cycle), the rendering process only renders one frame to be displayed. This prevents the rendering process from continuously sending two frames to be displayed within a single vsync signal cycle, resulting in frame drops and lag.

[0398] S903. The vsync signal monitoring module sends the acquired vsync signal period to the display process module.

[0399] The rendering process module can send a vsync signal when rendering an image frame. The vsync monitoring module can monitor the rising and falling edges of the vsync signal. The vsync signal can change periodically.

[0400] The vsync monitoring module can send a vsync signal cycle or a vsync signal to the display process module when it monitors the rising and falling edge changes of the vsync signal.

[0401] Optionally, the vsync monitoring module may notify the display process module once every vsync signal cycle. Alternatively, the vsync monitoring module may notify the display process module once every vsync signal detected.

[0402] S904: The display process module obtains an image frame from the buffer manager every m vsync signal cycles.

[0403] The display process module can obtain an image frame from the buffer manager for rendering every m vsync signal cycles, where m is a positive integer greater than or equal to 1.

[0404] The value of m is related to the camera application's display frame rate and vsync signal period. The display frame rate indicates the speed at which the application sends image frames for display. The higher the display frame rate, the faster the application sends image frames for display, resulting in a smoother display. The display frame rate is measured in FPS. For example, at a display frame rate of 120 FPS, the application sends an image frame every 8.3 ms; at a display frame rate of 60 FPS, the application sends an image frame every 16.6 ms; and at a display frame rate of 30 FPS, the application sends an image frame every 33.3 ms.

[0405] In some embodiments, if the display frame rate is greater than the screen refresh frame rate, for example, when the display frame rate is 120FPS and the vsync signal period is 16.6ms, it means that the rendering capability of the rendering process is lower than the display capability of the application. Within one vsync signal period, the rendering process renders one frame for each time, and the application needs to send two image frames for display.

[0406] In some embodiments, if the display frame rate is equal to the screen refresh frame rate, for example, the display frame rate is 60FPS and the vsync signal period is 16.6ms, then the rendering capability of the rendering process is equivalent to the display capability of the application. Within one vsync signal period, the rendering process renders one frame for each time, and the application needs to send one image frame for display.

[0407] In some embodiments, if the display frame rate is lower than the screen refresh frame rate, for example, when the display frame rate is 30FPS and the vsync signal period is 16.6ms, it means that the rendering capability of the rendering process is higher than the display capability of the application. Within two vsync signal periods, the rendering process renders two frames, and the application will display one frame. At this time, the image frame rendering capability is too fast, resulting in image frame accumulation.

[0408] To prevent image frame accumulation caused by the rendering process's rendering capabilities exceeding the application's display capabilities, the value of m can be determined based on the camera application's display frame rate and vsync signal period. The rendering process renders a new frame every m vsync signal periods, ensuring that the rendering process's rendering capabilities are comparable to the application's display capabilities, preventing image frame accumulation.

[0409] For example, the value of m can be determined based on m=1000ms / display frame rate / vsync signal period. The value of m can also be determined based on other methods, which are not limited in this application.

[0410] In some embodiments, if the display frame rate is greater than the screen refresh frame rate and the value of m is less than 1, the value of m is set to 1. For example, when the display frame rate is 120 FPS and the vsync signal period is 16.6 ms, the value of m is 1.

[0411] In some embodiments, if the display frame rate is equal to the screen refresh frame rate, the value of m is 1. For example, when the display frame rate is 60 FPS and the vsync signal period is 16.6 ms, the value of m is 1.

[0412] In some embodiments, if the display frame rate is less than the screen refresh frame rate, the value of m is greater than 1. For example, when the display frame rate is 30 FPS and the vsync signal period is 16.6 ms, the value of m is 2.

[0413] Optionally, in some embodiments, when the display sending process does not receive m vsync signal cycles sent by the vsync monitoring module after sending the image frame, the display sending process can determine the rendering time of the next image frame according to the timestamp sent by the timer.

[0414] Exemplarily, the display sending process determines the rendering time of the next image frame based on the timestamp sent by the timer, which can be implemented as follows: when the time interval between the timestamp of the previous image frame sent to the rendering process and the timestamp of the next image frame to be sent to the rendering process is greater than a preset time interval, the display sending process sends the next image frame to be sent to the rendering process for rendering. Exemplarily, the timestamp of the image frame in buffer0 is time t1, and the timestamp of the image frame in buffer1 is time 2. At time t1, after the display sending process sends the image frame in buffer0 to the rendering process, when it is determined at time 2 that the time interval between time 2 and time 1 is greater than the preset time interval, the display sending process sends the image frame in buffer1 to the rendering process.

[0415] Optionally, the display process sending the image frame in buffer 1 to the rendering process may include: the display process uploading the address of buffer 1 to the camera application. After receiving the address of buffer 1, the camera application may notify the rendering process module in the application framework layer to perform rendering and inform the rendering process module of the address of buffer 1. The rendering process module may extract the image frame from buffer 1 based on the address of buffer 1 and render the image frame. The rendering process module may then send the rendering result to the camera application.

[0416] Optionally, the display process can also directly send the address of buffer1 to the rendering process module without sending it to the rendering process module through the camera application, and this application does not limit this.

[0417] Alternatively, the rendering process module may directly render the image frame in buffer1, and when the rendering is complete, the camera application may be notified that the rendering is complete. The camera application may then retrieve the rendered image frame from buffer1 and display the rendered image frame on the display.

[0418] Optionally, in one possible implementation, the vsync monitoring module may not execute steps S903-S904. When the vsync monitoring module detects the mth vsync signal in the rendering process module, the vsync monitoring module may notify the display process module to obtain the image frame to be rendered. Then, when the vsync monitoring module detects the 2mth vsync signal in the rendering process module, the vsync monitoring module may notify the display process module to obtain the next image frame to be rendered. Similarly, the vsync monitoring module notifies the display process module to obtain a frame to be rendered every m vsync signal cycles.

[0419] Optionally, in another possible implementation, the display processing module may also receive a timestamp sent by a timer. When the time interval between the current time and the timestamp of the previous image frame sent for rendering is greater than or equal to a preset time interval, the display processing module may obtain the image frame to be rendered from the buffer manager.

[0420] S905 . In response to clicking the “camera” desktop icon to start the camera application, the camera request sending module sends a first camera request (initial zoom ratio) to the request queue processing module.

[0421] For example, as shown in Figure 8A, the user can click the camera application icon. In response to the user operation, the electronic device 100 starts the camera application and displays a preview interface of the camera application, which can be the photo preview interface shown in Figure 8B.

[0422] It is not limited to the photo preview interface. The preview interface of the camera application can also be a video preview interface, and this application does not limit this.

[0423] In response to starting the camera application, the camera request sending module may obtain an initial zoom factor and send a first camera request to the request queue processing module in the application framework layer.

[0424] The first camera request carries an initial zoom ratio and may also carry other parameter information, such as initial focal length, initial exposure, initial aperture, initial white balance, initial sharpness, initial contrast, and initial saturation.

[0425] For example, the initial zoom ratio may be 1x or 2.5x.

[0426] Optionally, the camera request sending module may first send the first camera request to the shooting parameter generation module in the application framework layer, and then the shooting parameter generation module sends the first camera request to the request queue processing module. This application does not limit this.

[0427] S906: The request queue processing module allocates a first buffer for the first camera request.

[0428] Among them, the first buffer is an idle buffer.

[0429] In some embodiments, the application framework layer may further include an Allocate Buffer interface for allocating a buffer for a camera request issued by a camera application. For example, the Allocate Buffer interface may specify the address of a buffer for storing the camera request.

[0430] In some embodiments, the request queue processing module may include a WaitForRequest module, a PrepareHALRequest module, and a SendRequestsBatch module.

[0431] The WaitForRequest module is used to wait for the first camera request to be executed in the request queue module if there are no empty buffers in the N buffers allocated to the camera application in the electronic device's buffer area. The PrepareHALRequest module is used to construct the HAL layer Capture request and the output buffer outputBuffers. The SendRequestsBatch module can be used to send the HAL layer Capture request constructed by the PrepareHALRequest module to the HAL layer.

[0432] If there are free buffers among the N buffers allocated to the camera application's camera requests, the PrepareHALRequest module can use the Allocate Buffer API to obtain the first buffer for the first camera request issued by the camera application. The module then stores the first camera request in the first buffer. The PrepareHALRequest module then constructs a HAL layer Capture request based on the first camera request. The PrepareHALRequest module then sends the HAL layer Capture request constructed based on the first camera request to the HAL layer via the SendRequestsBatch module.

[0433] For example, N can be 8, meaning that the number of buffers allocated by the electronic device 100 to the camera application for camera requests can be 8. N can also be other values, such as 7 or 9, and the present application does not limit the value of N. The following description uses the example of N being 8 as an example.

[0434] S907: The request queue processing module sends the first camera request and the address of the first buffer to the HAL layer.

[0435] Optionally, the address of the first buffer may be carried in the first camera request and sent to the HAL layer, or the address of the first buffer may be sent to the HAL layer independently of the first camera request, which is not limited in this application.

[0436] After receiving the first camera request and the address of the first buffer sent by the request queue processing module, the request queue processing module sends the first camera request and the address of the first buffer to the HAL layer.

[0437] S908 : The HAL layer sends the first camera request and the address of the first buffer to the camera module.

[0438] After receiving the first camera request and the address of the first buffer sent by the request queue processing module, the HAL layer sends the first camera request and the address of the first buffer to the camera module.

[0439] S909: The camera module collects and processes the original image stream to obtain image frame A (initial zoom ratio).

[0440] After receiving the first camera request and the address of the first buffer sent by the HAL layer, the camera module responds to the first camera request by turning on and starting to capture and process the raw image stream to obtain image frames. The image frames obtained by the camera module may include image frame A, which is an image frame obtained by the camera module at an initial zoom factor.

[0441] S910: The camera module sends the image frame A (initial zoom ratio) and the address of the first buffer to the HAL layer.

[0442] S911: The HAL layer then sends the image frame A (initial zoom ratio) and the address of the first buffer to the result processing module.

[0443] S912: The result processing module sends the image frame A (initial zoom ratio) and the address of the first buffer to the buffer manager.

[0444] After the camera module acquires the image frame A based on the initial zoom magnification, the camera module sends the image frame A (initial zoom magnification) and the address of the first buffer to the HAL layer.

[0445] After obtaining the image frame A (initial zoom ratio) and the address of the first buffer sent by the camera module, the HAL layer sends the image frame A (initial zoom ratio) and the address of the first buffer to the buffer manager.

[0446] S913: After m vsync signal cycles, the display sending process obtains image frame A from the buffer manager.

[0447] The display process module can obtain an image frame from the buffer manager every m vsync signal cycles. For example, after m vsync signal cycles have accumulated, the display process can obtain an image frame, such as image frame A, from the buffer manager.

[0448] Optionally, the display processing module may also receive a timestamp sent by a timer. When the time interval between the current time and the timestamp of the previous image frame sent for rendering is greater than or equal to a preset time interval, the display processing module may obtain the image frame to be rendered from the buffer manager. For example, the display processing module may obtain image frame A to be rendered from the buffer manager.

[0449] S914: The display sending process sends the image frame A or the address of the first buffer to the rendering process.

[0450] S915 : The rendering process renders the image frame A to obtain the rendered image frame A.

[0451] After m vsync signal cycles have accumulated, the display sending process can obtain image frame A from the buffer manager, and then send image frame A or the address of the first buffer to the rendering process.

[0452] After the rendering process obtains the image frame A or the address of the first buffer, the rendering process can render the image frame A to obtain the rendered image frame A.

[0453] In some examples, if the rendering process module obtains the address of the first buffer, the rendering process module may render image frame A in the buffer corresponding to the address of the first buffer. Alternatively, if the rendering process module obtains the address of the first buffer, the rendering process module may retrieve image frame A from the first buffer corresponding to the address of the first buffer and render image frame A.

[0454] S916: The rendering process sends the rendered image frame A or the address of the first buffer to the camera application.

[0455] S917: The camera application sends the rendered image frame A or the address of the first buffer to the display.

[0456] S918. The display shows the rendered image frame A.

[0457] In some embodiments, when the display displays the rendered image frame A, the request queue processing module may clear the buffer of the image frame A, for example, clear the first buffer corresponding to the address of the first buffer.

[0458] The rendering process module may send the rendered image frame A or the address of the first buffer to the camera application, and the camera application may then send the rendered image frame A or the address of the first buffer to the display.

[0459] After the display obtains the rendered image frame A or the address of the first buffer, the display may display the rendered image frame A.

[0460] In some examples, if the display obtains the address of the first buffer, the rendering process module can retrieve the rendered image frame A from the first buffer corresponding to the address of the first buffer and display the rendered image frame A.

[0461] In some embodiments, after m vsync signal cycles have elapsed and the display process obtains image frame A from the buffer manager, the display process may first send image frame A or the address of the first buffer to the camera application. The camera application then sends image frame A or the address of the first buffer to the rendering process, which renders image frame A to obtain a rendered image frame A. The rendering process then sends the rendered image frame A or the address of the first buffer to the camera application, which then sends the rendered image frame A or the address of the first buffer to the display, which then displays the rendered image frame A.

[0462] For how the electronic device renders and displays the picture frame A, please refer to the description in the embodiment of Figure 10 or Figure 11.

[0463] Optionally, the camera module can continuously / irregularly / periodically capture image frames and report them to the buffer manager through the HAL layer. After image frame A, the camera module can continuously / irregularly / periodically capture image frames B, C, D, E, etc., and render and display them.

[0464] Next, how the electronic device 100 renders and displays image frames will be described with reference to a timing diagram of a vsync signal cycle.

[0465] FIG. 10 shows a timing diagram of an electronic device 100 rendering and displaying an image frame using a vsync signal cycle.

[0466] For example, the display frame rate may be 30 FPS, and the vsync signal period is 33.3 ms, then m is 1, that is, the rendering process renders one image frame in each vsync signal period, and the camera application sends one image frame for display in each vsync signal period.

[0467] As shown in Figure 10, between vertical synchronization signal 1 (vsync1) and vertical synchronization signal 2 (vsync2), the camera application can send rendering instruction A to the rendering process. In response to rendering instruction A, the rendering process can render image frame A and fill the rendered image frame A into a buffer, such as the first buffer. At the same time, between vertical synchronization signal 1 (vsync) and vertical synchronization signal 2 (vsync2), the camera application can send image frame 0 or the buffer of image frame 0 to the display, so that the display can display image frame 0. That is, between time tms and time (t+33.3ms), the rendering process renders image frame A, and the display displays image frame 0.

[0468] Between vertical synchronization signal 2 (vsync2) and vertical synchronization signal 3 (vsync3), the camera application can send rendering instruction B to the rendering process. In response to rendering instruction B, the rendering process can render image frame B and fill the rendered image frame B into the buffer. At the same time, between vertical synchronization signal 2 (vsync2) and vertical synchronization signal 3 (vsync3), the camera application can send image frame A or the buffer of image frame A to the display, so that the display can display image frame A. That is, between time (t+33.3ms) and time (t+66.6ms), the rendering process renders image frame B, and the display displays image frame A.

[0469] Between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 4 (vsync4), the camera application can send rendering instruction C to the rendering process. In response to rendering instruction C, the rendering process can render image frame C and fill the rendered image frame C into the buffer. At the same time, between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 4 (vsync4), the camera application can send image frame B or the buffer of image frame B to the display, so that the display can display image frame B. That is, between time (t+66.6ms) and time (t+99.9ms), the rendering process renders image frame C, and the display displays image frame B.

[0470] Between vertical synchronization signal 4 (vsync4) and vertical synchronization signal 5 (vsync5), the camera application can send rendering instruction D to the rendering process. In response to rendering instruction D, the rendering process can render image frame D and fill the rendered image frame D into the buffer. At the same time, between vertical synchronization signal 4 (vsync4) and vertical synchronization signal 5 (vsync5), the camera application can send image frame C or the buffer of image frame C to the display, so that the display can display image frame C. That is, between time (t+99.9ms) and time (t+133.2ms), the rendering process renders image frame D, and the display displays image frame C.

[0471] Between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 6 (vsync6), the camera application can send rendering instruction E to the rendering process. In response to rendering instruction E, the rendering process can render image frame E and fill the rendered image frame E into the buffer. At the same time, between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 6 (vsync6), the camera application can send image frame D or the buffer of image frame D to the display, so that the display can display image frame D. That is, between time (t+133.2ms) and time (t+166.5ms), the rendering process renders image frame E, and the display displays image frame D.

[0472] Similarly, the camera application can render and display image frames as shown in Figure 10.

[0473] FIG. 11 shows a timing diagram of an electronic device 100 rendering and displaying image frames with different vsync signal periods.

[0474] For example, the display frame rate may be 30 FPS, and the vsync signal period is 16.6 ms, then m is 2, that is, the rendering process renders one image frame every two vsync signal periods, and the camera application sends one image frame for display every two vsync signal periods.

[0475] As shown in Figure 11, between vertical synchronization signal 1 (vsync1) and vertical synchronization signal 3 (vsync3), that is, between two vsync signal cycles, the camera application can send rendering instruction A to the rendering process. In response to rendering instruction A, the rendering process can render image frame A and fill the rendered image frame A into a buffer, such as the first buffer. At the same time, between vertical synchronization signal 1 (vsync) and vertical synchronization signal 3 (vsync3), the camera application can send image frame 0 or the buffer of image frame 0 to the display, so that the display can display image frame 0. That is, between time tms and time (t+33.3ms), the rendering process renders image frame A, and the display displays image frame 0.

[0476] Between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 5 (vsync5), that is, between two vsync signal cycles, the camera application can send rendering instruction B to the rendering process. In response to rendering instruction B, the rendering process can render image frame B and fill the rendered image frame B into the buffer. At the same time, between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 5 (vsync5), the camera application can send image frame A or the buffer of image frame A to the display, so that the display can display image frame A. That is, between time (t+33.3ms)ms and time (t+66.6ms), the rendering process renders image frame B, and the display displays image frame A.

[0477] Between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 7 (vsync7), that is, between two vsync signal cycles, the camera application can send rendering instruction C to the rendering process. In response to rendering instruction C, the rendering process can render image frame C and fill the rendered image frame C into the buffer. At the same time, between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 7 (vsync7), the camera application can send image frame B or the buffer of image frame B to the display, so that the display can display image frame B. That is, between time (t+66.6ms) and time (t+99.9ms), the rendering process renders image frame C, and the display displays image frame B.

[0478] Between vertical synchronization signal 7 (vsync7) and vertical synchronization signal 9 (vsync9), that is, between two vsync signal cycles, the camera application can send rendering instruction D to the rendering process. In response to rendering instruction D, the rendering process can render image frame D and fill the rendered image frame D into the buffer. At the same time, between vertical synchronization signal 7 (vsync7) and vertical synchronization signal 9 (vsync9), the camera application can send image frame C or the buffer of image frame C to the display, so that the display can display image frame C. That is, between time (t+99.9ms)ms and time (t+133.2ms), the rendering process renders image frame D, and the display displays image frame C.

[0479] Between vertical synchronization signal 9 (vsync9) and vertical synchronization signal 11 (vsync11), that is, between two vsync signal cycles, the camera application can send rendering instruction E to the rendering process. In response to rendering instruction E, the rendering process can render image frame E and fill the rendered image frame E into the buffer. At the same time, between vertical synchronization signal 9 (vsync9) and vertical synchronization signal 11 (vsync11), the camera application can send image frame D or the buffer of image frame D to the display, so that the display can display image frame D. That is, between time (t+133.2ms) and time (t+166.5ms), the rendering process renders image frame E, and the display displays image frame D.

[0480] Similarly, the camera application can render and display image frames as shown in Figure 11.

[0481] FIG12A shows a software architecture diagram of another electronic device 100 provided in an embodiment of the present application for capturing images at an initial zoom ratio.

[0482] As shown in Figure 12A, the software architecture of the electronic device 100 includes, but is not limited to, the interaction and cooperation between the application layer (such as the camera application), the application framework layer (framework), the hardware abstraction layer (HAL), and the hardware. The application layer mainly involves the camera request sending module in the camera application. The application framework layer mainly involves the request queue processing module, the allocation buffer interface, the result processing module, the rendering process surface flinger, the timer, the vsync signal monitoring module, the display sending thread, and the buffer manager. The hardware mainly includes the camera module and the display.

[0483] After the electronic device 100 receives a user operation to change the zoom ratio, the electronic device 100 may determine one or more transition zoom ratios, and sequentially capture and display images based on the one or more transition zoom ratios. This may include but is not limited to the following steps:

[0484] 1. The vsync signal monitoring module monitors the vsync signal cycle from the rendering process.

[0485] 2. The vsync signal monitoring module sends the vsync signal periodically to the display thread.

[0486] When the electronic device 100 receives the user's operation to start the camera application, the electronic device 100 starts the camera application. The vsync signal monitoring module can monitor the vsync signal cycle in the rendering process and send the vsync signal cycle to the display process.

[0487] 3. The timer sends the timing information to the display thread.

[0488] The timer can be used to send a timer period signal to the display thread. The timer period can be determined by the screen refresh rate of the electronic device 100. For example, if the screen refresh rate of the electronic device 100 is 30 Hz, the rendering process renders an image frame every 33.3 ms. Therefore, the timing information can be 33.3 ms.

[0489] It should be noted that steps 1 to 3 are executed continuously / irregularly / periodically after the camera application is opened.

[0490] 4. The camera app receives the user's zoom operation.

[0491] For example, a camera application may receive a user operation for a target zoom magnification option. To enable the camera application to smoothly switch from an initial zoom magnification to a target zoom magnification, the electronic device may determine one or more transitional zoom magnifications, and sequentially capture and display images based on the one or more transitional zoom magnifications.

[0492] 5. The camera application sends a zoom request to the shooting parameter generation module in the application framework layer.

[0493] In response to the user's zoom operation, the camera application may send a zoom request to the shooting parameter generation module in the application framework layer, where the zoom request carries a target zoom ratio.

[0494] Exemplarily, the camera application may send a zoom request to the shooting parameter generation module in the application framework layer through the camera request sending module.

[0495] In some embodiments, in response to a zoom request, the shooting parameter generation module may generate a zoom smoothing adjustment curve based on an initial zoom ratio and a target zoom ratio, wherein the zoom smoothing adjustment curve is used to determine one or more transitional zoom ratios.

[0496] 6. The HAL layer sends the image frame to the result processing module in the application architecture layer.

[0497] It is understood that before the user changes the zoom factor, the electronic device 100 has been capturing image frames at the initial zoom factor. Therefore, after the camera application receives the user's zoom operation, the image frames sent by the HAL layer to the result processing module are those captured at the initial zoom factor, as processing the camera request takes time.

[0498] 7. The result processing module sends the image frame to the image cropping module.

[0499] 8. The image cropping module sends the return frame time of the image frame to the shooting parameter generation module.

[0500] After receiving the image frame sent by the result processing module, the result processing module sends the image frame to the shooting parameter generation module.

[0501] The shooting parameter generation module may also obtain the frame return time of the image frame, and determine the transition zoom ratio based on the frame return time of the image frame and the zoom smoothing adjustment curve.

[0502] 9. The shooting parameter generation module sends the zoom request to the request queue processing module.

[0503] The zoom request carries a transition zoom ratio.

[0504] The shooting parameter generation module determines a transition zoom ratio based on the frame return time of the image frame and the zoom smoothing adjustment curve, and then sends the transition zoom ratio to the request queue processing module.

[0505] 10. The request queue processing module obtains the cache get buffer.

[0506] In some embodiments, the application framework layer may include an Allocate Buffer interface. In response to the zoom request sent by the shooting parameter generation module, the HAL request preparation module may send a buffer acquisition request to the Allocate Buffer interface to obtain a free buffer and allocate the free buffer for the zoom request.

[0507] 11. The request queue processing module allocates buffer for zoom requests.

[0508] The electronic device 100 may send the zoom request to the prepare HAL request module in the request queue processing module. When there is an empty buffer among the N buffers in the cache area of ​​the electronic device 100, the prepare HAL request module obtains the Allocate Buffer interface to allocate a buffer address for the zoom request and then stores the zoom request in the buffer.

[0509] 12. The request queue processing module sends a zoom request to the HAL layer.

[0510] After the application framework layer allocates a buffer for the zoom request, the application framework layer sends a zoom request to the HAL layer, and the zoom request carries the transition zoom ratio. The zoom request can also carry other data, which is not limited in this application.

[0511] Specifically, the HAL request preparation module can construct the zoom request into a HAL-layer Capture request and send it to the HAL layer through the batch request sending module. The HAL layer then sends the camera parameters in the Capture request and the buffer address of the Capture request to the camera module. The camera parameters in the Capture request can include, but are not limited to, the transition zoom ratio.

[0512] 13. The HAL layer sends a zoom request to the camera module.

[0513] In response to the zoom request sent by the application framework layer, the HAL layer sends the zoom request to the camera module.

[0514] Optionally, since the HAL layer processes camera requests and zoom requests in a first-in-first-out queue order, after receiving the zoom request sent by the request queue processing module, the HAL layer may not process the zoom request immediately.

[0515] 14. The shooting parameter generation module sends the transition zoom ratio to the image cropping module.

[0516] The shooting parameter generation module determines the transition zoom ratio based on the frame return moment of the image frame and the zoom smoothing adjustment curve, and the shooting parameter generation module also needs to send the transition zoom ratio to the image cropping module respectively.

[0517] Optionally, step 14 may be performed before step 9, or simultaneously with step 9, which is not limited in this application.

[0518] 15. The image cropping module sends the transition image frame to the buffer manager.

[0519] After receiving the transition zoom factor from the shooting parameter generation module, the image cropping module can determine a cropping ratio based on the zoom factor and transition zoom factor of the image frame sent by the result processing module, crop the image frame sent by the result processing module based on the cropping ratio, obtain a transition image frame, and send the transition image frame to the buffer manager. For details, please refer to the description in the embodiment of FIG12B , and this application will not elaborate on this.

[0520] In other embodiments, the image cropping module may also crop the image frames sent by the result processing module to obtain multiple transition image frames, not limited to one transition image frame. For details, please refer to the description in the embodiment of FIG. 15A or FIG. 15B , which will not be described in detail in this application.

[0521] 16. The display thread may determine the timing to obtain the next transitional image frame to be displayed from the buffer manager based on the vsync signal period or the timer period, and send a request to the buffer manager to obtain the image frame based on the timing of the next transitional image frame to be displayed.

[0522] 17. The display thread obtains the transition image frame from the buffer manager.

[0523] In a possible implementation, the display thread may be configured to obtain the next transition image frame to be displayed from the buffer manager after an interval of m vsync signal cycles from the time when the previous image frame is displayed.

[0524] In a possible implementation, the display thread may also obtain the next transition image frame to be displayed from the buffer manager after a display period of the timer has elapsed since the display of the previous image frame.

[0525] 18. The display thread sends the acquired transition image frame to the rendering process surface flinger.

[0526] 19. The rendering process surface flinger renders the image frame and sends the rendered transition image frame to the camera application.

[0527] 20. The camera application sends the rendered transition image frame to the display.

[0528] In some embodiments, after the display delivery thread obtains the transitional image frame from the buffer manager, it can first send the transitional image frame to the camera application. The camera application then sends the transitional image frame to the surface flinger rendering process. The surface flinger rendering process renders the image frame and sends the rendered transitional image frame to the camera application. Finally, the camera application sends the rendered transitional image frame to the display.

[0529] In some embodiments, the electronic device 100 may also receive user operations to change the shooting parameters of the camera application. For example, the electronic device 100 may receive user operations to change the zoom ratio of the camera application. After the zoom ratio of the camera application is changed, the camera application must re-send the target zoom ratio to the application framework layer so that the camera module can re-capture and display the image frames according to the target zoom ratio.

[0530] FIG12B shows a flow chart of another shooting method provided in an embodiment of the present application.

[0531] S1201: The electronic device 100 captures and displays image frames at an initial zoom ratio.

[0532] Before the electronic device 100 receives the user's operation to change the zoom ratio, the electronic device 100 captures and displays image frames at the initial zoom ratio. For details, please refer to the description in the embodiment of FIG. 9B .

[0533] S1202: The user clicks a target zoom ratio option.

[0534] When the user needs to change the zoom ratio, the user can click on other zoom ratio options in the camera application preview interface, so that the camera application can switch to the zoom ratio selected by the user.

[0535] For example, the target zoom magnification option may be the 2.5x zoom magnification option shown in Fig. 8B. The user clicking on the target zoom magnification option may be the user clicking on the 2.5x zoom magnification option shown in Fig. 8B.

[0536] For example, the target zoom magnification option may also be the 1x zoom magnification option shown in Fig. 8F. The target zoom magnification option clicked by the user may be the 1x zoom magnification option shown in Fig. 8F.

[0537] S1203: The camera application sends a second camera request (target zoom ratio) to the shooting parameter generation module.

[0538] In response to the user clicking the target zoom ratio option, the camera application needs to send a second camera request to the camera parameter generation module. The second camera request is used to send the target zoom ratio to the application framework layer.

[0539] The first camera request includes the target zoom ratio. The target camera request may also include other parameter information, such as initial focal length, initial exposure, initial aperture, initial white balance, initial sharpness, initial contrast, initial saturation, etc. The target camera request may not include other parameter information, and this application does not limit this.

[0540] S1204: The shooting parameter generation module obtains a zoom smoothing adjustment curve based on the initial zoom magnification and the target zoom magnification.

[0541] After the shooting parameter generation module obtains the second camera request sent by the camera application, the shooting parameter generation module may obtain the target zoom ratio from the second camera request.

[0542] In response to the target zoom magnification included in the second camera request, the shooting parameter generation module may obtain a zoom smoothing adjustment curve based on the initial zoom magnification and the target zoom magnification. The zoom smoothing adjustment curve is a curve starting from the initial zoom magnification and ending at the target zoom magnification. The total duration of the zoom smoothing adjustment curve is a first duration.

[0543] The time corresponding to the starting point of the zoom smooth adjustment curve may be the time when the user clicks the target zoom magnification option. The time corresponding to the starting point of the zoom smooth adjustment curve may also be the time when the shooting parameter generation module receives the second camera request sent by the camera application. The difference between the time corresponding to the end point of the zoom smooth adjustment curve and the time corresponding to the starting point of the zoom smooth adjustment curve is the first duration.

[0544] In some embodiments, the first duration can be a fixed duration, i.e., the zoom duration is a fixed duration. In other embodiments, the first duration can also be determined by the shooting parameter generation module based on the initial zoom magnification and the target zoom magnification, for example, based on the difference between the initial zoom magnification and the target zoom magnification, i.e., the zoom duration is not a fixed duration. The first duration can also be determined based on other methods, which are not limited in this application.

[0545] It should be noted that, if the initial zoom ratio and / or the target zoom ratio are different, the obtained zoom smoothing adjustment curve will also be different.

[0546] In some embodiments, the shooting parameter generation module may use a cubic Bezier curve to generate a zoom smoothing adjustment curve. A cubic Bezier curve is a mathematical curve used in two-dimensional graphics applications. The path of a cubic Bezier curve is described by a function B(t) given points P0 (starting point), P1 (control point), P2 (control point), and P3 (end point): B(t) = P0(1-t)3 + 3P1t(1-t)2 + 3P22(1-t) + P3t3 Formula (1)

[0547] As shown in formula (1), t in formula (1) represents the first duration, P0 represents the position point corresponding to the initial zoom ratio, and P3 represents the position point corresponding to the target zoom ratio. The shape of the Bezier curve can be adjusted by adjusting the two control points P1 and P2.

[0548] For example, the general shape of the cubic Bezier curve, such as an "S" shape, and the information of the control point P1 and the control point P2 corresponding to the shape of the cubic Bezier curve, that is, the relative position coordinates of the control point P1 and the position point P0, and the relative position coordinates of the control point P2 and the position point P3, can be determined through a limited number of experiments. The information of the control point P1 and the control point P2 is preset in the electronic device 10. Each time the shooting parameters are adjusted, the shooting parameter generation module can adjust the coordinate positions of the starting point and the end point according to the actual situation to obtain a smooth adjustment curve.

[0549] When a zoom smooth adjustment curve needs to be obtained, the shooting parameter generation module can adjust the shape of the cubic Bezier curve based on the initial zoom ratio and the target zoom ratio to obtain a zoom smooth adjustment curve that matches the zoom reduction operation or the zoom increase operation.

[0550] In some embodiments, the initial zoom ratio may be smaller than the target zoom ratio, that is, the zoom ratio of the camera application is increased. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13A .

[0551] As shown in Figure 13A, the horizontal axis represents time, and the vertical axis represents zoom ratio. The initial zoom ratio corresponds to time t1, and the target zoom ratio corresponds to time t2. The target zoom ratio is greater than the initial zoom ratio. The time difference between t2 and t1 is a first duration, and t2 is greater than t1. Time t1 can be the moment when the user clicks the target zoom ratio option, or it can be the moment when the shooting parameter generation module receives the second camera request from the camera application. As shown in Figure 13A, the zoom speed near the start and end of zoom is slower than the zoom speed in the intermediate time periods.

[0552] In other embodiments, the initial zoom ratio may be greater than the target zoom ratio, that is, the zoom ratio of the camera application is reduced. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13B .

[0553] As shown in Figure 13B, the horizontal axis represents time, and the vertical axis represents zoom ratio. The initial zoom ratio corresponds to time t1, and the target zoom ratio corresponds to time t2. The target zoom ratio is smaller than the initial zoom ratio. The time difference between t2 and t1 is the first duration, and t2 is greater than t1. Time t1 can be the moment when the user clicks the target zoom ratio option, or it can be the moment when the shooting parameter generation module receives the second camera request from the camera application. As shown in Figure 13B, the zoom speed near the start and end of zoom is slower than the zoom speed in the intermediate time periods.

[0554] It should be noted that the zoom smoothing adjustment curves shown in FIG. 13A and FIG. 13B are zoom smoothing adjustment curves under ideal conditions.

[0555] S1205: The camera module collects and processes the original image stream to obtain image frame 1 (initial zoom ratio).

[0556] S1206 : The camera module sends the image frame 1 (initial zoom ratio) and the address of the buffer storing the image frame 1 to the HAL layer.

[0557] S1207: The HAL layer then sends the image frame 1 (initial zoom ratio) and the address of the buffer storing the image frame 1 to the result processing module.

[0558] S1208: The result processing module then sends the image frame 1 (initial zoom ratio) and the address of the buffer storing the image frame 1 to the image cropping module.

[0559] Optionally, before the camera module captures and processes the raw image stream to obtain image frame 1, the camera application can periodically or irregularly send a camera request to the request queue processing module, instructing the camera module to capture image frames at the initial zoom factor. The camera module can allocate an idle buffer for image frame 1 based on the camera request corresponding to image frame 1. The camera application also sends the camera request corresponding to image frame 1 and the address of the idle buffer allocated for image frame 1 to the camera module.

[0560] After the camera module acquires image frame 1, it sends the camera module and the address of the buffer storing image frame 1 to the HAL layer. The HAL layer then sends image frame 1 (with the initial zoom factor) and the address of the buffer storing image frame 1 to the result processing module. The result processing module then sends image frame 1 (with the initial zoom factor) and the address of the buffer storing image frame 1 to the image cropping module.

[0561] In some embodiments, the result processing module may also send image frame 1 (initial zoom ratio) and the address of the buffer storing image frame 1 to the buffer manager, so that the buffer manager may store image frame 1 or the address of the buffer storing image frame 1.

[0562] S1209: The image cropping module sends the return frame time of the image frame 1 to the shooting parameter generation module.

[0563] S1210: The shooting parameter generation module needs to determine whether the frame return time of the image frame 1 is later than the time of the target zoom ratio in the zoom smoothing adjustment curve.

[0564] After the shooting parameter generation module obtains the frame return time of image frame 1, the shooting parameter generation module needs to determine whether the frame return time of image frame 1 is later than the time of the target zoom ratio in the zoom smoothing adjustment curve, such as time t2.

[0565] When the return frame moment of image frame 1 is earlier than the moment of the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module needs to continue to obtain other transition zoom ratios and obtain image frames based on other transition zoom ratios, that is, execute S1211-S1220.

[0566] When the return frame time of image frame 1 is later than the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module does not need to obtain other transition zoom ratios. The zoom ends and the image frame is collected and displayed based on the target transition zoom ratio, that is, S1221 is executed.

[0567] S1211 . The shooting parameter generation module determines a first transition zoom ratio based on the zoom smoothing adjustment curve and the frame return time of the image frame 1 .

[0568] In some embodiments, the first transition zoom ratio may also be referred to as transition zoom ratio 1 .

[0569] Before the electronic device 100 receives an instruction from the user to click on a target zoom magnification option, the electronic device 100 continuously captures and displays image frames at the initial zoom magnification.

[0570] When the electronic device 100 receives a user click on the target zoom factor option, for example, after time t1, the electronic device 100 also returns N frames of image frames captured at the initial zoom factor. If, after time t1, at time t3, the application framework layer obtains image frame 1 returned by the HAL, that is, the return time of image frame 1 is time t3, and image frame 1 is an image frame captured at the initial zoom factor.

[0571] Exemplarily, the image frame 1 may be the image frame shown in FIG. 8B or the image frame shown in FIG. 8F .

[0572] When the return frame moment of image frame 1 is earlier than the target zoom ratio in the zoom smooth adjustment curve, in order to ensure smooth switching from the initial zoom ratio to the target zoom ratio, the shooting parameter generation module needs to determine the first transition zoom ratio based on the zoom smooth adjustment curve and the return frame moment of image frame 1.

[0573] Optionally, after obtaining time t3, the shooting parameter generation module may also obtain the transitional zoom magnification 1 based on a moment before time t3 and the zoom smoothing adjustment curve. Optionally, after obtaining time t3, the shooting parameter generation module may also obtain the transitional zoom magnification 1 based on a moment after time t3 and the zoom smoothing adjustment curve. This application only uses the example of the shooting parameter generation module obtaining the transitional zoom magnification 1 based on time t3 and the zoom smoothing adjustment curve.

[0574] In some embodiments, the initial zoom ratio may be smaller than the target zoom ratio, that is, the zoom ratio of the camera application is increased. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13C .

[0575] Exemplarily, as shown in FIG13C , if the shooting parameter generation module obtains the return frame moment of image frame 1 as moment t3, the shooting parameter generation module can obtain the corresponding transition zoom ratio 1 from the zoom smoothing adjustment curve based on moment t3.

[0576] In some embodiments, the initial zoom ratio may be greater than the target zoom ratio, that is, the zoom ratio of the camera application is reduced. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13D .

[0577] Exemplarily, as shown in FIG13D , if the shooting parameter generation module obtains the return frame moment of image frame 1 as moment t3, the shooting parameter generation module can obtain the corresponding transition zoom ratio 1 from the zoom smoothing adjustment curve based on moment t3.

[0578] S1212: The shooting parameter generation module sends the third camera request (first transition zoom ratio) to the request queue processing module.

[0579] The third camera request includes a transition zoom factor of 1. Optionally, the second camera request may also include other shooting parameters, such as initial exposure, initial aperture, initial white balance, initial sharpness, initial contrast, and initial saturation. The third camera request may not include other shooting parameters, and this application does not limit this.

[0580] In some embodiments, the first transition zoom ratio may also be sent to the request queue processing module in a time-sharing manner together with the third camera request, which is not limited in this application.

[0581] S1213: The request queue processing module allocates a second buffer for the third camera request.

[0582] In response to the third camera request sent by the shooting parameter generating module, the request queue processing module may allocate a second buffer for the third camera request.

[0583] S1214: The request queue processing module sends the third camera request and the address of the second buffer to the HAL layer.

[0584] S1215 : The HAL layer sends the third camera request and the address of the second buffer to the camera module.

[0585] After allocating the second buffer for the third camera request, the request queue processing module sends the third camera request and the address of the second buffer to the HAL layer, and the HAL layer then sends the third camera request and the address of the second buffer to the camera module.

[0586] After receiving the third camera request, the camera module may capture image frames based on the transition zoom factor 1 carried in the third camera request.

[0587] It is understandable that, based on the analysis of the embodiment of FIG4 , before the HAL layer processes the third camera request, there are N camera requests to be processed before the third camera request. The third camera request will not be processed immediately.

[0588] S1216: The shooting parameter generation module sends the first transition zoom ratio to the image cropping module.

[0589] After the shooting parameter generation module determines the first transitional zoom ratio based on the zoom smoothing adjustment curve and the return frame time of image frame 1, it not only sends a third camera request to the request queue processing module but also sends the first transitional zoom ratio to the image cropping module. This allows the image cropping module to crop image frame 1 based on the first transitional zoom ratio and promptly display the image frame corresponding to the first transitional zoom ratio, thereby accelerating the electronic device's response to the user switching the zoom ratio.

[0590] Optionally, S1216 may be executed simultaneously with S1212, or S1216 may be executed before S1212, which is not limited in this application.

[0591] S1217: The image cropping module determines a cropping ratio 1 based on the zoom magnification of the image frame 1 and the first transition zoom magnification.

[0592] S1218 . The image cropping module crops the image frame 1 based on the cropping ratio 1 to obtain a transition image frame 1 .

[0593] For example, the zoom magnification of the image frame 1 may be an initial zoom magnification. The image cropping module may determine the cropping ratio 1 based on the zoom magnification of the image frame 1 and the first transition zoom magnification.

[0594] The image cropping module may crop the image frame 1 based on the cropping ratio 1 to obtain a cropped image 1, and then scale the cropped image 1 to obtain a transition image frame 1. The zoom ratio of the transition image frame 1 may be a first transition zoom ratio.

[0595] In some embodiments, the cropped image 1 can be obtained based on formula (2) and formula (3). Cropping ratio 1 = zoom ratio of image frame 1 / first transition zoom ratio Formula (2) Size of cropped image 1 = size of image frame 1 * cropping ratio 1 Formula (3)

[0596] In some embodiments, if the zoom magnification of image frame 1 is greater than the zoom magnification of the first transition zoom magnification, then crop ratio 1 is greater than 1, and the size of cropped image 1 is greater than the size of image frame 1. If the zoom magnification of image frame 1 is less than the zoom magnification of the first transition zoom magnification, then crop ratio 1 is less than 1, and the size of cropped image 1 is less than the size of image frame 1. If the zoom magnification of image frame 1 is equal to the zoom magnification of the first transition zoom magnification, then crop ratio 1 is equal to 1, and the size of cropped image 1 is equal to the size of image frame 1.

[0597] Exemplarily, when the zoom ratio of the image frame 1 is less than the first transition zoom ratio, that is, the cropping ratio 1 is less than 1, the zoom ratio changes from small to large.

[0598] As shown in FIG14A , the electronic device 100 may crop the image frame 1 based on the size of the cropping image 1 to obtain the cropping image 1. Thereafter, the electronic device 100 further scales the cropping image 1 to obtain the transition image frame 1.

[0599] In some embodiments, the transition image frame 1 can be obtained based on formula (4) and formula (5). Scaling ratio 1 = first transition zoom ratio / zoom ratio of image frame 1 Formula (4) Size of transition image frame 1 = size of cropped image 1 * scaling ratio 1 Formula (5)

[0600] In some embodiments, if the zoom magnification of image frame 1 is greater than the zoom magnification of the first transition zoom magnification, then scaling factor 1 is less than 1, and the size of cropped image 1 is greater than the size of image frame 1. If the zoom magnification of image frame 1 is less than the zoom magnification of the first transition zoom magnification, then scaling factor 1 is greater than 1, and the size of cropped image 1 is smaller than the size of image frame 1. If the zoom magnification of image frame 1 is equal to the zoom magnification of the first transition zoom magnification, then scaling factor 1 is equal to 1, and the size of cropped image 1 is equal to the size of image frame 1.

[0601] Exemplarily, when the zoom ratio of image frame 1 is less than the first transition zoom ratio, that is, zoom ratio 1 is greater than 1.

[0602] As shown in FIG14A , the electronic device 100 can stretch the cropped image 1 based on the cropped image 1 to obtain a transitional image frame 1. The size of the transitional image frame 1 is equal to the size of the image frame 1. Based on this method, the image frame 1 can be cropped to obtain the transitional image frame 1.

[0603] In some embodiments, when the zoom ratio of image frame 1 is greater than the first transition zoom ratio, that is, the cropping ratio 1 is greater than 1, the zoom ratio changes from large to small. In order to make it possible to crop the transition image frame 1 based on image frame 1. The electronic device 100 can acquire the original image frame based on the preset zoom ratio, and the image frame 1 can be cropped based on the original image frame. When it is necessary to obtain the transition image frame 1 based on image frame 1, the electronic device 100 can crop the transition image frame 1 based on the original image frame. The preset zoom ratio can be a minimum value, such as 0.5x. In this way, the original image frame acquired by the electronic device 100 at the minimum value of 0.5x can be used as input for any other zoom ratio greater than the minimum value of 0.5x, and any other image frame with a zoom ratio greater than the minimum value of 0.5x can be cropped based on the original image frame acquired at the minimum value of 0.5x.

[0604] The minimum value is not limited to 0.5x, and can also be other values, which is not limited in this application.

[0605] S1219 . The image cropping module sends the transition image frame 1 and the address of the buffer storing the transition image frame 1 to the buffer manager.

[0606] In some embodiments, the address of the buffer storing the transition image frame 1 and the address of the buffer storing the image frame 1 may be the same or different.

[0607] S1220 . The electronic device 100 renders and displays the transition image frame 1 in the manner of rendering and displaying the image frame A as shown in S912 - S917 .

[0608] The display process module may obtain an image frame from the buffer manager every m vsync signal cycles. For example, after m vsync signal cycles have accumulated, the display process may obtain an image frame, such as transition image frame 1, from the buffer manager.

[0609] Optionally, the display processing module may also receive a timestamp sent by a timer. When the time interval between the current time and the timestamp of the previous image frame sent for rendering is greater than or equal to a preset time interval, the display processing module may obtain the image frame to be rendered from the buffer manager. For example, the display processing module may obtain transition image frame 1 to be rendered from the buffer manager.

[0610] After accumulating m vsync signal cycles, the display sending process can obtain the transition image frame 1 from the buffer manager, and then send the transition image frame 1 or the address of the buffer storing the transition image frame 1 to the camera application, and the camera application then sends the transition image frame 1 or the address of the buffer storing the transition image frame 1 to the rendering process.

[0611] After the rendering process obtains the transition image frame 1 or the address of the buffer storing the transition image frame 1, the rendering process may render the transition image frame 1 to obtain the rendered transition image frame 1.

[0612] In some examples, if the rendering process module obtains the address of the buffer storing transition image frame 1, the rendering process module may render transition image frame 1 in the buffer corresponding to the address of the buffer storing transition image frame 1. Alternatively, if the rendering process module obtains the address of the buffer storing transition image frame 1, the rendering process module may retrieve transition image frame 1 from the buffer corresponding to the address of the buffer storing transition image frame 1 and render transition image frame 1.

[0613] The rendering process module may send the rendered transition image frame 1 or the address of the buffer storing the rendered transition image frame 1 to the camera application. The camera application then sends the transition image frame 1 or the address of the buffer storing the rendered transition image frame 1 to the display.

[0614] After the display obtains the transition image frame 1 or the address of the buffer storing the rendered transition image frame 1 , the display may display the rendered transition image frame 1 .

[0615] In some examples, if the display obtains the address of the buffer storing the rendered transition image frame 1, the rendering process module can take out the rendered transition image frame 1 from the buffer corresponding to the address of the buffer storing the rendered transition image frame 1 and display the rendered transition image frame 1.

[0616] In this way, the shooting parameter generation module can obtain multiple transition zoom ratios, and cut out the image frames corresponding to the multiple transition zoom ratios in sequence according to the method of S1210-S1219.

[0617] For example, after image frame 1, the shooting parameter generation module may further obtain the frame return time of image frame 2, which is the image frame after image frame 1. The shooting parameter generation module needs to determine whether the frame return time of image frame 2 is later than the time of the target zoom factor in the zoom smoothing adjustment curve, such as time t2.

[0618] When the return frame time of the image frame 2 is earlier than the time of the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module needs to continue to obtain other transition zoom ratios and obtain image frames based on the other transition zoom ratios.

[0619] In some embodiments, the initial zoom ratio may be smaller than the target zoom ratio, that is, the zoom ratio of the camera application is increased. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13C .

[0620] Exemplarily, as shown in FIG13C , if the shooting parameter generation module obtains the return frame moment of image frame 2 as moment t4, the shooting parameter generation module can obtain the corresponding transition zoom ratio 2 from the zoom smoothing adjustment curve based on moment t4.

[0621] In some embodiments, the initial zoom ratio may be greater than the target zoom ratio, that is, the zoom ratio of the camera application is reduced. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13D .

[0622] Exemplarily, as shown in FIG13D , if the shooting parameter generation module obtains the return frame moment of image frame 2 as moment t4, the shooting parameter generation module can obtain the corresponding transition zoom ratio 2 from the zoom smoothing adjustment curve based on moment t4.

[0623] After determining the transition zoom magnification 2, the electronic device 100 may crop, render, and display the image frame corresponding to the transition zoom magnification 2 in a manner similar to S1210-S1219.

[0624] For example, after image frame 2, the shooting parameter generation module may further obtain the frame return time of image frame 3, which is the image frame after image frame 2. The shooting parameter generation module needs to determine whether the frame return time of image frame 3 is later than the time of the target zoom factor in the zoom smoothing adjustment curve, such as time t2.

[0625] When the return frame time of the image frame 3 is earlier than the time of the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module needs to continue to obtain other transition zoom ratios and obtain image frames based on the other transition zoom ratios.

[0626] In some embodiments, the initial zoom ratio may be smaller than the target zoom ratio, that is, the zoom ratio of the camera application is increased. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13C .

[0627] Exemplarily, as shown in FIG13C , if the shooting parameter generation module obtains the return frame moment of image frame 3 as moment t5, the shooting parameter generation module can obtain the corresponding transition zoom ratio 3 from the zoom smoothing adjustment curve based on moment t5.

[0628] In some embodiments, the initial zoom ratio may be greater than the target zoom ratio, that is, the zoom ratio of the camera application is reduced. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG13D .

[0629] Exemplarily, as shown in FIG13D , if the shooting parameter generation module obtains the return frame moment of image frame 3 as moment t5, the shooting parameter generation module can obtain the corresponding transition zoom ratio 3 from the zoom smoothing adjustment curve based on moment t5.

[0630] After determining the transition zoom magnification 3, the electronic device 100 may crop, render, and display the image frame corresponding to the transition zoom magnification 3 in a manner similar to S1210-S1219.

[0631] This process continues in this manner until the result processing module obtains an image frame returned by the HAL layer at a time later than the target zoom factor in the zoom smoothing adjustment curve, for example, time t2. Zooming ends, and the shooting parameter generation module no longer determines other transitional zoom factors. After the result processing module obtains another image frame returned by the HAL layer, it proceeds to step S1220.

[0632] Optionally, during the zoom process, the HAL layer can obtain the zoom magnification of the image frame and the image frame return time, and obtain the zoom curve shown in Figure 13C or the zoom curve shown in Figure 13D based on the zoom magnification of the image frame and the image frame return time. The zoom curve shown in Figure 13C or Figure 13D reflects the zoom process of the image frame displayed by the electronic device 100 during the zoom process.

[0633] As shown in Figure 13C or Figure 13D, if the user clicks the target zoom factor option at time t1, the time when the HAL layer returns image frame 1 to the application architecture layer is time t6. It takes time for the application architecture layer to send a camera request to the HAL layer, it also takes time for the HAL layer to send a camera request to the camera module, and it also takes time for the camera module to process the image. The time difference between time t1 and time t6 is the first duration. The first duration is the delay in communication between layers and the delay in image processing. If the HAL layer starts returning images captured at the transition zoom factor to the application architecture layer at or after time t6, the HAL layer can obtain the zoom curve shown in Figure 13C or Figure 13D.

[0634] Based on the above analysis, we can see that, first, the calculation process of the transition zoom ratio is completed in the application architecture layer, which reduces the interaction steps between the application layer and the application architecture layer, saves the time for the application layer to send the transition zoom ratio to the application architecture layer, and after the HAL layer returns the image frame, the application framework layer can send the zoom request to the HAL layer in a timely manner. Secondly, when the HAL layer returns the image frame to the application architecture layer, the application architecture layer can determine the transition zoom ratio in real time based on the return frame time of the image frame returned by the HAL layer, so that the zoom curve corresponding to the image frame returned by the HAL layer during the zoom process conforms to the change law of the zoom smooth adjustment curve, making the zoom process smoother, and the zoom ratio can transition from the initial zoom ratio to the target zoom ratio according to the change law of the zoom smooth adjustment curve, without image jitter or freeze, thereby improving the user's visual experience.

[0635] S1221. The electronic device 100 executes the embodiment shown in FIG12C.

[0636] When the return time of image frame 1 is later than the target zoom ratio in the zoom smoothing curve, the electronic device 100 executes the embodiment shown in FIG12C .

[0637] FIG12C shows a schematic flow chart of another shooting method provided in an embodiment of the present application.

[0638] S1301: The shooting parameter generation module sends the fourth camera request (target zoom ratio) to the request queue processing module.

[0639] When the return frame time of image frame 1 is later than the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module does not need to obtain other transition zoom ratios, and the shooting parameter generation module can send the target zoom ratio to the HAL layer.

[0640] In response to the frame return time of the image frame 1 being later than the time of the target zoom magnification in the zoom smoothing adjustment curve, the shooting parameter generating module may send a fourth camera request to the request queue processing module.

[0641] The fourth camera request includes the target zoom ratio. Optionally, the fourth camera request may also include other shooting parameters, such as initial exposure, initial aperture, initial white balance, initial sharpness, initial contrast, and initial saturation. The fourth camera request may not include other shooting parameters, and this application does not limit this.

[0642] In some embodiments, the target zoom ratio and the fourth camera request may also be sent to the request queue processing module in a time-sharing manner, which is not limited in this application.

[0643] S1302: The request queue processing module allocates a third buffer for the fourth camera request.

[0644] In response to the fourth camera request sent by the shooting parameter generating module, the request queue processing module may allocate a third buffer for the fourth camera request.

[0645] S1303: The request queue processing module sends the fourth camera request and the address of the third buffer to the HAL layer.

[0646] S1304: The HAL layer sends the fourth camera request and the address of the third buffer to the camera module.

[0647] After allocating the third buffer for the fourth camera request, the request queue processing module sends the fourth camera request and the address of the third buffer to the HAL layer, and the HAL layer then sends the fourth camera request and the address of the third buffer to the camera module.

[0648] After receiving the fourth camera request, the camera module may capture image frames based on the target zoom ratio carried in the fourth camera request.

[0649] It is understandable that, based on the analysis of the embodiment of FIG4 , before the HAL layer processes the fourth camera request, there are N camera requests to be processed before the fourth camera request. The fourth camera request will not be processed immediately.

[0650] S1305: The image cropping module needs to determine whether the zoom ratio of the image frame 1 is the same as the target zoom ratio.

[0651] When the return frame time of image frame 1 is later than the target zoom ratio in the zoom smoothing adjustment curve, the image cropping module does not need to obtain other transition zoom ratios, and the image cropping module directly sends the target zoom ratio to the image cropping module.

[0652] In order to determine whether the image cropping module needs to crop the image, the image cropping module needs to determine whether the zoom ratio of the image frame 1 is the same as the target zoom ratio.

[0653] When the zoom ratio of image frame 1 is the same as the target zoom ratio, it means that image frame 1 is an image frame captured by the camera module with the target zoom ratio. In this case, the image cropping module does not need to crop image frame 1, but directly sends image frame 1 or the address of the buffer storing image frame 1 to the buffer manager to execute S1306.

[0654] When the zoom ratio of image frame 1 is different from the target zoom ratio, it means that image frame 1 is not an image frame captured by the camera module at the target zoom ratio, but an image frame captured at a transition zoom ratio. In this case, the image cropping module needs to crop image frame 1 and execute S1307.

[0655] S1306 . The electronic device 100 renders and displays the image frame 1 in the manner of rendering and displaying the image frame A as shown in S912 - S917 .

[0656] If the zoom magnification of image frame 1 is the same as the target zoom magnification, the image cropping module does not need to crop image frame 1 and directly sends image frame 1 or the address of the buffer storing image frame 1 to the buffer manager. This allows the display process to periodically obtain image frames from the buffer manager and render them according to m vsync signals. For details, please refer to the description in S1219, and this application will not repeat them here.

[0657] S1307 : The image cropping module determines a cropping ratio 2 based on the zoom ratio of the image frame 1 and the target zoom ratio.

[0658] S1308 : The image cropping module crops the image frame 1 based on the cropping ratio 2 to obtain a transition image frame 2 .

[0659] For example, the zoom magnification of the image frame 1 may be an initial zoom magnification. The image cropping module may determine the cropping ratio 2 based on the zoom magnification of the image frame 1 and the target zoom magnification.

[0660] The image cropping module may crop the image frame 1 based on the cropping ratio 2 to obtain the transition image frame 2. The zoom ratio of the transition image frame 2 may be the target zoom ratio.

[0661] S1309 : The image cropping module sends the transition image frame 2 and the address of the buffer storing the transition image frame 2 to the buffer manager.

[0662] In some embodiments, the address of the buffer storing the transition image frame 2 and the address of the buffer storing the image frame 1 may be the same or different.

[0663] S1310 : The electronic device 100 renders and displays the transition image frame 2 in the manner of rendering and displaying the image frame A as shown in S912 - S917 .

[0664] The display sending process module may obtain an image frame from the buffer manager every m vsync signal cycles. For example, when m vsync signal cycles have accumulated, the display sending process may obtain an image frame, such as transition image frame 2, from the buffer manager.

[0665] Optionally, the display processing module may also receive a timestamp sent by a timer. When the time interval between the current time and the timestamp of the previous image frame sent for rendering is greater than or equal to a preset time interval, the display processing module may obtain the image frame to be rendered from the buffer manager. For example, the display processing module may obtain transition image frame 2 to be rendered from the buffer manager.

[0666] After accumulating m vsync signal cycles, the display sending process can obtain the transition image frame 2 from the buffer manager, and then send the transition image frame 2 or the address of the buffer storing the transition image frame 2 to the camera application, and the camera application then sends the transition image frame 2 or the address of the buffer storing the transition image frame 2 to the rendering process.

[0667] After the rendering process obtains the transition image frame 2 or the address of the buffer storing the transition image frame 2, the rendering process can render the transition image frame 1 to obtain the rendered transition image frame 2.

[0668] In some examples, if the rendering process module obtains the address of the buffer storing transition image frame 2, the rendering process module may render transition image frame 2 in the buffer corresponding to the address of the buffer storing transition image frame 2. Alternatively, if the rendering process module obtains the address of the buffer storing transition image frame 2, the rendering process module may retrieve transition image frame 2 from the buffer corresponding to the address of the buffer storing transition image frame 2 and render transition image frame 2.

[0669] The rendering process module may send the rendered transition image frame 2 or the address of the buffer storing the rendered transition image frame 2 to the camera application. The camera application then sends the transition image frame 2 or the address of the buffer storing the rendered transition image frame 2 to the display.

[0670] After the display obtains the transition image frame 2 or the address of the buffer storing the rendered transition image frame 2 , the display may display the rendered transition image frame 2 .

[0671] In some examples, if the display obtains the address of the buffer storing the rendered transition image frame 2, the rendering process module can take out the rendered transition image frame 2 from the buffer corresponding to the address of the buffer storing the rendered transition image frame 2 and display the rendered transition image frame 2.

[0672] And so on, more image frames can be obtained.

[0673] FIG. 14B shows a schematic diagram of how another electronic device 100 processes a camera request and a zoom request.

[0674] As shown in Figure 4, the application framework periodically sends camera requests to the HAL layer. Before the HAL layer responds to the zoom request corresponding to the user's zoom operation, it must process other pending camera requests. The HAL layer only processes the zoom request after all previous camera requests have been processed.

[0675] For example, as shown in Figure 14B, before the HAL layer processes the zoom request, the HAL layer has six other camera requests to be processed, namely camera request 1, camera request 2, camera request 3, camera request 4, camera request 5, and camera request 6. If at time tms, the HAL layer receives zoom request 1, zoom request 1 carries a transition zoom magnification 1, and places zoom request 1 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at time tms, the HAL layer removes camera request 1 from the queue and processes camera request 1, which carries the initial zoom magnification. In response to processing camera request 1, the camera module can capture image frame A based on the initial zoom magnification and send it to the application framework layer through the HAL layer. After acquiring image frame A, the application framework layer, if determining that the return time of image frame A is later than the target zoom magnification in the zoom smoothing adjustment curve, can determine a cropping ratio a based on the zoom magnification of image frame A and transitional zoom magnification 1, and crop image frame A based on the cropping ratio a to obtain transitional image frame A1, which has a transitional zoom magnification of 1. The application framework layer then sends transitional image frame A1 to the camera application, which can display transitional image frame A1.

[0676] At (t+33.3) ms, the HAL layer receives zoom request 2, which carries a transition zoom magnification 2, and places zoom request 2 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at (t+33.3) ms, the HAL layer takes out camera request 2 from the queue and processes camera request 2, which carries an initial zoom magnification. In response to processing camera request 2, the camera module can capture image frame B based on the initial zoom magnification and send it to the application framework layer through the HAL layer. After obtaining image frame B, the application framework layer can determine that the return frame time of image frame B is later than the target zoom magnification in the zoom smoothing adjustment curve. The application framework layer can determine the cropping ratio b based on the zoom magnification of image frame B and the transition zoom magnification 2, and crop image frame B based on the cropping ratio b to obtain transition image frame B1. Transition image frame B1 has a transition zoom magnification 2. The application framework layer then sends the transition image frame B1 to the camera application, and the camera application can display the transition image frame B1.

[0677] At (t+66.6) ms, the HAL layer receives zoom request 3, which carries a transition zoom factor 3, and places zoom request 3 in a queue of pending requests in chronological order. Following the first-in-first-out principle, at (t+66.6) ms, the HAL layer removes camera request 3 from the queue and processes it. Camera request 3 carries the initial zoom factor. In response to processing camera request 3, the camera module captures image frame C based on the initial zoom factor and sends it to the application framework layer through the HAL layer. After acquiring image frame C, the application framework layer determines that the return time of image frame C is later than the target zoom factor in the zoom smoothing adjustment curve, and that the zoom factor of image frame C is different from the target zoom factor. Based on the zoom factor of image frame C and the transition zoom factor 3, the application framework layer determines a cropping ratio c, and crops image frame C based on the cropping ratio c, to obtain transition image frame C1, which has a transition zoom factor 3. The application framework layer then sends the transition image frame C1 to the camera application, and the camera application can display the transition image frame C1.

[0678] At (t+99.9) ms, the HAL layer receives zoom request 4. Since the return frame time of image frame D is later than the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module no longer confirms other transition zoom ratios and directly sends the target zoom ratio to the HAL layer. Therefore, zoom request 4 carries the target zoom ratio and places zoom request 4 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at (t+99.9) ms, the HAL layer takes out camera request 4 from the queue and processes camera request 4. Camera request 4 carries the initial zoom ratio. In response to processing camera request 4, the camera module can capture image frame D based on the initial zoom ratio and send it to the application framework layer through the HAL layer. After acquiring image frame D, if the zoom magnification of image frame D differs from the target zoom magnification, the application framework layer can determine a cropping ratio d based on the zoom magnification of image frame D and the target zoom magnification, and crop image frame D based on the cropping ratio d to obtain transitional image frame D1, which has the target zoom magnification. The application framework layer then sends transitional image frame D1 to the camera application, which can display transitional image frame D1.

[0679] At (t+133.2) ms, the HAL layer receives the zoom request 5, which carries the target zoom magnification, and places the zoom request 5 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at (t+133.2) ms, the HAL layer takes out the camera request 5 from the queue and processes the camera request 5, which carries the initial zoom magnification. In response to processing the camera request 5, the camera module can capture the image frame E based on the initial zoom magnification and send it to the application framework layer through the HAL layer. After the application framework layer obtains the image frame E, if the zoom magnification of the image frame E is different from the target zoom magnification, the application framework layer can determine the cropping ratio e based on the zoom magnification of the image frame E and the target zoom magnification, and crop the image frame E based on the cropping ratio e to obtain the transition image frame E1, which has the target zoom magnification. The application framework layer then sends the transition image frame E1 to the camera application, and the camera application can display the transition image frame E1.

[0680] At (t+166.5) ms, the HAL layer receives zoom request 6, which carries the target zoom magnification, and places zoom request 6 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at (t+166.5) ms, the HAL layer takes out camera request 6 from the queue and processes camera request 6, which carries the initial zoom magnification. In response to processing camera request 6, the camera module can capture image frame F based on the initial zoom magnification and send it to the application framework layer through the HAL layer. After the application framework layer obtains image frame F, if the zoom magnification of image frame F is different from the target zoom magnification, the application framework layer can determine the cropping ratio f based on the zoom magnification of image frame F and the target zoom magnification, and crop image frame F based on the cropping ratio f to obtain a transition image frame F1, which is the target zoom magnification. The application framework layer then sends the transition image frame F1 to the camera application, and the camera application can display the transition image frame F1.

[0681] At (t+199.8) ms, the HAL layer receives zoom request 7, which carries the target zoom magnification, and places zoom request 7 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at (t+199.8) ms, the HAL layer takes out camera request 7 from the queue and processes camera request 7, which carries the initial zoom magnification. In response to processing camera request 7, the camera module can capture image frame G based on the transition zoom magnification 1 and send it to the application framework layer through the HAL layer. After obtaining image frame G, if the zoom magnification of image frame G is different from the target zoom magnification, the application framework layer can determine the cropping ratio g based on the zoom magnification of image frame G and the target zoom magnification, and crop image frame G based on the cropping ratio g to obtain transition image frame G1, which is the target zoom magnification. The application framework layer then sends the transition image frame G1 to the camera application, and the camera application can display the transition image frame G1.

[0682] By analogy, the electronic device 100 may process zoom request 2 and zoom request 3 in the same manner as processing zoom request 1.

[0683] At (t+299.7) ms, the HAL layer receives zoom request 10, which carries the target zoom ratio, and places zoom request 10 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at (t+299.7) ms, the HAL layer takes out zoom request 4 from the queue and processes zoom request 4, which carries the target zoom ratio. In response to processing zoom request 4, the camera module can capture image frame H based on the target zoom ratio and send it to the application framework layer through the HAL layer. After the application framework layer obtains the image frame H, if the zoom ratio of image frame H is the same as the target zoom ratio, the application framework layer does not need to crop the image frame G. The application framework layer can directly send image frame G to the camera application, and the camera application can display image frame G.

[0684] By analogy, the electronic device 100 may process zoom request 5 to zoom request 10 in the same manner as that of processing zoom request 4.

[0685] As shown in Figure 14B , immediately after the HAL layer receives zoom request 1, the application framework layer can immediately respond to the transitional zoom magnification 1 carried in zoom request 1 and obtain transitional image frame A1, where the zoom magnification of transitional image frame A1 is transitional zoom magnification 1. This eliminates the need to wait for multiple camera requests preceding zoom request 1 to complete before executing zoom request 1 and obtaining the image frame corresponding to transitional zoom magnification 1. This shortens the time it takes the HAL layer to process zoom request 1 and increases the speed with which the camera application responds to user zoom operations.

[0686] FIG. 14C shows a timing diagram of another electronic device 100 rendering and displaying image frames in a vsync signal cycle.

[0687] For example, the display frame rate may be 30 FPS, and the vsync signal period is 33.3 ms, then m is 1, that is, the rendering process renders one image frame in each vsync signal period, and the camera application sends one image frame for display in each vsync signal period.

[0688] As shown in Figure 14C , between vertical synchronization signal 1 (vsync1) and vertical synchronization signal 2 (vsync2), the camera application can issue rendering instruction A to the rendering process. In response to rendering instruction A, the rendering process can render transitional image frame A1 and store the rendered transitional image frame A1 in a buffer. Simultaneously, between vertical synchronization signal 1 (vsync) and vertical synchronization signal 2 (vsync2), the camera application can send image frame 0 or the image frame 0 buffer to the display, allowing the display to display image frame 0. That is, between time tms and time (t+33.3ms), the rendering process renders transitional image frame A1, and the display displays image frame 0.

[0689] Between vertical synchronization signal 2 (vsync2) and vertical synchronization signal 3 (vsync3), the camera application can send rendering instruction B to the rendering process. In response to rendering instruction B, the rendering process can render transition image frame B1 and fill the rendered transition image frame B1 into the buffer. At the same time, between vertical synchronization signal 2 (vsync2) and vertical synchronization signal 3 (vsync3), the camera application can send transition image frame A1 or the buffer of transition image frame A1 to the display, so that the display can display transition image frame A1. That is, between time (t+33.3ms)ms and time (t+66.6ms), the rendering process renders transition image frame B1, and the display displays transition image frame A1.

[0690] Between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 4 (vsync4), the camera application can send rendering instruction C to the rendering process. In response to rendering instruction C, the rendering process can render transition image frame C1 and fill the rendered transition image frame C1 into the buffer. At the same time, between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 4 (vsync4), the camera application can send transition image frame B1 or the buffer of transition image frame B1 to the display, so that the display can display transition image frame B1. That is, between time (t+66.6ms)ms and time (t+99.9ms), the rendering process renders transition image frame C1, and the display displays transition image frame B1.

[0691] Between vertical synchronization signal 4 (vsync4) and vertical synchronization signal 5 (vsync5), the camera application can send rendering instruction D to the rendering process. In response to rendering instruction D, the rendering process can render transition image frame D1 and fill the rendered transition image frame D1 into the buffer. At the same time, between vertical synchronization signal 4 (vsync4) and vertical synchronization signal 5 (vsync5), the camera application can send transition image frame C1 or the buffer of transition image frame C1 to the display, so that the display can display transition image frame C1. That is, between time (t+99.9ms)ms and time (t+133.2ms), the rendering process renders transition image frame B1, and the display displays transition image frame C1.

[0692] Between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 6 (vsync6), the camera application can send rendering instruction E to the rendering process. In response to rendering instruction E, the rendering process can render transition image frame E1 and fill the rendered transition image frame E1 into the buffer. At the same time, between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 6 (vsync6), the camera application can send transition image frame D1 or the buffer of transition image frame D1 to the display, so that the display can display transition image frame D1. That is, between time (t+133.2ms)ms and time (t+166.5ms), the rendering process renders transition image frame E1, and the display displays transition image frame D1.

[0693] Between vertical synchronization signal 6 (vsync6) and vertical synchronization signal 7 (vsync7), the camera application can send a rendering instruction F to the rendering process. In response to rendering instruction F, the rendering process can render transition image frame F1 and fill the rendered transition image frame F1 into the buffer. At the same time, between vertical synchronization signal 6 (vsync6) and vertical synchronization signal 7 (vsync7), the camera application can send transition image frame E1 or the buffer of transition image frame E1 to the display, so that the display can display transition image frame E1. That is, between time (t+166.5ms)ms and time (t+133.2ms), the rendering process renders transition image frame F1, and the display displays transition image frame E1.

[0694] Between vertical synchronization signal 7 (vsync7) and vertical synchronization signal 8 (vsync8), the camera application can send a rendering instruction G to the rendering process. In response to rendering instruction G, the rendering process can render transition image frame G1 and fill the rendered transition image frame G1 into the buffer. At the same time, between vertical synchronization signal 7 (vsync7) and vertical synchronization signal 8 (vsync8), the camera application can send transition image frame F1 or the buffer of transition image frame F1 to the display, so that the display can display transition image frame F1. That is, between time (t+133.2ms)ms and time (t+166.5ms), the rendering process renders transition image frame G1, and the display displays transition image frame F1.

[0695] Similarly, between vertical synchronization signal 10 (vsync10) and vertical synchronization signal 11 (vsync11), the camera application can send a rendering instruction H to the rendering process. In response to rendering instruction H, the rendering process can render transition image frame H1 and fill the rendered transition image frame H1 into the buffer. At the same time, between vertical synchronization signal 10 (vsync10) and vertical synchronization signal 11 (vsync11), the camera application can send transition image frame I1 or the buffer of transition image frame I1 to the display, so that the display can display transition image frame I1. That is, between time (t+266.4ms)ms and time (t+299.7ms), the rendering process renders transition image frame H1, and the display displays transition image frame I1.

[0696] Similarly, the camera application can render and display image frames based on the vsync signal cycle in the manner shown in FIG14C .

[0697] As can be seen from Figure 14C, the rendering process can send the image frames cropped by the application framework layer or the image frames sent by the HAL layer received by the application framework layer to the rendering process for rendering and then display according to a fixed vsync signal period. This can ensure that the image frames are evenly sent to the rendering process for rendering and then display, ensuring that the camera application evenly displays the image frames.

[0698] Based on the foregoing description, it can be seen that in order to ensure that the electronic device 100 can smoothly switch from the initial zoom magnification to the target zoom magnification, the electronic device 100 can generate a zoom smoothing adjustment curve based on the initial zoom magnification and the target zoom magnification. Based on the description of the embodiment of Figures 13A-13D, it can be seen that the slope of the zoom smoothing adjustment curve changes from small to large and then to small, that is, the speed of change of the transition zoom magnification first changes from small to large and then to small.

[0699] In some embodiments, if the electronic device 100 obtains multiple transition zoom ratios according to the method shown in the embodiments of Figures 13C and 13D, since the slope of the zoom smooth adjustment curve changes from small to large and then to small, that is, the change speed of the transition zoom ratio in the zoom smooth adjustment curve first changes from small to large and then to small, it can be seen from Figures 13C and 13D that the change difference between two adjacent transition zoom ratios in the multiple transition zoom ratios is too large, for example, the change value between transition zoom ratio 1 and transition zoom ratio 2, and the change value between transition zoom ratio 2 and transition zoom ratio 3 are quite different, and the change value between transition zoom ratio 2 and transition zoom ratio 3, and the change value between transition zoom ratio 3 and the target zoom ratio are quite different, that is, the sudden change of zoom ratio affects the user's visual effect.

[0700] In order to reduce the change value of the zoom ratio corresponding to two adjacent image frames during the zooming process, between the two adjacent transition zoom ratios shown in Figures 13C and 13D, the electronic device 100 can further determine a transition zoom ratio to reduce the change value of the zoom ratio corresponding to the two adjacent image frames, so as to ease the change speed of the zoom ratio corresponding to the two adjacent image frames. Wherein, a is a positive integer greater than or equal to 1, and the value of a can be a preset value, for example, a can be 1 or 2. The following embodiments of the present application are described with a being 1.

[0701] FIG15A shows a flow chart of another shooting method provided in an embodiment of the present application.

[0702] S1501-S1520 can refer to the description of S1201-S1219 in the embodiment of Figure 12C, and this application will not repeat them here.

[0703] S1521. The shooting parameter generation module notifies the rendering process to switch the vsync signal period from the first value to the second value, where the ratio of the second value to the first value is (a+1).

[0704] The first value and the second value are both positive integers greater than or equal to 1, the second value is greater than the first value, and the ratio of the second value to the first value is a+1.

[0705] Wherein, a is 0 or a positive integer greater than or equal to 1, and the value of a can be a preset value.

[0706] For example, if the value of a is 1, the ratio of the second value to the first value may be 2, that is, the rendering process shortens the vsync signal period by half. For example, the first value may be 33.3ms and the second value may be 16.7ms.

[0707] For another example, if the value of a is 2, the ratio of the second value to the first value may be 3, that is, the rendering process shortens the vsync signal period by one third. For example, the first value may be 33.3ms and the second value may be 8.35ms.

[0708] The following examples of this application are described with a being 1.

[0709] In some embodiments, when a is 0, no frame is inserted. The specific implementation is similar to the embodiment of Figure 12C, and this application will not repeat it here.

[0710] Optionally, S1521 may be executed before, after, or simultaneously with any step after S1501, and this application does not limit this.

[0711] S1522: The image cropping module sends the return frame time of the transition image frame 3 to the shooting parameter generation module. The return frame time of the transition image frame 3 is equal to the sum of the return frame time of the image frame 1 and the preset duration.

[0712] The return frame time of the transition image frame 1 is equal to the return frame time of the image frame 1.

[0713] Exemplarily, the return frame time of image frame 1 may be time t3.

[0714] Exemplarily, the return frame time of the transition image frame 3 may be time t7.

[0715] Optionally, the preset duration may be related to the image output frame rate of the camera application and the value of a, for example, preset duration = (a + 1) / (1 / (image output frame rate). For example, if the image output frame rate of the camera application is 30 pfs, that is, the HAL layer returns an image frame to the application framework layer every 33.3 ms, when a is 1, the preset duration may be 16.7 ms.

[0716] For another example, if the output frame rate of the camera application is 60pfs, that is, the HAL layer returns an image frame to the application framework layer every 16.7ms, when a is 1, the preset duration can be 8.35ms.

[0717] Based on the above analysis, in order to reduce the change value of the zoom ratio corresponding to two adjacent image frames, the image cropping module can add one or more transition zoom ratios between two adjacent transition zoom ratios, and crop the image frame based on the newly added one or more transition zoom ratios to obtain a newly added transition image frame, and render and display the newly added transition image frame.

[0718] For example, two adjacent transition zoom magnifications may be transition zoom magnification 1 and transition zoom magnification 2. The shooting parameter generation module may determine transition zoom magnification 4 between transition zoom magnification 1 and transition zoom magnification 2, crop the image frame returned by the HAL layer based on transition zoom magnification 4 to obtain transition image frame 3, and render and display transition image frame 3. Similarly, during the zoom process, the number of image frames may be increased to slow down the change in zoom magnification between two adjacent image frames.

[0719] For example, if the camera application's display frame rate is 30PFS before zooming, when a is 1, the camera application's display frame rate can be switched from 30PFS to 60PFS during zooming. When a is 2, the camera application's display frame rate can be switched from 30PFS to 90PFS during zooming. This increases the camera application's display frame rate, making the zooming process smoother.

[0720] Optionally, S1522 may also be executed before, after, or simultaneously with any step after S1508, and this application does not limit this.

[0721] Optionally, S1522 may also be executed by a shooting parameter generation module, which is not limited in this application.

[0722] Optionally, S1522 may also be executed by a shooting parameter generation module, which is not limited in this application.

[0723] S1523 : The shooting parameter generation module determines a second transition zoom ratio based on the zoom smoothing adjustment curve and the frame return time of the transition image frame 3 .

[0724] After receiving the return frame time of the transition image frame 3 sent by the image cropping module, the shooting parameter generation module can determine the second transition zoom ratio based on the zoom smoothing adjustment curve and the return frame time of the transition image frame 3.

[0725] Exemplarily, the second transition zoom ratio may be a transition zoom ratio of 4.

[0726] The shooting parameter generation module can determine the preset duration based on the output frame rate of the camera application and a. For example, the output frame rate is 30pfs, that is, the HAL layer returns an image frame to the application framework layer every 33.3ms. When a is 1, the preset duration can be 16.7ms.

[0727] The shooting parameter generation module may determine time t7 based on time t3 and a preset duration (eg, 16.7 ms).

[0728] Exemplarily, the return frame time of the transition image frame 3 may be time t7.

[0729] In some embodiments, the initial zoom ratio can be less than the target zoom ratio, i.e., the zoom ratio applied by the camera is increased. The shooting parameter generation module can obtain the zoom smoothing adjustment curve shown in FIG16A. For a description of how the shooting parameter generation module shown in FIG16A obtains transition zoom ratio 1, transition zoom ratio 2, and transition zoom ratio 3, please refer to the description in the embodiment of FIG13C, and this application will not repeat it here.

[0730] Exemplarily, as shown in FIG16A , the shooting parameter generation module may obtain the corresponding transition zoom ratio 4 from the zoom smoothing adjustment curve based on the frame return time (eg, time t7 ) of the transition image frame 3 .

[0731] In some embodiments, the initial zoom ratio can be greater than the target zoom ratio, i.e., the zoom ratio applied by the camera is reduced. The shooting parameter generation module can obtain the zoom smoothing adjustment curve shown in FIG16B. For a description of how the shooting parameter generation module shown in FIG16B obtains transition zoom ratio 1, transition zoom ratio 2, and transition zoom ratio 3, please refer to the description in the embodiment of FIG13D, and this application will not repeat it here.

[0732] The shooting parameter generation module can obtain the corresponding transition zoom ratio 4 from the zoom smoothing adjustment curve based on the return frame time (for example, time t7) of the transition image frame 3.

[0733] After determining the transition zoom magnification 4, the electronic device 100 may crop, render, and display the image frame corresponding to the transition zoom magnification 4 in a manner similar to S1210 - S1219 .

[0734] In some embodiments, after the electronic device 100 displays the image frame according to the transition zoom magnification 1 and the transition zoom magnification 4, the electronic device 100 may also obtain the image frame returned by the HAL layer and obtain the time when the image frame was returned, that is, time t4. The shooting parameter generation module needs to determine whether time t4 is earlier than the time of the target zoom magnification in the zoom smoothing adjustment curve, such as time t2.

[0735] When time t4 is earlier than the target zoom factor in the zoom smoothing adjustment curve, the shooting parameter generation module needs to continue to obtain other transition zoom factors and obtain image frames based on the other transition zoom factors. For example, the shooting parameter generation module can determine transition zoom factor 2 from the zoom smoothing adjustment curve based on time t4, and crop, render, and display the image frame corresponding to transition zoom factor 2 according to the method of S1210-S1219.

[0736] The shooting parameter generation module can determine the time t8 based on the time t2 and a preset time length (eg, 16.7 ms).

[0737] Exemplarily, the return frame time of the transition image frame 3 may be time t8.

[0738] In some embodiments, the initial zoom ratio may be smaller than the target zoom ratio, that is, the zoom ratio of the camera application is increased. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG16A .

[0739] Exemplarily, as shown in FIG16A , the shooting parameter generation module may obtain the corresponding transition zoom ratio 5 from the zoom smoothing adjustment curve based on the frame return time (eg, time t8 ) of the transition image frame 3 .

[0740] In some embodiments, the initial zoom ratio may be greater than the target zoom ratio, that is, the zoom ratio of the camera application is reduced. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG16B .

[0741] The shooting parameter generation module can obtain the corresponding transition zoom ratio 5 from the zoom smoothing adjustment curve based on the return frame time of the transition image frame 3 (for example, time t8).

[0742] After determining the transition zoom magnification 5, the electronic device 100 may crop, render, and display the image frame corresponding to the transition zoom magnification 5 in a manner similar to S1210-S1219.

[0743] In some embodiments, after the electronic device 100 displays the image frame according to the transition zoom magnification 2 and the transition zoom magnification 5, the electronic device 100 may also obtain the image frame returned by the HAL layer and obtain the time when the image frame was returned, that is, time t5. The shooting parameter generation module needs to determine whether time t5 is earlier than the time of the target zoom magnification in the zoom smoothing adjustment curve, such as time t2.

[0744] When time t5 is earlier than the target zoom factor in the zoom smoothing adjustment curve, the shooting parameter generation module needs to continue to obtain other transition zoom factors and obtain image frames based on the other transition zoom factors. For example, the shooting parameter generation module can determine transition zoom factor 3 from the zoom smoothing adjustment curve based on time t5, and crop, render, and display the image frame corresponding to transition zoom factor 3 according to the method of S1210-S1219.

[0745] The shooting parameter generation module can determine the time t9 based on the time t3 and a preset time length (eg, 16.7 ms).

[0746] Exemplarily, the return frame time of the transition image frame 4 may be time t9.

[0747] In some embodiments, the initial zoom ratio may be smaller than the target zoom ratio, that is, the zoom ratio of the camera application is increased. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG16A .

[0748] Exemplarily, as shown in FIG16A , the shooting parameter generation module may obtain the corresponding transition zoom ratio 6 from the zoom smoothing adjustment curve based on the frame return time (eg, time t9) of the transition image frame 4 .

[0749] In some embodiments, the initial zoom ratio may be greater than the target zoom ratio, that is, the zoom ratio of the camera application is reduced. The shooting parameter generation module may obtain a zoom smoothing adjustment curve as shown in FIG16B .

[0750] The shooting parameter generation module can obtain the corresponding transition zoom ratio 6 from the zoom smoothing adjustment curve based on the return frame time of the transition image frame 4 (for example, time t9).

[0751] After determining the transition zoom magnification 6, the electronic device 100 may crop, render, and display the image frame corresponding to the transition zoom magnification 6 in a manner similar to S1210-S1219.

[0752] This process continues until the result processing module receives an image frame returned by the HAL layer at a time later than the target zoom factor in the zoom smoothing adjustment curve, for example, time t2. At this point, the zoom operation ends, and the shooting parameter generation module no longer determines other transitional zoom factors. The application framework layer only sends the target zoom factor to the HAL layer.

[0753] In some embodiments, after the electronic device 100 displays the image frame according to the transition zoom magnification 3 and the transition zoom magnification 6, the electronic device 100 may also obtain the image frame returned by the HAL layer and obtain the time when the image frame was returned, that is, time t10. The shooting parameter generation module needs to determine whether time t10 is earlier than the time of the target zoom magnification in the zoom smoothing adjustment curve, such as time t2.

[0754] When time t10 is later than the target zoom ratio in the zoom smoothing adjustment curve, for example, time t2, the shooting parameter generation module no longer determines other transition zoom ratios, and the application framework layer only sends the target zoom ratio to the HAL layer.

[0755] Optionally, during the zoom process, the HAL layer can obtain the zoom magnification of the image frame and the image frame return time, and obtain the zoom curve shown in Figure 16A or the zoom curve shown in Figure 16B based on the zoom magnification of the image frame and the image frame return time. The zoom curve shown in Figure 16A or 16B reflects the zoom process of the image frame displayed by the electronic device 100 during the zoom process.

[0756] S1524: The shooting parameter generation module sends the second transition zoom ratio to the image cropping module.

[0757] S1525 : The image cropping module determines a cropping ratio 3 based on the zoom magnification of the image frame 1 and the second transition zoom magnification.

[0758] S1526 . The image cropping module crops the image frame 1 based on the cropping ratio 3 to obtain a transition image frame 3 .

[0759] For example, the zoom magnification of the image frame 1 may be an initial zoom magnification. The image cropping module may determine the cropping ratio 3 based on the zoom magnification of the image frame 1 and the second transition zoom magnification.

[0760] The image cropping module may crop the image frame 1 based on the cropping ratio 3 to obtain the transition image frame 3. The zoom ratio of the transition image frame 3 may be the second transition zoom ratio.

[0761] How the image cropping module crops the image frame 1 based on the cropping ratio 3 to obtain the transition image frame 3 can be referred to the description in the embodiment of FIG14A , and this application will not elaborate on it here.

[0762] S1527 . The image cropping module sends the transition image frame 3 and the address of the buffer storing the transition image frame 3 to the buffer manager.

[0763] In some embodiments, the address of the buffer storing the transition image frame 3 and the address of the buffer storing the image frame 3 may be the same or different.

[0764] S1528. The electronic device 100 renders and displays the transition image frame 3 in the manner of rendering and displaying the image frame 1 as shown in S912-S917.

[0765] In S1521, the rendering process has switched the vsync signal period from the first value to the second value and has notified the buffer manager of the switch from the first value to the second value. The rendering process's rendering speed has increased. To ensure that the camera application can also display image frames rendered by the rendering process in a timely manner, the camera application's display frame rate is also increased accordingly. For example, the camera application's display frame rate may be switched from the third value to the fourth value, where the ratio of the fourth value to the third value is a+1.

[0766] Here, a is a positive integer greater than or equal to 1, and the value of a can be a preset value, for example, a can be 1 or 2.

[0767] For example, if the value of a is 1, the fourth value and the third value may be 2, that is, the display frame rate of the camera application is doubled. For example, the fourth value may be 60PFS and the third value may be 30PFS.

[0768] For another example, if the value of a is 2, the ratio of the second value to the first value may be 3, that is, the display frame rate of the camera application is increased three times. For example, the fourth value may be 90PFS and the third value may be 30PFS.

[0769] The following examples of this application are described with a being 1.

[0770] Since the vsync signal period is shortened by (a+1) times, the display frame rate of the camera application is correspondingly increased by (a+1) times. The rendering process still obtains one image frame from the buffer manager every m vsync signal periods, renders it, and sends it to the display.

[0771] Optionally, the display processing module may also receive a timestamp sent by a timer. When the time interval between the current time and the timestamp of the previous image frame sent for rendering is greater than or equal to a preset time interval, the display processing module may obtain the image frame to be rendered from the buffer manager. For example, the display processing module may obtain transition image frame 3 to be rendered from the buffer manager.

[0772] After accumulating m vsync signal cycles, the display sending process can obtain the transition image frame 3 from the buffer manager, and then send the transition image frame 3 or the address of the buffer storing the transition image frame 3 to the camera application, and the camera application then sends the transition image frame 3 or the address of the buffer storing the transition image frame 3 to the rendering process.

[0773] After the rendering process obtains the transition image frame 3 or the address of the buffer storing the transition image frame 3 , the rendering process may render the transition image frame 3 to obtain the rendered transition image frame 3 .

[0774] In some examples, if the rendering process module obtains the address of the buffer storing the transition image frame 3, the rendering process module may render the transition image frame 3 in the buffer corresponding to the address of the buffer storing the transition image frame 3. Alternatively, if the rendering process module obtains the address of the buffer storing the transition image frame 3, the rendering process module may retrieve the transition image frame 3 from the buffer corresponding to the address of the buffer storing the transition image frame 3 and render the transition image frame 3.

[0775] The rendering process module may send the rendered transition image frame 3 or the address of the buffer storing the rendered transition image frame 3 to the camera application. The camera application then sends the transition image frame 3 or the address of the buffer storing the rendered transition image frame 3 to the display.

[0776] After the display obtains the transition image frame 3 or the address of the buffer storing the rendered transition image frame 3 , the display may display the rendered transition image frame 3 .

[0777] In some examples, if the display obtains the address of the buffer storing the rendered transition image frame 3, the rendering process module can take out the rendered transition image frame 3 from the buffer corresponding to the address of the buffer storing the rendered transition image frame 3 and display the rendered transition image frame 3.

[0778] FIG15B shows a flow chart of another shooting method provided in an embodiment of the present application.

[0779] S1601. The shooting parameter generation module notifies the rendering process to switch the vsync signal period from the second value to the first value.

[0780] In some embodiments, when the return frame moment of image frame 1 is later than the moment of the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module does not need to obtain other transition zoom ratios. The zoom ends and the image frame is collected and displayed based on the target transition zoom ratio.

[0781] The image cropping module also needs to notify the rendering process and then switch the vsync signal period from the second value to the first value, that is, switch the vsync signal period to the value before zooming.

[0782] For example, the first value may be 33.3ms, and the second value may be 16.6ms. Before the user clicks the target zoom magnification option, the vsync signal period is 33.3ms. After the user clicks the target zoom magnification option, the vsync signal period is 16.6ms. After the zoom is completed, the vsync signal period switches back to 33.3ms.

[0783] Optionally, the display frame rate of the camera application needs to be switched from the fourth value to the third value, that is, the display frame rate of the camera application needs to be switched to the value before zooming.

[0784] For example, the third value may be 30 PFS, and the fourth value may be 60 PFS. Before the user clicks the target zoom magnification option, the display frame rate of the camera application is set to 30 PFS. After the user clicks the target zoom magnification option, the display frame rate of the camera application is set to 60 PFS. After the zoom is completed, the display frame rate of the camera application is switched back to 30 PFS.

[0785] S1602-S1611 can refer to the description of S1301-S1310 in the embodiment of Figure 12C, and this application will not go into details here.

[0786] FIG. 17A is a schematic diagram showing how another electronic device 100 processes a camera request and a zoom request.

[0787] As shown in Figure 4, the application framework periodically sends camera requests to the HAL layer. Before the HAL layer responds to the zoom request corresponding to the user's zoom operation, it must process other pending camera requests. The HAL layer only processes the zoom request after all previous camera requests have been processed.

[0788] For example, as shown in Figure 16A , before the HAL layer processes the zoom request, it has six other camera requests to process: camera request 1, camera request 2, camera request 3, camera request 4, camera request 5, and camera request 6. If the HAL layer receives zoom request 1 at time tms, which carries a transition zoom factor of 1, it places zoom request 1 in the pending request queue in chronological order.

[0789] Following the first-in-first-out principle, at time tms, the HAL layer removes camera request 1 from the queue and processes it. Camera request 1 carries the initial zoom factor. In response to processing camera request 1, the camera module captures image frame A based on the initial zoom factor and sends it to the application framework layer through the HAL layer. After acquiring image frame A, the application framework layer, if it determines that the return time of image frame A is later than the target zoom factor in the zoom smoothing adjustment curve, determines transitional zoom factor 1 based on the return time of image frame A and the zoom smoothing adjustment curve. The application framework layer then determines crop ratio a1 based on the zoom factor of image frame A and transitional zoom factor 1, and crops image frame A based on crop ratio a1 to obtain transitional image frame A1, which has transitional zoom factor 1. The application framework layer then sends transitional image frame A1 to the camera application, which displays it.

[0790] Based on the above analysis, in order to reduce the change in zoom ratio between two adjacent image frames, the image cropping module can add one or more transitional zoom ratios between two adjacent transitional zoom ratios, crop the image frames based on the one or more newly added transitional zoom ratios to obtain newly added transitional image frames, and render and display the newly added transitional image frames. At the same time, the electronic device 100 needs to switch the vsync signal period from the first value to the second value and switch the display frame rate of the camera application from the third value to the fourth value.

[0791] This embodiment of the present application uses a newly added transitional zoom ratio as an example for illustration. In response to the newly added transitional zoom ratio, the vsync signal period needs to be shortened by half, and the camera application's display frame rate is doubled. For example, the vsync signal period is switched from 33.3ms to 16.7ms, and the camera application's display frame rate is switched from 30PFS to 60PFS.

[0792] As shown in Figure 17A, at time (t+16.7) ms, the application framework layer can determine time t7 based on the return time of image frame A and the preset duration, and determine a transitional zoom magnification of 4 based on time t7 and the focus smoothing adjustment curve. The application framework layer then determines a cropping ratio a2 based on the zoom magnification of image frame A and the transitional zoom magnification of 4, and crops image frame A based on the cropping ratio a2 to obtain transitional image frame A2, which has a transitional zoom magnification of 4. The application framework layer then sends transitional image frame A2 to the camera application, which can display transitional image frame A2.

[0793] As shown in Figure 17A, at time (t+33.4) ms, the HAL layer receives zoom request 2, which carries a transition zoom factor 2, and places it in a queue of pending requests in chronological order. Following the first-in-first-out principle, at time (t+33.4) ms, the HAL layer removes camera request 2 from the queue and processes it. Camera request 2 carries the initial zoom factor. In response to processing camera request 2, the camera module captures image frame B based on the initial zoom factor and sends it to the application framework layer through the HAL layer. After acquiring image frame B, the application framework layer determines transition zoom factor 2 based on the return time of image frame B and the zoom smoothing adjustment curve. The application framework layer then determines a cropping ratio b1 based on the zoom factor of image frame B and transition zoom factor 2, and crops image frame B based on cropping ratio b1 to obtain transition image frame B1, which has a transition zoom factor 2. The application framework layer then sends the transition image frame B1 to the camera application, and the camera application can display the transition image frame B1.

[0794] As shown in Figure 17A, at time (t+50.1) ms, the application framework layer can determine time t8 based on the return time of image frame B and the preset duration, and determine a transitional zoom magnification of 5 based on time t8 and the focus smoothing adjustment curve. The application framework layer then determines a cropping ratio b2 based on the zoom magnification of image frame B and the transitional zoom magnification of 5, and crops image frame B based on the cropping ratio b2 to obtain transitional image frame B2, which has a transitional zoom magnification of 5. The application framework layer then sends transitional image frame B2 to the camera application, which can display transitional image frame B2.

[0795] As shown in Figure 17A, at time (t+66.8) ms, the HAL layer receives zoom request 3, which carries a transitional zoom factor 3, and places it in a queue of pending requests in chronological order. Following the first-in-first-out principle, at time (t+66.8) ms, the HAL layer removes camera request 3 from the queue and processes it. Camera request 3 carries the initial zoom factor. In response to processing camera request 3, the camera module captures image frame C based on the initial zoom factor and sends it to the application framework layer through the HAL layer. After acquiring image frame C, the application framework layer determines the transitional zoom factor 3 based on the return time of image frame C and the zoom smoothing adjustment curve. The application framework layer then determines a cropping ratio c1 based on the zoom factor of image frame C and the transitional zoom factor 3, and crops image frame C based on the cropping ratio c1 to obtain transitional image frame C1, which has a transitional zoom factor 3. The application framework layer then sends the transition image frame C1 to the camera application, and the camera application can display the transition image frame C1.

[0796] As shown in Figure 17A, at time (t+83.5) ms, the application framework layer can determine time t9 based on the return time of image frame C and the preset duration, and determine the transition zoom magnification of 6 based on time t9 and the focus smoothing adjustment curve. The application framework layer then determines the cropping ratio c2 based on the zoom magnification of image frame B and the transition zoom magnification of 6, and crops image frame C based on the cropping ratio c2 to obtain transition image frame C2, which has a transition zoom magnification of 6. The application framework layer then sends transition image frame C2 to the camera application, which can display transition image frame C2.

[0797] As shown in Figure 17A, at (t+100.2) ms, the HAL layer receives zoom request 4. Since the return frame time of image frame D is later than the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module no longer confirms other transition zoom ratios and directly sends the target zoom ratio to the HAL layer. Therefore, zoom request 4 carries the target zoom ratio and is placed in the request queue to be processed in chronological order.

[0798] At the same time, the electronic device 100 needs to switch the vsync signal period from the second value to the first value again, and switch the display frame rate of the camera application from the fourth value to the third value again. For example, the vsync signal period is switched from 16.6ms to 33.3ms again, and the display frame rate of the camera application is switched from 60PFS to 30PFS again.

[0799] According to the first-in-first-out principle, at (t+100.2) ms, the HAL layer takes out camera request 4 from the queue and processes camera request 4, which carries the initial zoom magnification. In response to processing camera request 4, the camera module can capture image frame D based on the initial zoom magnification and send it to the application framework layer through the HAL layer. After the application framework layer obtains image frame D, if the zoom magnification of image frame D is different from the target zoom magnification, the application framework layer can determine the cropping ratio d based on the zoom magnification of image frame D and the target zoom magnification, and crop image frame D based on the cropping ratio d to obtain a transition image frame D1, which is the target zoom magnification. The application framework layer then sends the transition image frame D1 to the camera application, and the camera application can display the transition image frame D1.

[0800] As shown in Figure 17A, at time (t+133.5) ms, the HAL layer receives zoom request 5, which carries the target zoom factor, and places it in a queue of pending requests in chronological order. Following the first-in-first-out principle, at time (t+133.5) ms, the HAL layer removes camera request 5 from the queue and processes it. Camera request 5 carries the initial zoom factor. In response to processing camera request 5, the camera module captures image frame E based on the initial zoom factor and sends it to the application framework layer through the HAL layer. After acquiring image frame E, if the zoom factor of image frame E differs from the target zoom factor, the application framework layer determines a cropping ratio e based on the zoom factor and the target zoom factor, and crops image frame E based on the cropping ratio e to obtain a transitional image frame E1, which has the target zoom factor. The application framework layer then sends the transition image frame E1 to the camera application, and the camera application can display the transition image frame E1.

[0801] As shown in Figure 17A, at time (t+166.8) ms, the HAL layer receives zoom request 6, which carries the target zoom factor, and places it in a queue of pending requests in chronological order. Following the first-in-first-out principle, at time (t+166.8) ms, the HAL layer removes camera request 6 from the queue and processes it. Camera request 6 carries the initial zoom factor. In response to processing camera request 6, the camera module captures image frame F based on the initial zoom factor and sends it to the application framework layer via the HAL layer. After acquiring image frame F, if the zoom factor of image frame F differs from the target zoom factor, the application framework layer determines a cropping ratio f based on the zoom factor and the target zoom factor, and crops image frame F based on the cropping ratio f to obtain a transitional image frame F1, which has the target zoom factor. The application framework layer then sends the transition image frame F1 to the camera application, and the camera application can display the transition image frame F1.

[0802] As shown in Figure 17A, at time (t+200.1) ms, the HAL layer receives zoom request 7, which carries the target zoom factor, and places zoom request 7 in a queue of pending requests in chronological order. Following the first-in-first-out principle, at time (t+200.1) ms, the HAL layer removes camera request 7 from the queue and processes it. Camera request 7 carries the initial zoom factor. In response to processing camera request 7, the camera module captures image frame G based on transitional zoom factor 1 and sends it to the application framework layer through the HAL layer. After acquiring image frame G, if the zoom factor of image frame G differs from the target zoom factor, the application framework layer determines a cropping ratio g based on the zoom factor and the target zoom factor, and crops image frame G based on the cropping ratio g to obtain transitional image frame G1, which has the target zoom factor. The application framework layer then sends the transition image frame G1 to the camera application, and the camera application can display the transition image frame G1.

[0803] By analogy, the electronic device 100 may process zoom request 2 and zoom request 3 in the same manner as processing zoom request 1.

[0804] As shown in Figure 17A, at (t+300) ms, the HAL layer receives zoom request 10, which carries the target zoom magnification, and places zoom request 10 in the request queue to be processed in chronological order. According to the first-in-first-out principle, at (t+300) ms, the HAL layer takes out zoom request 4 from the queue and processes zoom request 4, which carries the target zoom magnification. In response to processing zoom request 4, the camera module can capture image frame H based on the target zoom magnification and send it to the application framework layer through the HAL layer. After the application framework layer obtains image frame H, if the zoom magnification of image frame H is the same as the target zoom magnification, the application framework layer does not need to crop image frame G. The application framework layer can directly send image frame G to the camera application, and the camera application can display image frame G.

[0805] By analogy, the electronic device 100 may process zoom request 5 to zoom request 10 in the same manner as that of processing zoom request 4.

[0806] As can be seen from Figure 17A, on the one hand, after the HAL layer receives zoom request 1, the application framework layer can immediately respond to the transition zoom magnification 1 carried in zoom request 1 to obtain transition image frame A1, and the zoom magnification of transition image frame A1 is transition zoom magnification 1. There is no need to wait for multiple camera requests preceding zoom request 1 to be executed before executing zoom request 1 and obtaining the image frame corresponding to transition zoom magnification 1. This shortens the time it takes the HAL layer to process zoom request 1 and increases the speed at which the camera application responds to user zoom operations. On the other hand, during the zoom process, by adding transition image frames A2, B2, and C2, the change in the transition zoom magnification between two adjacent frames can be slowed down, thereby reducing the speed of change in the zoom magnification corresponding to the two adjacent frames, making the zoom process smoother.

[0807] FIG. 17B shows a timing diagram of another electronic device 100 rendering and displaying image frames in a vsync signal cycle.

[0808] As shown in Figure 17B , between vertical synchronization signal 1 (vsync1) and vertical synchronization signal 2 (vsync2), the camera application can issue rendering instruction A1 to the rendering process. In response to rendering instruction A1, the rendering process can render transitional image frame A1 and store the rendered transitional image frame A1 in a buffer. Simultaneously, between vertical synchronization signal 1 (vsync) and vertical synchronization signal 2 (vsync2), the camera application can send image frame 0 or the image frame 0 buffer to the display, allowing the display to display image frame 0. That is, between time tms and time (t+16.7ms), the rendering process renders transitional image frame A1, and the display displays image frame 0.

[0809] Based on the above analysis, in order to reduce the change in zoom ratio between two adjacent image frames, the image cropping module can add one or more transitional zoom ratios between two adjacent transitional zoom ratios, crop the image frames based on the one or more newly added transitional zoom ratios to obtain newly added transitional image frames, and render and display the newly added transitional image frames. At the same time, the electronic device 100 needs to switch the vsync signal period from the first value to the second value and switch the display frame rate of the camera application from the third value to the fourth value.

[0810] This embodiment of the present application uses a newly added transitional zoom ratio as an example for illustration. In response to the newly added transitional zoom ratio, the vsync signal period needs to be shortened by half, and the camera application's display frame rate is doubled. For example, the vsync signal period is switched from 33.3ms to 16.7ms, and the camera application's display frame rate is switched from 30PFS to 60PFS.

[0811] For example, before the electronic device 100 responds to the user clicking the target magnification option, the display frame rate of the camera application can be 30FPS, and the vsync signal period is 33.3ms, then m is 1, that is, the rendering process renders one image frame in each vsync signal period, and the camera application sends one image frame for display in each vsync signal period.

[0812] After the electronic device 100 responds to the user clicking the target magnification option, the camera application's display frame rate can be 60 FPS, the vsync signal period is 16.7 ms, and m is still 1. That is, the rendering process still renders one image frame per vsync signal period, and the camera application displays one image frame per vsync signal period. The only difference is that the vsync signal period is shortened, and the camera application's display frame rate is increased.

[0813] Between vertical synchronization signal 2 (vsync2) and vertical synchronization signal 3 (vsync3), the camera application can send rendering instruction A to the rendering process. In response to rendering instruction A, the rendering process can render transition image frame A2 and fill the rendered transition image frame A2 into the buffer. At the same time, between vertical synchronization signal 2 (vsync2) and vertical synchronization signal 3 (vsync3), the camera application can send transition image frame A1 or the buffer of transition image frame A1 to the display, so that the display can display transition image frame A1. That is, between time (t+16.7ms)ms and time (t+33.4ms), the rendering process renders transition image frame A2, and the display displays transition image frame A1.

[0814] Between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 4 (vsync4), the camera application can send rendering instruction B1 to the rendering process. In response to rendering instruction B1, the rendering process can render transition image frame B1 and fill the rendered transition image frame B1 into the buffer. At the same time, between vertical synchronization signal 3 (vsync3) and vertical synchronization signal 4 (vsync4), the camera application can send transition image frame A2 or the buffer of transition image frame A2 to the display, so that the display can display transition image frame A2. That is, between time (t+33.4ms)ms and time (t+50.1ms), the rendering process renders transition image frame B1, and the display displays transition image frame A2.

[0815] Between vertical synchronization signal 4 (vsync4) and vertical synchronization signal 5 (vsync5), the camera application can send rendering instruction B2 to the rendering process. In response to rendering instruction B2, the rendering process can render transition image frame B2 and fill the rendered transition image frame B2 into the buffer. At the same time, between vertical synchronization signal 4 (vsync4) and vertical synchronization signal 5 (vsync5), the camera application can send transition image frame B1 or the buffer of transition image frame B1 to the display, so that the display can display transition image frame B1. That is, between time (t+50.1ms)ms and time (t+66.8ms), the rendering process renders transition image frame B2, and the display displays transition image frame B1.

[0816] Between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 6 (vsync6), the camera application can send rendering instruction C1 to the rendering process. In response to rendering instruction C1, the rendering process can render transition image frame C1 and fill the rendered transition image frame C1 into the buffer. At the same time, between vertical synchronization signal 5 (vsync5) and vertical synchronization signal 6 (vsync6), the camera application can send transition image frame B2 or the buffer of transition image frame B2 to the display, so that the display can display transition image frame B2. That is, between time (t+66.8ms)ms and time (t+83.5ms), the rendering process renders transition image frame C1, and the display displays transition image frame B2.

[0817] Between vertical synchronization signal 6 (vsync6) and vertical synchronization signal 7 (vsync7), the camera application can send rendering instruction C2 to the rendering process. In response to rendering instruction C2, the rendering process can render transition image frame C2 and fill the rendered transition image frame C2 into the buffer. At the same time, between vertical synchronization signal 6 (vsync6) and vertical synchronization signal 7 (vsync7), the camera application can send transition image frame C1 or the buffer of transition image frame C1 to the display, so that the display can display transition image frame C1. That is, between time (t+83.5ms)ms and time (t+100.2ms), the rendering process renders transition image frame C2, and the display displays transition image frame C1.

[0818] Based on the above analysis, at (t+100.2) ms, the HAL layer receives the zoom request 4. Since the return frame time of the image frame D is later than the target zoom ratio in the zoom smoothing adjustment curve, the shooting parameter generation module no longer confirms other transition zoom ratios and directly sends the target zoom ratio to the HAL layer. Before receiving the vertical synchronization signal 7 (vsync7), the electronic device 100 needs to switch the vsync signal period from the second value to the first value again, and switch the display frame rate of the camera application from the fourth value to the third value again. For example, the vsync signal period is switched from 16.6ms to 33.3ms again, and the display frame rate of the camera application is switched from 60PFS to 30PFS again.

[0819] Then, between vertical synchronization signal 7 (vsync7) and vertical synchronization signal 8 (vsync8), the camera application can send rendering instruction D to the rendering process. In response to rendering instruction D, the rendering process can render transition image frame D1 and fill the rendered transition image frame D1 into the buffer. At the same time, between vertical synchronization signal 7 (vsync7) and vertical synchronization signal 8 (vsync8), the camera application can send transition image frame C2 or the buffer of transition image frame C2 to the display, so that the display can display transition image frame C2. That is, between time (t+100.2)ms and time (t+133.5)ms, the rendering process renders transition image frame D1, and the display displays transition image frame C2.

[0820] Between vertical synchronization signal 8 (vsync8) and vertical synchronization signal 9 (vsync9), the camera application can send rendering instruction E to the rendering process. In response to rendering instruction E, the rendering process can render transition image frame E1 and fill the rendered transition image frame E1 into the buffer. At the same time, between vertical synchronization signal 7 (vsync7) and vertical synchronization signal 8 (vsync8), the camera application can send transition image frame D1 or the buffer of transition image frame D1 to the display, so that the display can display transition image frame D1. That is, between time (t+133.5)ms and time (t+166.8)ms, the rendering process renders transition image frame E1, and the display displays transition image frame D1.

[0821] Between vertical synchronization signal 9 (vsync9) and vertical synchronization signal 10 (vsync10), the camera application can send a rendering instruction F to the rendering process. In response to rendering instruction F, the rendering process can render transition image frame F1 and fill the rendered transition image frame F1 into the buffer. At the same time, between vertical synchronization signal 9 (vsync9) and vertical synchronization signal 10 (vsync10), the camera application can send transition image frame E1 or the buffer of transition image frame E1 to the display, so that the display can display transition image frame E1. That is, between time (t+166.8)ms and time (t+200.1)ms, the rendering process renders transition image frame F1, and the display displays transition image frame E1.

[0822] Between vertical synchronization signal 10 (vsync10) and vertical synchronization signal 11 (vsync11), the camera application can send a rendering instruction G to the rendering process. In response to rendering instruction G, the rendering process can render transition image frame G1 and fill the rendered transition image frame G1 into the buffer. At the same time, between vertical synchronization signal 10 (vsync10) and vertical synchronization signal 11 (vsync11), the camera application can send transition image frame F1 or the buffer of transition image frame F1 to the display, so that the display can display transition image frame F1. That is, between time (t+200.1)ms and time (t+233.4)ms, the rendering process renders transition image frame G1, and the display displays transition image frame F1.

[0823] Similarly, between vertical synchronization signal 13 (vsync13) and vertical synchronization signal 14 (vsync14), the camera application can send rendering instruction H to the rendering process. In response to rendering instruction H, the rendering process can render transition image frame H1 and fill the rendered transition image frame H1 into the buffer. At the same time, between vertical synchronization signal 13 (vsync13) and vertical synchronization signal 14 (vsync14), the camera application can send transition image frame I1 or the buffer of transition image frame I1 to the display, so that the display can display transition image frame I1. That is, between time (t+300)ms and time (t+333.3)ms, the rendering process renders transition image frame H1, and the display displays transition image frame I1.

[0824] Similarly, the camera application can render and display image frames based on the vsync signal cycle in the manner shown in FIG17B .

[0825] As can be seen from Figure 17B, on the one hand, the rendering process can send the image frames cropped by the application framework layer or the image frames sent by the application framework layer to the rendering process for rendering and display according to a fixed vsync signal period, which can ensure that the image frames are evenly sent to the rendering process for rendering and display, ensuring that the camera application displays the image frames evenly. On the other hand, during the zoom process, the vsync signal period and the disp...

Claims

1. A method for generating shooting parameters, characterized in that: The electronic device includes a shooting parameter generation module, an image cropping module, and a rendering process, and the method includes: The electronic device receives and responds to a first operation of a user on a camera application, and acquires a target shooting parameter, wherein the target shooting parameter is a shooting parameter set by the first operation; The shooting parameter generation module determines a first transition shooting parameter according to a return frame time of a first image frame and the target shooting parameter, wherein the first image frame is an image frame acquired after the electronic device receives the first operation; The image cropping module processes the first image frame according to the first transition shooting parameter and the shooting parameters of the first image frame to obtain a first transition image frame, wherein the shooting parameters of the first transition image frame include the first transition shooting parameter; The rendering process renders the first transition image according to a vertical synchronization signal period of the rendering process; The electronic device displays the rendered first transition image within the camera application.

2. The method according to claim 1, characterized in that The shooting parameter generation module determines the first transition shooting parameter according to the frame return time of the first image frame and the target shooting parameter, specifically including: The shooting parameter generation module determines the first transition shooting parameter according to the initial shooting parameter, the return frame time of the first image frame and the target shooting parameter, wherein the initial shooting parameter is the shooting parameter set before the electronic device receives the first operation.

3. The method according to claim 2, characterized in that The shooting parameter generation module determines the first transition shooting parameter according to the initial shooting parameter, the frame return time of the first image frame and the target shooting parameter, specifically including: The shooting parameter generation module acquires a smooth adjustment curve based on the initial shooting parameter and the target shooting parameter, wherein the starting point of the smooth adjustment curve is the initial shooting parameter and the end point of the smooth adjustment curve is the target shooting parameter; The shooting parameter generation module determines the first transition shooting parameter from the smooth adjustment curve based on the frame return time of the first image frame.

4. The method according to claim 3, characterized in that The rendering process renders the first transition image according to the vertical synchronization signal period of the rendering process, specifically including: When the time interval between the first time and the display time of the previous image frame is greater than or equal to m first vertical synchronization signal cycles, the rendering process renders the first transition image at the first time.

5. The method according to claim 4, characterized in that After the shooting parameter generation module determines the first transition shooting parameter according to the frame return time of the first image frame and the target shooting parameter, the method further includes: The shooting parameter generation module determines the second transition shooting parameter according to the return frame time of the first image frame, the preset duration and the target shooting parameter: The image cropping module processes the first image frame according to the second transition shooting parameter and the shooting parameter of the first image frame to obtain a second transition image frame, wherein the shooting parameter of the second transition image frame includes the second transition shooting parameter; The rendering process renders the second transition image according to a vertical synchronization signal period of the rendering process; The electronic device displays the rendered second transition image within the camera application.

6. The method according to claim 5, characterized in that The rendering process renders the second transition image according to the vertical synchronization signal period of the rendering process, specifically including: When the time interval between the second time and the display time of the previous image frame is greater than or equal to m second vertical synchronization signal cycles, the rendering process renders the second transition image at the second time.

7. The method according to claim 6, characterized in that The second vertical synchronization signal period is shorter than the first vertical synchronization signal period.

8. The method according to claim 6 or 7, characterized in that: After the shooting parameter generation module determines the second transition shooting parameter according to the return frame time of the first image frame, the preset duration and the target shooting parameter, the method further includes: The shooting parameter generation module obtains a return frame time of a second image frame, where the second image frame is an image frame acquired after the electronic device receives the first operation; The shooting parameter generation module determines whether the frame return time of the second image frame is later than the time corresponding to the target shooting parameter in the smooth adjustment curve; When the frame return time of the second image frame is later than the time corresponding to the target shooting parameter in the smooth adjustment curve, the electronic device switches the vertical synchronization signal period from the second vertical synchronization signal period to the first vertical synchronization signal period.

9. The method according to any one of claims 6 to 8, characterized in that: After the electronic device receives and responds to a first operation of the user on the camera application, the method further includes: The electronic device switches the vertical synchronization signal period from the first vertical synchronization signal period to the second vertical synchronization signal period.

10. The method according to any one of claims 2 to 9, characterized in that: The electronic device further includes a camera module. Before the electronic device receives and responds to a first operation of a user on a camera application, the method further includes: The electronic device receives and obtains the initial shooting parameters in response to a second operation of starting a camera application; The camera module captures image frames based on the initial shooting parameters.

11. The method according to claim 3, characterized in that The shooting parameter generation module determines the first transition shooting parameter from the smooth adjustment curve based on the frame return time of the first image frame, specifically including: The shooting parameter generation module determines whether the frame return time of the first image frame is earlier than the time corresponding to the target shooting parameter in the smooth adjustment curve; In a case where the frame return time of the first image frame is earlier than the time corresponding to the target shooting parameter, the shooting parameter generation module determines the first transition shooting parameter from the smooth adjustment curve based on the frame return time of the first image frame.

12. The method according to claim 1, characterized in that The electronic device further includes a camera module, and the method further includes: When the returning frame time of the first image frame is earlier than the time corresponding to the target shooting parameter, the camera module captures image frames based on the first transition shooting parameter.

13. The method according to claim 11 or 12, characterized in that: The method further comprises: When the returning frame time of the first image frame is later than the time corresponding to the target shooting parameter, the camera module collects image frames based on the target shooting parameter.

14. The method according to any one of claims 2 to 13, characterized in that: When the initial shooting parameter is greater than the target shooting parameter, the first transition shooting parameter is less than the initial shooting parameter and greater than the target shooting parameter; When the initial shooting parameter is smaller than the target shooting parameter, the first transition shooting parameter is larger than the initial shooting parameter and smaller than the target shooting parameter.

15. The method according to any one of claims 1 to 14, characterized in that: The electronic device comprises an application framework layer and a hardware abstraction layer; The first image frame is an image frame sent by the hardware abstraction layer to the application framework layer after the electronic device receives the first operation, and the return frame time of the first image frame is the time when the hardware abstraction layer sends the first image frame to the application framework layer.

16. The method according to claim 15, characterized in that The photography parameter generation module, the image cropping module and the rendering process are functional modules in the application framework layer.

17. The method according to claim 12, characterized in that The electronic device includes an application framework layer and a hardware abstraction layer; the camera module collects image frames based on the first transition shooting parameter, specifically including: In response to the first image frame returning time being earlier than the time corresponding to the target shooting parameter, the application framework layer sends a first request to the HAL layer, where the first request carries the first transition shooting parameter; The HAL layer sends the first request to the camera module; The first transition shooting parameter acquires the second image frame, specifically including: In response to the first request, the camera module captures the second image frame based on the first transition shooting parameter.

18. The method according to claim 13, characterized in that In the case where the return frame time of the first image frame is later than the time corresponding to the target shooting parameter, the method further includes: The image cropping module determines whether the shooting parameters of the first image frame are the same as the target shooting parameters; In a case where the shooting parameters of the first image frame are different from the target shooting parameters, the image cropping module processes the first image frame according to the shooting parameters of the first image frame and the target shooting parameters to obtain a third transition image frame; The rendering process renders the third transition image according to a vertical synchronization signal period of the rendering process; The electronic device displays the rendered third transition image within the camera application.

19. The method according to claim 18, characterized in that The method further comprises: In a case where the shooting parameters of the first image frame are the same as the target shooting parameters, the rendering process renders the first image frame according to a vertical synchronization signal period of the rendering process; The electronic device displays the rendered first image frame within the camera application.

20. The method according to any one of claims 1 to 19, characterized in that: The image cropping module processes the first image frame according to the first transition shooting parameter and the shooting parameter of the first image frame to obtain a first transition image frame, specifically including: The image cropping module changes the field of view FOV in the first image frame to obtain the first transition image frame.

21. The method according to claim 20, characterized in that The image cropping module changes the field of view FOV in the first image frame to obtain the first transition image frame, specifically including: The image cropping module multiplies the field of view FOV of the first image frame by a first proportionality coefficient to obtain the first transition image frame, wherein the first proportionality coefficient is a ratio of the first transition shooting parameter to the shooting parameter of the first image frame.

22. The method according to claim 21, characterized in that The FOV of the first image frame is a first FOV, the FOV of the first transition image frame is a second FOV, and the second FOV is a product of the first FOV and the first proportional coefficient.

23. The method according to any one of claims 20-21, characterized in that: The size of the first image frame is a first size, and the image cropping module changes the field of view FOV in the first image frame to obtain the first transition image frame, specifically including: The image cropping module crops the first image frame from a first size to a second size to obtain a first image; the second size is the product of the first size and a second proportional coefficient, and the second proportional coefficient is the reciprocal of the first proportional coefficient; The image cropping module changes the first image at the second size from the first FOV to the second FOV; The image cropping module scales the first image from the second size to the first size to obtain the first transition image frame.

24. The method according to any one of claims 2 to 10, characterized in that: The initial shooting parameters include a first zoom ratio, and the target shooting parameters include a second zoom ratio.

25. The method according to claim 24, characterized in that Before the electronic device receives and responds to a first operation of the user on the camera application, the method further includes: The camera application displays a first preview interface, the first preview interface includes a zoom ratio control, and the zoom ratio control indicates that the zoom ratio value of the image frame displayed in the first preview interface is the first zoom ratio; After the electronic device receives and responds to a first operation of the user on the camera application, the method further includes: The camera application displays a second preview interface, the second preview interface includes the zoom ratio control, and the zoom ratio control indicates that the zoom ratio value of the image frame displayed in the second preview interface is the second zoom ratio.

26. The method according to any one of claims 4 to 9, characterized in that: The method further comprises: When the time interval between the first time and the display time of the previous image frame is less than the m first vertical synchronization signal cycles, the electronic device stores the first image frame in a cache buffer queue of image frames to be displayed.

27. The method according to claim 26, characterized in that The electronic device further includes a display transmission thread and a buffer manager, the buffer manager is used to temporarily store the image frame, and after the image cropping module processes the first image frame according to the first transition shooting parameter and the shooting parameter of the first image frame to obtain the first transition image frame, the method further includes: When the number of image frames in the buffer manager is greater than or equal to a first threshold, the display sending thread sends the first transition image to the rendering process for rendering; When the image frame in the buffer manager is smaller than the first threshold, the buffer manager temporarily stores the first transition image frame.

28. The method according to claim 27, characterized in that After the buffer manager temporarily stores the first transition image frame, the method further includes: The display sending thread obtains the first transition image frame from the buffer manager.

29. The method according to any one of claims 4 to 9 or claims 26 to 28, characterized in that: The electronic device also includes a vsync signal monitoring module, and the vsync signal monitoring module is used to monitor the vsync signal of the rendering process module.

30. The method according to claim 29, characterized in that The method further comprises: The vsync signal monitoring module sends the vsync signal to the display sending process module; When the time interval between the first time and the display time of the previous image frame is greater than or equal to m first vertical synchronization signal cycles, the rendering process renders the first transition image at the first time, specifically including: When the display process module determines based on the vsync signal that the time interval between the first time and the display time of the previous image frame is greater than or equal to the m vertical synchronization vsync signal cycles, the rendering process renders the first transition image at the first time.

31. The method according to any one of claims 4 to 9 or claims 26 to 30, characterized in that: The value of m is determined by the frame rate of the electronic device and the vsync signal cycle interval.

32. The method according to any one of claims 4 to 9 or claims 26 to 31, characterized in that: The value of m is equal to the first value divided by the frame rate, and then divided by the vsync signal period interval.

33. The method according to any one of claims 1 to 32, characterized in that The electronic device further includes a timer module, and the method further includes: The timer sets a first timestamp for the first transition image; When the display thread module determines that the time interval between the first timestamp and the timestamp of the previous image frame sent by the display process module to the rendering process is greater than a second threshold, the rendering process renders the first transition image according to the vertical synchronization signal period of the rendering process.

34. The method according to any one of claims 1 to 33, characterized in that The shooting parameters include any of the following: zoom ratio, exposure, aperture, white balance, sharpness, contrast, and saturation.

35. An electronic device, characterized in that: It includes a camera, one or more processors and one or more memories; wherein the camera, the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the method described in any one of claims 1-34 is executed.

36. A chip system, the chip system is applied to electronic equipment, the chip system comprises one or more processors, characterized in that: The processor is configured to call computer instructions so as to execute the method according to any one of claims 1 to 34.

37. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the method as claimed in any one of claims 1 to 34 is executed.

Images