Image processing method, readable storage medium, program product and electronic device

Abstract

An image processing method, a readable storage medium, a program product and an electronic device (10), which relate to the technical field of terminals. The image processing method comprises: during the process of generating a first animation on the basis of a first animation curve (L1), regardless of whether the drawing and rendering duration of one image frame of the first animation exceeds one Vsync period (t0), an electronic device (10) determining an animation moment of an image frame following the current image frame to be the sum of an animation moment of the current image frame and the Vsync period (t0); on the basis of the determined animation moment of the following image frame and the first animation curve (L1), determining the position of an animation element in the following image frame; and then, on the basis of the determined position, the electronic device (10) drawing, rendering and synthesizing the following image frame. Thus, it can be ensured that animation moments corresponding to image frames of a first animation form an arithmetic sequence, such that when an electronic device (10) displays the first animation according to a fixed refresh rate, the displayed first animation is consistent with a first animation curve (L1), thereby helping to improve the user experience.

Description

Image processing method, readable storage medium, program product and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202410902976.1, filed on July 5, 2024, and entitled "Image processing method, readable storage medium, program product and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminal, and in particular, to an image processing method, a readable storage medium, a program product and an electronic device. BACKGROUND

[0003] To improve user experience, electronic devices usually set animations on graphical user interfaces (GUIs). The animations displayed on the user interfaces are usually composed of multiple consecutive images. The electronic devices can draw, render, compose and display the images of the animations frame by frame according to vertical synchronization (Vsync) signals. The period of the Vsync signals (Vsync period) is associated with the refresh frequency of the display screen, for example, the Vsync period is the inverse of the refresh frequency of the display screen.

[0004] For an image, the process from the beginning of drawing to the display usually includes three processing stages: drawing / rendering, composing and displaying. Generally, the electronic devices can implement the three processing stages of an image through three consecutive Vsync periods, for example, the electronic devices draw and render the content of the image in the first Vsync period, compose the content of the image in the second Vsync period, and display the composed image in the third Vsync period. However, if a certain image is relatively complex, or the hardware resources of the electronic devices are insufficient when drawing and rendering the image, the image may not be able to be drawn and rendered within a Vsync period, which may cause the electronic devices to fail to complete the drawing / rendering and composition of the image before the Vsync period of displaying the image arrives. In this way, the electronic devices can only continue to display the previous image of the image, causing the animation effect to be not smooth and affecting the user experience. SUMMARY

[0005] In view of this, the present application provides an image processing method, a readable storage medium, a program product and an electronic device, which are beneficial to alleviate or avoid the GUI lag displayed by the electronic devices.

[0006] In a first aspect, an embodiment of the present application provides an image processing method applied to an electronic device, the method comprising: displaying a first interface, the first interface comprising a first display element; detecting a first operation of a user on the first interface; in response to the first operation, generating and displaying a first animation of movement of the first display element based on an animation curve, wherein the first animation comprises a plurality of consecutive images, positions of the first display element in adjacent two images of the plurality of consecutive images are equal in interval at corresponding time points on the animation curve, and the plurality of consecutive images comprises a first image, a drawing and rendering time length of the first image is greater than a first time length, the first time length being an inverse of a refresh frequency of a display screen of the electronic device.

[0007] In the method, even if a drawing and rendering time length in the first animation is greater than the first time length (for example, a Vsync period below), the electronic device generates each image of the first animation using animation time points with the same time interval (that is, positions of the first display element in each image of the first animation are equal in interval at corresponding time points on the animation curve). In this way, it can be ensured that positions of the first display element (for example, an animation element below) in each image of the first animation displayed by the electronic device change over time according to the animation curve, thereby improving user experience.

[0008] In a possible implementation of the first aspect, the first operation comprises: an operation of pressing by the user at a first position in the first interface and then lifting after sliding to a second position, wherein: a sliding speed of the user at the second position is greater than a first speed, or an acceleration of the user at the second position is greater than a first acceleration.

[0009] In the method, the first operation can be a throw-and-slide operation below.

[0010] Alternatively, the first operation can also be a mid-air gesture triggering a throw-and-slide animation below.

[0011] Alternatively, the first animation can be a throw-and-slide animation below.

[0012] In a possible implementation of the first aspect, the generating and displaying of the first animation of movement of the first display element based on the animation curve comprises: detecting a first vertical synchronization signal, drawing and rendering a second image, wherein the second image is any image in the plurality of consecutive images except a last image; in a case where an in-between frame condition is met after the drawing and rendering of the second image is completed, drawing and rendering a third image before detecting a second vertical synchronization signal, wherein the second vertical synchronization signal is a first vertical synchronization signal after the drawing and rendering of the second image is completed, and the third image is a next image of the second image in the plurality of consecutive images; in a case where the in-between frame condition is not met after the drawing and rendering of the second image is completed, drawing and rendering the third image after detecting the second vertical synchronization signal.

[0013] In a possible implementation of the first aspect, the inserting condition comprises any one of the following conditions: a time length between a time point at which the drawing and rendering of the second image is completed and a time point at which the first operation is detected is less than a second time length; and a number of free buffers in a buffer queue used by an application program corresponding to the first display element to draw and render images is greater than a buffer threshold.

[0014] In this implementation, the electronic device triggers the inserting in a case where a time length from the first operation is triggered is less than the second time length. That is, within the second time length before the first animation starts, the number of free buffers in the buffer queue does not need to be concerned. Since the first animation may be stuck for a period of time before it starts, triggering the inserting within the second time length can draw and render the image frames of the first animation in advance, which is beneficial to alleviate or avoid the sticking.

[0015] In this implementation, the inserting is triggered when the number of free buffers in the buffer queue used by the application program corresponding to the first display element to draw and render images is greater than the buffer threshold, and the inserting is not triggered when the number of free buffers in the buffer queue is less than or equal to the buffer threshold. In this way, the sticking caused by the buffer queue being occupied by the image frames inserted for a long time can be avoided.

[0016] In a possible implementation of the first aspect, the drawing and rendering of the second image comprises: obtaining a first position in the animation curve corresponding to the first time point, and drawing and rendering the first display element in the second image based on the first position, wherein: in a case where the second image is a first frame image in a plurality of continuous image frames, the first time point is any time point in the animation curve, and in a case where the second image is not the first frame image in the plurality of continuous image frames, the first time point is a second time point corresponding to a position of the first display element in a previous frame image of the first frame image in the animation curve and the first time length.

[0017] In a possible implementation of the first aspect, the drawing and rendering of the third image comprises: obtaining a second position in the animation curve corresponding to the third time point, and drawing and rendering the first display element in the third image based on the second position, wherein the third time point is a sum of the first time point and the first time length.

[0018] In a possible implementation of the first aspect, the method further comprises: adjusting the buffer threshold based on a historical state of the buffers in the buffer queue.

[0019] In a possible implementation of the first aspect, the adjusting the buffer threshold based on the historical state of the buffers in the buffer queue comprises: decreasing the buffer threshold when the buffer queue is not occupied for a first number of vertical synchronization signal periods, or a ratio of a number of vertical synchronization signal periods in which the buffer queue is occupied to the first number is less than a first ratio, or the buffer queue is not occupied for a third time length, or a ratio of a time length in which the buffer queue is occupied to the third time length is less than a second ratio; increasing the buffer threshold when the buffer queue is occupied for a second number of vertical synchronization signal periods, or a ratio of a number of vertical synchronization signal periods in which the buffer queue is occupied to the second number is greater than a third ratio, or the buffer queue is occupied for a fourth time length, or a ratio of a time length in which the buffer queue is occupied to the fourth time length is greater than a fourth ratio.

[0020] In this implementation, the electronic device can dynamically adjust the number of buffer thresholds based on the historical state of the buffers. For example, the buffer threshold is decreased when the buffer queue is not occupied for a long time to make full use of the buffer queue, or the buffer threshold is increased when the buffer queue is occupied for a long time to avoid stuttering due to the buffer being occupied.

[0021] In a second aspect, the present application provides an electronic device, comprising: a memory, configured to store one or more programs; and a processor, configured to execute the one or more programs to enable the electronic device to implement the image processing method in the first aspect and any possible implementation of the first aspect.

[0022] In a third aspect, a computer-readable storage medium is provided, the computer-readable storage medium includes one or more programs, and when the one or more programs are executed on an electronic device, the electronic device is enabled to implement the image processing method in the first aspect and any possible implementation of the first aspect.

[0023] In a fourth aspect, a computer program product is provided, and when the computer program product is run on an electronic device, the electronic device is enabled to implement the image processing method in the first aspect and any possible implementation of the first aspect.

[0024] It should be understood that the beneficial effects of the second aspect to the fourth aspect described above can refer to the beneficial effects described in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0025] FIG. 1 shows a process diagram of a mobile phone 10 processing and displaying image frames according to some embodiments of the present application.

[0026] Figure 2 illustrates a process of processing and displaying the image frames in Figure 1 according to some embodiments of the application.

[0027] Figure 3A illustrates an animation and an animation curve according to some embodiments of the application.

[0028] Figure 3B illustrates a process of processing the image frames in a first animation according to some embodiments of the application.

[0029] Figure 3C illustrates a correspondence between the positions of the animation elements in some of the image frames in a first animation and an animation curve according to some embodiments of the application.

[0030] Figure 3D illustrates another process of processing the image frames in a first animation according to some embodiments of the application.

[0031] Figure 3E illustrates a change in the buffer threshold and the number of free buffers according to some embodiments of the application.

[0032] Figure 4 illustrates a change in the state of the buffers in a process of processing the image frames in a first animation according to some embodiments of the application.

[0033] Figure 5 illustrates another change in the state of the buffers in a process of processing the image frames in a first animation according to some embodiments of the application.

[0034] Figure 6 illustrates a software architecture of a mobile phone 10 according to some embodiments of the application.

[0035] Figure 7 illustrates a flowchart of an image processing method according to some embodiments of the application.

[0036] Figure 8 illustrates a user interface U6 according to some embodiments of the application.

[0037] Figure 9 illustrates a flowchart of another image processing method according to some embodiments of the application.

[0038] Figure 10 illustrates a flowchart of yet another image processing method according to some embodiments of the application.

[0039] Figure 11 illustrates a flowchart of still another image processing method according to some embodiments of the application.

[0040] Figure 12 illustrates a structure of a mobile phone 10 according to some embodiments of the application. DETAILED DESCRIPTION

[0041] Illustrative embodiments of the present application include, but are not limited to, an image processing method, a readable storage medium, and an electronic device.

[0042] It should be noted that the image processing method provided by the embodiments of the present application can be applied to any electronic device, including but not limited to user equipment (UE), mobile station (MS), mobile terminal (MT), etc. For example, the electronic device can be a mobile phone, a wearable device, a tablet computer (Pad), a laptop computer, a virtual reality (VR) device, an augmented reality (AR) device, an industrial control device, a car machine, a terminal device in a smart grid, a terminal device in transportation safety, a terminal device in a smart city, etc. For ease of description, the electronic device is taken as a mobile phone 10 in the following description of the technical solutions of the present application.

[0043] The technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings.

[0044] As described above, if a certain frame of image is relatively complex, or the hardware resources of the electronic device are insufficient when drawing the frame of image, the frame of image can not be drawn and rendered within a Vsync period, which can cause the electronic device to fail to complete the drawing / rendering and composition of the frame of image before the Vsync period of displaying the frame of image arrives. In this way, the electronic device can only continue to display the previous frame of image of the frame of image, resulting in unsmooth animation effect and affecting user experience.

[0045] Exemplarily, FIG. 1 shows a process diagram of a mobile phone 10 processing and displaying image frames according to some embodiments of the present application.

[0046] Suppose that the image frames to be displayed by the mobile phone 10 are frame F1, frame F2, frame F3, frame F4, frame F5, frame F6, frame F7, …, in sequence. Referring to FIG. 1:

[0047] In Vsync period P1, the mobile phone 10 draws and renders frame F1.

[0048] In Vsync period P2, the mobile phone 10 draws and renders frame F2, and composes frame F1.

[0049] In Vsync period P3, the mobile phone 10 draws and renders frame F3, composes frame F2, and displays frame F1.

[0050] In Vsync period P4, the mobile phone 10 draws and renders frame F4, composites frame F3, and displays frame F2.

[0051] Since frame F4 is not drawn and rendered completely in Vsync period P4, in Vsync period P5, the mobile phone 10 continues to draw and render frame F4 and displays frame F3.

[0052] Since frame F4 is not composited in Vsync period P5, in Vsync period P6, the mobile phone 10 draws and renders frame F5, composites frame F4, and displays frame F3.

[0053] In Vsync period P7, the mobile phone 10 draws and renders frame F6, composites frame F5, and displays frame F4.

[0054] In Vsync period P8, the mobile phone 10 draws and renders frame F7, composites frame F6, and displays frame F5.

[0055] As can be seen from the above process, in Vsync period P5 and Vsync period P6, the mobile phone 10 displays frame F3, causing stuttering.

[0056] In some embodiments, to avoid stuttering caused by the mobile phone 10 displaying the same screen in two adjacent Vsync periods due to the time length of drawing / rendering a frame image exceeding a Vsync period, the mobile phone 10 can draw and render each frame image in advance in a manner of drawing and rendering multiple frame images in one Vsync period (for example, interpolation), so that each frame image is completed in drawing and rendering before the corresponding Vsync period of the composite image arrives. Interpolation refers to: in one Vsync period, after drawing and rendering the image data of a frame image, the image data of the next frame image is continued to be drawn and rendered. In this way, stuttering of the screen displayed by the mobile phone 10 due to the time length of drawing / rendering a frame image exceeding a Vsync period can be avoided.

[0057] For example, referring to FIG. 2, the process of drawing, rendering, compositing, and displaying the image frames in the scenario shown in FIG. 1 can include:

[0058] In Vsync period P1, the mobile phone 10 draws and renders frame F1. After drawing frame F1, the mobile phone 10 can continue to draw and render frame F2.

[0059] In Vsync period P2, the mobile phone 10 composites frame F1, and draws and renders frame F3 and frame F4.

[0060] In Vsync period P3, the mobile phone 10 composites frame F2 and displays frame F1, and the mobile phone 10 continues to draw frame F4.

[0061] In Vsync period P4, the mobile phone 10 draws and renders frame F5, frame F6 and frame F7, composites frame F3, and displays frame F2.

[0062] In Vsync period P5, the mobile phone 10 draws and renders frame F8, frame F9 and frame F10, composites frame F4, and displays frame F3.

[0063] In Vsync period P6, the mobile phone 10 draws and renders frame F11, composites frame F5, and displays frame F4.

[0064] In Vsync period P7, the mobile phone 10 displays frame F5 and composites frame F6 (at the same time, frame F11 or a subsequent frame of frame F11 is also drawn and rendered, not shown).

[0065] In Vsync period P8, the mobile phone 10 displays frame F6 and composites frame F6 (at the same time, frame F11 or a subsequent frame of frame F11 is also drawn and rendered, not shown).

[0066] As can be seen from the above process, although the time consumed by the drawing and rendering of frame F4 is greater than one Vsync period (hereinafter, an image frame whose time consumed by the drawing and rendering is greater than one Vsync period is referred to as an overtime frame), since frame F4 is started to be drawn and rendered in Vsync period P2, when the Vsync period (Vsync period P5) for compositing frame F4 arrives, frame F4 has been completed to be drawn and rendered, so that the mobile phone 10 can composite frame F4 in Vsync period P5 and display frame F4 in Vsync period P6, avoiding the screen freezing of the mobile phone 10 caused by displaying the same frame in two (or more) adjacent Vsync periods.

[0067] In some embodiments, the position of an animation element (a display element moving in the animation) in the animation displayed in the mobile phone 10 changes over time according to a preset animation curve (a curve of the position of the animation element changing over time, such as a Bezier curve, a quadratic curve, a cubic curve, etc.), so as to improve the visual experience of the user. However, in the process of interpolating the image frames of the animation, if a certain frame image is an overtime frame, the position of the animation element in the animation displayed in the mobile phone 10 changes over time according to a rule inconsistent with the animation curve, so that the user feels visually unsmooth, affecting the user experience.

[0068] Specifically:

[0069] In the process of generating a frame of the first animation based on the first animation curve, the mobile phone 10 can first determine the animation time of the frame (i.e., the time corresponding to the position of the animated element in the frame on the first animation curve); then, based on the animation time of the frame and the first animation curve, determine the position of the animated element in the frame (for example, substituting the animation time of the frame into the animation curve to obtain the position of the animated element); then, the frame can be drawn, rendered, composited, and displayed according to the position of the animated element. Specifically, if a frame is not a timeout frame, the animation time of the next frame can be the sum of the animation time of the current frame and the Vsync period; if a frame is a timeout frame, the mobile phone 10 can adjust the animation time of the next frame to be greater than or less than the sum of the animation time of the timeout frame and the Vsync period.

[0070] It should be noted that the animation time of the first frame in the first animation can be any time T on the first animation curve.

[0071] It should be noted that the animation moment refers to the time parameter used to determine the position of the animated elements in a frame of the first animation when the first animation is generated. For example, if the first animation curve is represented as a function y = f(t) of position y changing over time, the position of the animated element in a frame is based on y. d =f(t) d It is certain that, in t d This can be the animation moment corresponding to the frame image.

[0072] It should be noted that the position of the animated element in each frame of the first animation can be the same as the corresponding position of the animation moment in the animation curve, or it can have the same offset as the corresponding position of the animation moment in the animation curve, or it can have a linear relationship (or other certain functional relationship) with the corresponding position of the animation moment in the animation curve. No limitation is imposed here. For example, suppose the animation moments of the n (n is an integer greater than 1) frames in the first animation are from t1 to t... n Animation time t1 to t n The corresponding positions on the first curve are y1 to y2. n Then the positions of the animated elements in the n frames of the first animation are y′1 to y′1 respectively. n Then y1 to y n and y′1 to y′ n The relationship between them can be represented as: y′ k = a×y1+b, where a is a real number not equal to 0, and b is any real number.

[0073] Based on the above process, if each frame image in the first animation is not an overtime frame, the animation time of each frame image in the first animation is an arithmetic progression with a tolerance of a Vsync period; if there is an overtime frame in the first animation, the animation time of each frame image in the first animation is not an arithmetic progression. Since the refresh frequency of the display screen in the mobile phone 10 is usually unchanged during the display of the first animation (equivalent to the duration of each frame image of the first animation displayed by the mobile phone 10 being the same, and the time points at which the mobile phone 10 updates the frame images of the first animation being an arithmetic progression), if the animation time of each frame image in the first animation is not an arithmetic progression, it will cause the change rule of the position of the animation element in the first animation displayed by the mobile phone 10 to be mismatched with the first animation curve, affecting the user experience.

[0074] Exemplarily, FIG. 3A shows a schematic diagram of an animation and an animation curve according to some embodiments of the present application.

[0075] As shown in FIG. 3A, the mobile phone 10 displays a contact interface U1, and after detecting a user's fling operation (an operation in which the user taps and drags up on the display screen from point B1 to point B2 and then lifts up, wherein the sliding speed or acceleration of the user's fling operation at point B2 is greater than a preset value) on the display screen, the mobile phone 10 can draw, render, synthesize and display an animation (as a first animation) in which the contact information (as an animation element) in the contact interface U1 moves upward, and the change rule of the position y of the contact information with time t satisfies a first animation curve L1: y = f(t).

[0076] Assuming that the animation of moving the contact information upwards includes 11 frames of images (frame F1 to frame F11), the inverse of the refresh frequency of the display screen is t0 (i.e., the Vsync period is t0), the reference point B3 in the animation element before the start of the first animation corresponds to the position y0 on the animation curve L1 at the time t0, and the mobile phone 10 generates the frame F1, the process can include that: the mobile phone 10 can first determine the animation time of the frame F1 as t0+t0=2t0 according to the animation time t0 corresponding to the position y0 of the animation element in the last frame of image and the Vsync period t0, and determine the position of the animation element in the frame F1 as y1=f(2t0) based on the first animation curve L1, and then draw, render, synthesize and display the interface U2 corresponding to the frame F1, wherein the position difference of the reference point B3 in the interface U1 and the interface U2 is y1-y0. Similarly, the animation times corresponding to the frames F2 to F11 are 3t0, 4t0, 5t0, 6t0, 7t0, 8t0, 9t0, 10t0, 11t0 and 12t0 (an arithmetic sequence with a tolerance of t0), i.e., in the frames F2 to F11, the positions of the animation element are the function values y2=f(3t0), y3=f(4t0), y4=f(5t0), y5=f(6t0), y6=f(7t0), y7=f(8t0), y8=f(9t0), y9=f(10t0), y 10 11 .

[0077] However, if there are timeout frames in the frames F1 to F11 due to the complexity of a certain frame of the first animation, insufficient hardware resources of the mobile phone 10, etc., the mobile phone 10 will adjust the time interval between the animation time of the timeout frame and the animation time of the next image frame of the timeout frame based on the time when the timeout frame is completed, so that the time interval is greater than or less than a Vsync period, thereby causing the animation times of the frames F1 to F11 not to be an arithmetic sequence. During the process of the mobile phone 10 displaying the frames F1 to F11, the display period of each frame of image is a Vsync period, which causes the change rule of the position of the animation element with time in the animation displayed by the mobile phone 10 to be inconsistent with the animation curve.

[0078] For example, assuming that the generation and display process of the frames F1 to F11 shown in FIG. 3A adopts the form shown in FIG. 2, referring to FIG. 3B, the drawing and rendering time of the frame F4 is t0+dt (dt is greater than 0, t0+dt is greater than the Vsync period t0) (i.e., the frame F4 is a timeout frame), then the drawing and rendering time of the frame F5 is t0+dt+t0 (i.e., the frame F5 is a timeout frame), and the drawing and rendering time of the frame F6 is t0+dt+t0+t0 (i.e., the frame F6 is a timeout frame), the mobile phone 10 will adjust the time interval between the animation time of the frame F4 and the animation time of the frame F5 based on the time when the frame F4 is completed, so that the time interval is greater than a Vsync period, thereby causing the animation times of the frames F1 to F11 not to be an arithmetic sequence. ′ ​the animation time of the frame F5' will no longer be y5=f(6t0), but y5'=f(6t0+Δt) (Δt≠0). The animation times of the frames F6' to F11' after the frame F5' are also adjusted to y6'=f(7t0+Δt), y7'=f(8t0+Δt), y8'=f(9t0+Δt), y9'=f(10t0+Δt), y 10 '=f(11t0+Δt), y 11 '=f(12t0+Δt), that is, the animation times of the frames F5' to F11' are equally spaced (t0), while the animation times of the frames F4 and F5' are 6t0+Δt-5t0=t0+Δt>t0. Thus, the adjusted animation times of the frames F1 to F4 and F5' to F11' are 2t0, 3t0, 4t0, 5t0, 6t0+Δt, 7t0+Δt, 8t0+Δt, 9t0+Δt, 10t0+Δt, 11t0+Δt, and 12t0+Δt, respectively.

[0079] During the display of the frames F1 to F4 and F5' to F11' by the mobile phone 10, the durations of the frames are still the same (e.g., the Vsync period t0), that is, the times at which the mobile phone 10 updates the frames F1 to F4 and F5' to F11' also form an arithmetic sequence with a tolerance of the Vsync period t0. Referring to FIG. 3C, if the position of the animation element in the first animation displayed by the mobile phone 10 is consistent with the first animation curve, the reference point B3 of the animation element in the interface U3 corresponding to the frame F4 is located at the position y4, and the reference point B3 of the animation element in the interface corresponding to the frame F5 should be at the position y5 shown in the interface U4. However, the actual generated frame F5' by the mobile phone 10 corresponds to the interface U4 shown in FIG. 3C, and the position O of the reference point B3 of the animation element in the interface U4 is y5'. Thus, the next frame of the interface displayed by the mobile phone 10 after the interface U3 is the interface U5, which causes the rule of the change of the animation element over time to be inconsistent with the first animation curve L1.

[0080] Therefore, an image processing method is provided in the embodiments of the present application. In the method, during generation of a first animation based on a first animation curve, even if the time length of drawing and rendering of a frame image of the first animation exceeds a Vsync period (i.e., the frame image is an overtime frame), a mobile phone determines an animation time of a next frame image of the overtime frame as a sum of the animation time of the overtime frame and the Vsync period, and determines a position of an animation element in the next frame image of the overtime frame based on the first animation curve and the determined animation time. Then, the mobile phone can draw, render, and compose the next frame image of the overtime frame based on the determined position.

[0081] It should be noted that in the case that a frame image in the first animation is not a timeout frame, the electronic device also determines the animation time of the next frame image of the frame image as the sum of the animation time of the frame image and a Vsync period.

[0082] In this way, the interval between the animation times of two adjacent frame images in the first animation is the same (i.e., one Vsync period), which ensures that the position of the animation element in the first animation displayed by the electronic device 10 conforms to the first animation curve.

[0083] For example, for the case shown in FIG. 3B, in the case that the frame F4 is a timeout frame, the mobile phone 10 can set the animation time of the frame F5 as the sum of the animation time of the frame F4 (5t0) and a Vsync period t0, i.e., 6t0. Since the frames F1 to F3 and the frames F5 to F10 are not timeout frames, the animation time of each frame image in the frames F2 to F4 and the frames F6 to F11 is the sum of the animation time of the previous frame image and the Vsync period t0. In this way, referring to FIG. 3D, the animation times of the frames F1 to F11 are 2t0, 3t0, 4t0, 5t0, 6t0, 7t0, 8t0, 9t0, 10t0, 11t0 and 12t0 respectively, which is consistent with the case shown in FIG. 3A, and ensures that the position of the animation element in the first animation displayed by the mobile phone 10 conforms to the first animation curve. For example, the frame F5 generated in the manner shown in FIG. 3D corresponds to the interface U4 shown in FIG. 3C, rather than the interface U5, and the interface displayed by the mobile phone 10 in the next frame after displaying the interface U3 is the interface U4, and the position of the animation element changes over time in accordance with the first animation curve L1.

[0084] In some embodiments, the mobile phone 10 can comprehensively consider the time length between the current time and the time when the next Vsync period arrives, and the usage of the buffer for storing the image data obtained by drawing and rendering, in the process of interpolating the image frames of the first animation. In this way, the total time length of drawing and rendering of a frame image can be avoided from being too long.

[0085] It should be noted that the buffer is used to store the image data obtained by drawing and rendering in the mobile phone 10.

[0086] It is noted that each application that needs to display a frame of picture has one or more buffer queues, and each buffer queue includes a plurality of buffers. During the generation of a frame of picture, a thread of the application that draws and renders the picture can take an idle buffer (indicates that the idle buffer is occupied by the thread that draws and renders the picture) from the buffer queue, draw and render the frame content in the idle buffer, and put the buffer back to the buffer queue after the drawing and rendering is completed. During the composition of a frame of picture, a thread or process (e.g., surface flinger (SF)) that composites the picture in the mobile phone 10 can take a buffer that has been rendered with content from the buffer queue of each application that needs to display content in the frame of picture during the Vsync period of the frame of picture, and render the content of the taken buffer to a frame buffer.

[0087] It is noted that the state of a buffer in the buffer queue can be one of four: dequeued, queued, acquired, and free. Dequeued indicates that a buffer is taken out of the buffer queue by an application and used for drawing and rendering of picture data. Queued indicates that a buffer has stored picture data that has been drawn and rendered. Acquired indicates that a buffer is taken out of the buffer queue by the SF and used for the composition of a frame of picture. Free indicates that a buffer is in an idle state.

[0088] In some embodiments, after the mobile phone 10 completes the drawing and rendering of a frame of picture in a Vsync period, the mobile phone 10 can determine whether to draw and render the next frame of picture in the Vsync period according to whether the frame insertion condition is satisfied. Exemplarily, the frame insertion condition can include that the number of buffers in the buffer queue that are in the free state is greater than a buffer threshold (e.g., 1, 2, 3, 4, etc. (other values are also possible, which are not limited herein)).

[0089] In some embodiments, the buffer threshold can also be dynamically adjusted according to the historical state of the buffers in the buffer queue. For example, if it is detected that the buffer queue is not full or the ratio of the time period that the buffer queue is full to the preset time period is less than a preset ratio (or the ratio of the number of Vsync periods that the buffer queue is full to the preset number of Vsync periods is less than a preset ratio) for a continuous preset time period (or a preset number of Vsync periods, which can be 3, 5, 6, 7 Vsync periods (or other values, which are not limited herein)), the buffer threshold can be decreased to make full use of the buffer. For another example, if it is detected that the buffer queue is full or the ratio of the time period that the buffer queue is full to the preset time period (or the ratio of the number of Vsync periods that the buffer queue is full to the preset number of Vsync periods is greater than or equal to a preset ratio) is greater than or equal to a preset ratio for a continuous preset time period (or a preset number of Vsync periods), the buffer threshold can be increased to reserve enough free buffers.

[0090] For example, referring to FIG. 3E, in the first to sixth Vsync periods: the buffer threshold is 2, and the number of free buffers is 3, 3, 2, 2, 1 and 2, respectively. The mobile phone 10 can determine that the buffers in the buffer queue are not full (all greater than 0) in the six Vsync periods from the first to sixth Vsync periods, and the mobile phone 10 can decrease the buffer threshold from 2 to 1 from the seventh Vsync period. In this way, the mobile phone 10 can perform more frame insertion, and improve the utilization rate of the buffer.

[0091] Continuing to refer to FIG. 3E, after the buffer threshold is decreased to 1, in the twentieth to twenty-fifth Vsync periods: the buffer threshold is 1, and the number of free buffers is 1, 0, 0, 0, 1 and 0, respectively. The buffers in the buffer queue are full in four Vsync periods (the ratio 4 / 6 is greater than a preset ratio (for example, 0.3, or other values)), and the mobile phone 10 can increase the buffer threshold from 1 to 2 from the twenty-sixth Vsync period. In this way, it can be avoided that the time period of one frame of image drawing and rendering is greater than one Vsync period due to the full buffer.

[0092] Based on the above frame insertion condition, it can reduce the possibility that the time period of one frame of image drawing is greater than one Vsync period.

[0093] For example, FIG. 4 shows a schematic diagram of the state of each buffer in the buffer queue in the situation shown in FIG. 2 according to some embodiments of the present application.

[0094] Referring to FIG. 4, the buffer queue includes six buffers: buffer B1, buffer B2, buffer B3, buffer B4, buffer B5 and buffer B6.

[0095] In Vsync period P1: the states of buffer B3 to buffer B6 are idle; before the rendering of frame F1 is completed, the state of buffer B1 is dequeued and the state of buffer B2 is idle; after the rendering of frame F1 is completed, the state of buffer B1 is enqueued and the state of buffer B2 is dequeued.

[0096] In Vsync period P2: the states of buffer B5 and buffer B6 are idle, the state of buffer B2 is enqueued; before the synthesis of frame F1 is completed, the state of buffer B1 is acquired, and after the synthesis of frame F1 is completed, the state of buffer B1 is idle; before the rendering of frame F3 is completed, the state of buffer B3 is dequeued and the state of buffer B4 is idle; before the rendering of frame F4 is completed, the state of buffer B3 is enqueued and the state of buffer B4 is dequeued.

[0097] In Vsync period P3: the states of buffer B5 and buffer B6 are idle, the state of buffer B3 is enqueued, the state of buffer B4 is dequeued, and the state of buffer B1 is idle; before the synthesis of frame F2 is completed, the state of buffer B2 is acquired, and after the synthesis of frame F2 is completed, the state of buffer B2 is idle.

[0098] In Vsync period P4: the states of buffer B6 and buffer B4 are enqueued; before the synthesis of frame F3 is completed, the state of buffer B3 is acquired, and after the synthesis of frame F3 is completed, the state of buffer B3 is idle; before the rendering of frame F5 is completed, the state of buffer B1 is dequeued, and after the rendering of frame F5 is completed, the state of buffer B1 is enqueued; before frame F6 starts to draw and render, the state of buffer B2 is idle, from after the rendering of frame F5 is completed to before the rendering of frame F6 is completed, the state of buffer B2 is dequeued, and after the rendering of frame F6 is completed, the state of buffer B2 is enqueued; before frame F6 starts to draw and render, the state of buffer B5 is idle, and after the rendering of frame F6 is completed, the state of buffer B5 is dequeued.

[0099] In Vsync period P5: the states of buffer B1, buffer B2 and buffer B5 are enqueued; before the synthesis of frame F4 is completed, the state of buffer B4 is acquired, after the synthesis of frame F4 is completed and before frame F10 starts to draw and render, the state of buffer B4 is idle, and after frame F10 starts to draw and render, the state of buffer B4 is dequeued; before the rendering of frame F8 is completed, the state of buffer B3 is dequeued, and after the rendering of frame F8 is completed, the state of buffer B3 is enqueued; before frame F9 starts to draw and render, the state of buffer B2 is idle, from after the rendering of frame F8 is completed to before the rendering of frame F9 is completed, the state of buffer B6 is dequeued, and after the rendering of frame F9 is completed, the state of buffer B6 is enqueued. That is to say, after frame F10 starts to draw, all the buffers of the buffer queue are not idle.

[0100] In Vsync period P6: the states of buffer B2 to buffer B6 are enqueued; the state of buffer B1 is acquired before the synthesis of frame F5 is completed, and the state of buffer B1 is idle after the synthesis of frame F5 is completed. When the Vsync signal of Vsync period P6 arrives, the mobile phone 10 attempts to acquire an idle buffer for drawing and rendering frame F11 from the buffer queue, but since the states of buffer B1 to buffer B6 are all not idle, the mobile phone 10 needs to wait until the Vsync signal of the next Vsync period arrives to acquire an idle buffer.

[0101] In Vsync period P7: the states of buffer B3 to buffer B6 are enqueued, and the state of buffer B1 is dequeued; the state of buffer B2 is acquired before the synthesis of frame F6 is completed, and the state of buffer B2 is idle after the synthesis of frame F6 is completed.

[0102] Based on the above process, it can be known that since the states of all buffers in the buffer queue are not idle when the Vsync signal of Vsync period P6 arrives, the idle buffer for drawing and rendering frame F11 needs to wait for one Vsync period to be acquired, so that the time length of drawing and rendering of frame F11 exceeds one Vsync period.

[0103] In the embodiments of the present application, since the insertion frame condition includes that the number of buffers in the idle state in the buffer queue is greater than the buffer threshold, it can be avoided that the buffers in the buffer queue are occupied due to insertion frame, so that the time length of drawing and rendering of a frame of image does not exceed one Vsync period due to waiting for acquiring an idle buffer for drawing and rendering.

[0104] For example, FIG. 5 shows a process diagram of generating the foregoing frames F1 to F11 according to some embodiments of the present application.

[0105] As shown in FIG. 5:

[0106] After the Vsync signal of Vsync period P1 arrives, the mobile phone 10 draws and renders frame F1. Moreover, after frame F1 is drawn, the number of idle buffers (5, i.e., buffers B2 to B5) in the buffer queue is greater than the buffer threshold (for example, 1), so that the mobile phone 10 can draw and render frame F2 (insertion frame F2).

[0107] After the Vsync signal of Vsync period P2 arrives, the mobile phone 10 synthesizes frame F1 and draws and renders frame F3. Moreover, after frame F1 is drawn, the number of idle buffers (3, i.e., buffers B4 to B5) in the buffer queue is greater than 1, so that the mobile phone 10 can draw and render frame F4 (insertion frame F4).

[0108] After the Vsync signal of the Vsync period P3 arrives, the mobile phone 10 composites the frame F2, displays the frame F1, and the mobile phone 10 continues to draw the frame F4.

[0109] After the Vsync signal of the Vsync period P4 arrives, the mobile phone 10 first draws and renders the frame F5, composites the frame F3, and displays the frame F2. After the frame F5 is drawn and rendered, the number of free buffers (3, i.e., the buffers B2, B5, and B6) in the buffer queue is greater than 1, and the mobile phone 10 can draw and render the frame F6 (interpolated frame F6). After the frame F6 is drawn and rendered, the number of free buffers (2, i.e., the buffers B5 and B6) in the buffer queue is greater than 1, and the mobile phone 10 can draw and render the frame F7 (interpolated frame F7).

[0110] After the Vsync signal of the Vsync period P5 arrives, the mobile phone 10 first draws and renders the frame F8, composites the frame F4, and displays the frame F3. After the frame F8 is drawn and rendered, the number of free buffers (2, i.e., the buffers B4 and B6) in the buffer queue is greater than 1, and the mobile phone 10 can draw and render the frame F6 (interpolated frame F9). After the frame F6 is drawn and rendered, the number of free buffers (1, i.e., the buffers B5 and B6) in the buffer queue is equal to 1, and the mobile phone 10 can not interpolate frames.

[0111] After the Vsync signal of the Vsync period P6 arrives, the mobile phone 10 first draws and renders the frame F10, composites the frame F5, and displays the frame F4. After the frame F10 is drawn and rendered, the number of free buffers (0) in the buffer queue is less than 1, and the mobile phone 10 can not interpolate frames.

[0112] After the Vsync signal of the Vsync period P7 arrives, the mobile phone 10 first draws and renders the frame F11, composites the frame F6, and displays the frame F5.

[0113] Based on the above process, it can be seen that the drawing and rendering of the frame F11 lasts for one Vsync period, which avoids the case that the drawing and rendering of the frame F11 lasts for more than one Vsync period as shown in FIG. 4.

[0114] In some embodiments, the first operation can be an operation of triggering a flicking animation, such as an operation in which a user slides from one position (as a start position) to another position (as an end position) on the screen and then lifts up, and a speed or acceleration of the user's sliding at the end position is greater than or equal to a preset value. In some embodiments, the first animation can be a flicking animation, i.e., an animation in which a display element in the first interface is moved according to a first animation curve after detecting the first operation of the user in the first interface. For example, the first animation can be an animation in which an animated element moves in a first direction and a speed of the movement gradually decreases (or the speed first remains unchanged and then decreases). For another example, the first animation can be an animation in which an animated element moves a first distance in a first direction and then moves a second distance in a direction opposite to the first direction, where the first distance is greater than the second distance, and a speed of the movement of the animated element in the first direction gradually decreases (or the speed first remains unchanged and then decreases), and a speed of the movement of the animated element in the second direction first increases and then decreases (or the speed first remains unchanged and then decreases).

[0115] In some embodiments, the first operation can also be a flicking animation-triggering air gesture, such as an operation in which a user waves a hand in a certain direction (such as an up-down direction, a left-right direction, etc. of a display interface of the electronic device).

[0116] In some embodiments, the first animation curve can be an arbitrary curve of a position of an animated element changing with time according to user demand. For example, a rate of change of the position with time (i.e., a speed) in the animation curve can decrease with an increase of time, or first remain unchanged and then decrease with an increase of time, or first decrease, then increase, and then decrease with a change of time, etc.

[0117] In some embodiments, the first animation curve can be a curve of a plurality of curves, a Bezier curve, a hyperbolic curve, a trigonometric function (such as a sine function, a cosine function, a tangent function, an inverse tangent function, etc.), or a combination of the above curves, etc.

[0118] In some embodiments, the first animation curve can be a curve obtained by splicing a plurality of curves, and a part of the curves in the first animation curve can be represented by one function, and another part of the curves can be represented by one or more other functions.

[0119] In some embodiments, the position of the animation element determined based on the animation time and the animation curve can be a relative position or an absolute position. For example, the position of the animation element can be an absolute position coordinate of a reference position (e.g., top, bottom, left, right, center, or other position) in the animation element on the display screen of the mobile phone 10. For another example, the position of the animation element can be a relative movement position of a reference position (e.g., top, bottom, left, right, center, or other position) in the animation element relative to the previous frame image or the first frame image of the first animation.

[0120] For ease of understanding, the software architecture of the mobile phone 10 is first introduced below.

[0121] FIG. 6 shows a schematic diagram of a software architecture of a mobile phone 10 according to some embodiments of the present application.

[0122] As shown in FIG. 6, the software architecture of the mobile phone 10 can be a layered architecture, which can include an application layer, a framework layer, a native layer, and the like.

[0123] The application layer can include application programs in the mobile phone 10.

[0124] In some embodiments, the application programs in the application layer can draw and render the image frames corresponding to the first animation according to the detected first operation of the user when running. For example, the application program can include a UI thread for drawing the display content of the application program and a rendering thread for rendering the display content of the application program. For a frame image in the first animation, the display content corresponding to the frame image can be drawn by the UI thread, and the content of the frame image can be rendered by the rendering thread according to the content drawn by the UI thread.

[0125] In some embodiments, the application programs in the application layer can also trigger the drawing management module to generate the first animation and send information of the first operation (e.g., the time when the first operation is detected (i.e., the time when the first animation starts), etc.) to the animation management module according to the detected first operation of the user.

[0126] The framework layer provides application programming interfaces (APIs) and programming frameworks for the application programs of the application. The application program framework layer includes some pre-defined functions, such as the drawing management module, the animation management module, the interpolation decision module, and the like.

[0127] In some embodiments, the drawing management module can call the corresponding hardware resources (e.g., buffer, processor resources, etc.) to draw and render the i-th frame image according to the Vsync signal and the position of the animation element sent by the animation management module when detecting that the application triggers the generation of the first animation. The drawing management module can also pass the buffer state information (e.g., the state of each buffer in the buffer queue, the number of buffers in different states, etc.) to the interpolation decision module and obtain the interpolation decision result (interpolation or no interpolation) of whether to interpolate from the interpolation decision module. In addition, the drawing management module can also generate a software Vsync signal of the next frame (i+1-th frame) and trigger the drawing and rendering of the i+1-th frame image in the case that the received interpolation decision result is interpolation.

[0128] The animation management module can determine the animation time of each frame image according to the information of the received first operation and the duration of the drawing and rendering of each frame image, determine the position of the animation element in each frame image based on the animation time, and pass the determined position of the animation element to the drawing management module.

[0129] For example, in the case that the duration of the drawing and rendering of a frame image is greater than one Vsync period (i.e., the frame image is an overtime frame), the animation management module can determine the animation time of the next frame image as the sum of the animation time of the overtime frame and the Vsync period, and obtain the position of the animation element in the next frame image based on the determined animation time and the animation curve and pass it to the drawing management module.

[0130] For another example, in the case that the duration of the drawing and rendering of a frame image is less than or equal to one Vsync period, the animation management module can also determine the animation time of the next frame image as the sum of the animation time of the frame image and the Vsync period, and obtain the position of the animation element in the next frame image based on the determined animation time and the animation curve and pass it to the drawing management module.

[0131] In some embodiments, the animation management module can also pass the start and end information of the animation (e.g., the time when the first animation starts and the time when the animation ends) to the interpolation decision module.

[0132] The interpolation decision module is configured to determine whether to interpolate according to the start and end information of the animation and the buffer state information. For example, the interpolation decision module can determine interpolation in the case that the aforementioned interpolation condition is met, and determine no interpolation in the case that the aforementioned interpolation condition is not met.

[0133] In some embodiments, the draw management module, the animation management module, and the in-between frame decision module can run in threads of respective applications, such as a main thread, a draw thread (UI thread), and the like of respective applications. In some embodiments, the draw management module, the animation management module, and the in-between frame decision module can run in applications that need to display user interfaces in the mobile phone 10.

[0134] In some other embodiments, the framework layer can further include other modules, which are not limited herein. For example, the framework layer can further include a view system (VS), an input manager service (IMS), a scroll, an overscroll, an SF, a choreographer, and the like.

[0135] The view system includes visual controls and methods for managing and implementing the visual controls. For example, the view system can include a view root implementation (viewrootimpl) and an animation utility implementation (animationutilsimpl). The viewrootimpl can notify other modules that a first animation starts after detecting a user-triggered operation that triggers the first animation according to a call from the other modules (e.g., a call to an onFling method, which is a predefined function that can be used to detect a user operation (e.g., a swipe operation, a flick operation) on the mobile phone 10). The animationutilsimpl can be used to update and manage the state of an animation. For example, the animationutilsimpl can be used to determine an animation time for a frame image in the first animation based on the scroll / overscroll or based on the animation management module.

[0136] The IMS includes the input event reading module, the input event filtering module, and the input event dispatching module described above, and is responsible for a service of managing input events corresponding to external input devices. In some embodiments, the IMS can dispatch a detected operation to the viewrootimpl, so that the viewrootimpl can identify a first operation (e.g., a flick operation) of a user.

[0137] The scroll / overscroll can be used to determine an animation time for a frame image in the first animation. If a frame image is an overtime frame in the animation time determined by the scroll / overscroll, the interval between the animation time of the next frame image and the animation time of the overtime frame can be greater than a Vsync period.

[0138] SF is configured to synthesize the image data of the drawing and rendering of the application to obtain a frame of image. For example, SF can synthesize the image data in the plurality of buffers corresponding to a frame of image to obtain an image and store the obtained image in the frame buffer, so that the mobile phone 10 can read the image from the frame buffer and display the image on the display screen.

[0139] The choreographer can be configured to process the Vsync signal and the message of the application to control the drawing and rendering of the image. For example, the Vsync signal for the interpolation can be triggered in a Vsync signal period to instruct the corresponding thread (e.g., the drawing thread and the rendering thread) to draw and render one or more frames of image in a Vsync signal period.

[0140] In some embodiments, the framework layer can further include more or less modules, which are not limited herein.

[0141] The native layer is configured to provide support for the application layer and the framework layer. The native layer can include native libraries, such as a graphical user interface library (libgui), a hardware-accelerated user interface library (libhwui), and the like.

[0142] The libgui can be configured to manage the buffer queue, such as increasing or decreasing the number of buffers in the buffer queue according to the call of other modules, and the like. The libhwui can be configured to call the graphics processing hardware (e.g., a hardware compositor, a graphics processing unit, and the like) in the mobile phone 10 to render the image.

[0143] In some embodiments, the native layer can further include more or less modules, which are not limited herein.

[0144] In some embodiments, the software architecture of the mobile phone 10 can include more layers, such as a kernel layer, and the like, which are not limited herein.

[0145] The technical solutions of the present application will be described below in conjunction with the software architecture shown in FIG. 6.

[0146] First, the scheme of the mobile phone 10 for interpolating the image in the first animation will be described.

[0147] Exemplarily, FIG. 7 shows a flowchart of an image processing method according to some embodiments of the present application. As shown in FIG. 7, the flowchart includes the following steps:

[0148] S701, detecting that a user triggers a first operation of a first animation.

[0149] After detecting that the user triggers the first operation of the first animation, the mobile phone 10 triggers the image processing method provided in the embodiments of the present application.

[0150] In some embodiments, the first operation can be an operation in which a user slides (e.g., acceleration slide, etc.) from one position on the screen (as a start position) to another position (as an end position) and then lifts up, and a speed or acceleration at which the user slides at the end position is greater than a preset value (e.g., 0, or other numerical value) (the operation can also be referred to as a throw slide operation). For example, for the scenario shown in FIG. 3A, after detecting an operation in which the user accelerates and slides up from B1 to B2 and then lifts up (as the first operation), the mobile phone 10 can detect an operation in which the user triggers an animation in which the contact information moves upward (as the first animation).

[0151] In other embodiments, the first operation can also be other operations (e.g., a mid-air gesture that triggers a throw slide animation) that can trigger the first animation, which are not limited herein.

[0152] It should be noted that for different first animations, the mobile phone 10 can detect whether the operation of the user is the first operation for triggering the first animation by the form of the first operation corresponding to the first animation.

[0153] For example, for the throw slide operation, the mobile phone 10 can listen to whether the throw slide operation is detected by registering a callback of the onfling method of the viewrootimpl. The mobile phone 10 can detect the first operation of the user for triggering the first animation when the callback of the onfling method indicates that the throw slide operation is detected. It should be noted that the logic or manner in which the viewrootimpl detects the throw slide operation can be defined by the developer according to different operating systems or devices, which are not limited herein.

[0154] S702, detecting a Vsync signal, drawing and rendering an i-th frame image of the first animation.

[0155] After detecting the first operation of the user for triggering the first animation, the mobile phone 10 can draw and render an i-th frame image of the first animation after detecting a Vsync signal. The i-th frame image can be the first frame image in a plurality of continuous frame images corresponding to the first animation, which has not been completely drawn and rendered. That is, the first i-1 frame images in the plurality of continuous frame images have been completely drawn and rendered.

[0156] For example, after detecting the Vsync signal, if there currently exists an image frame (as the i-th frame image) that has been started to be drawn and rendered but has not been completely rendered, the mobile phone 10 can continue to draw and render the image frame.

[0157] For another example, after detecting the Vsync signal, if there currently does not exist an image frame that has been started to be drawn and rendered but has not been completely rendered, the first frame image in the image frame that has not been started to be drawn can be started to be drawn and rendered.

[0158] In some embodiments, the phone 10 can determine the position of the animation element in the i-th frame image based on the animation time and the first animation curve, and draw and render the i-th frame image according to the position of the animation element in the i-th frame image. For example, for the case shown in FIG. 3A, when the i-th frame image is frame F1, the phone 10 can determine the position of the animation element in frame F1 as y1 based on the animation time 2t0 and the first animation curve L1, and draw and render the i-th frame image with the animation element at y1.

[0159] It should be noted that when step S702 is performed for the first time, the i-th frame image can be the first frame image of the first animation.

[0160] It should be noted that in some embodiments, the phone 10 can further set a switch for starting or stopping the image processing method provided in the embodiments of the present application. When the switch is on, the phone 10 performs the subsequent steps S703 to S704, otherwise the phone 10 draws and renders the image frames of the first animation according to the process defined by the operating system of the phone 10 (for example, the aforementioned scrolling, the process defined by the scrolling).

[0161] S703, when the i-th frame image is drawn and rendered and the interpolation condition is met, start drawing and rendering the (i+1)-th frame image of the first animation at the current Vsync period.

[0162] After the i-th frame image is drawn and rendered, the phone 10 can start drawing and rendering the (i+1)-th frame image of the first animation at the current Vsync period when the interpolation condition is met (i.e., the image can continue to be drawn and rendered in the current Vsync period).

[0163] In some embodiments, the interpolation condition can include that the number of buffers in the idle state in the buffer queue is greater than a buffer threshold (for example, 1, 2, 3, 4, etc., or other values, which are not limited herein). In this way, it can be ensured that the buffer queue will not be full due to too many interpolations, and the drawing and rendering time of a frame image will not be prolonged due to the full buffer queue.

[0164] For example, if the buffer threshold is 1, for the case shown in FIG. 5, after the phone 10 draws and renders frame F8 (as the i-th frame image) at Vsync period P5, the number of buffers in the idle state in the buffer queue is 2 (buffer B4 and buffer B6), which is greater than the buffer threshold (i.e., the interpolation condition is met), and the phone 10 can continue to draw and render frame F9 (as the (i+1)-th frame image).

[0165] In some embodiments, if an application corresponds to multiple buffer queues (for example, the application has multiple windows), the application can be inserted in frame if the insertion condition is met in each of the multiple buffer queues. That is, in this case, the insertion condition can be that the number of idle buffers in each of the multiple buffer queues is greater than the buffer threshold (for example, 1, 2, 3, 4, or other values, which are not limited herein). In this way, it can be avoided that the image frames of the buffers corresponding to different windows of the same application do not match, resulting in abnormal image synthesis (for example, the number of image frames of multiple windows corresponding to multiple buffers is inconsistent, resulting in inconsistent refresh frequencies of two window images, causing the electronic device to be unable to determine which refresh frequency to refresh the displayed image).

[0166] For example, referring to FIG. 8, the interface U6 of the video application includes a video window W1 and a hot review window W2. The video window W1 is drawn and rendered based on the buffer queue Q1, and the number of idle buffers in the buffer queue Q1 is 2. The hot review window W2 is drawn and rendered based on the buffer queue Q2, and the number of idle buffers in the buffer queue Q2 is 3. After detecting the upward flicking operation (as the first operation) of the user in the hot review window W2 region, the electronic device can display the comment content in the hot review window W2 by the animation moving upward (as the first animation). If the buffer threshold is 1, in the case shown in FIG. 8, since the number of idle buffers in the buffer queue Q1 and the buffer queue Q2 is greater than the buffer threshold (1), it can be determined that the insertion condition is met. If the buffer threshold is 2, in the case shown in FIG. 8, since the number of idle buffers in the buffer queue Q1 (2) is not greater than the buffer threshold (2), and the number of idle buffers in the buffer queue Q2 (3) is greater than the buffer threshold (2), it can be determined that the insertion condition is not met.

[0167] In some embodiments, if an application corresponds to multiple buffer queues, the image frames of the application can also not be inserted in frame.

[0168] In some embodiments, the frame insertion condition can also be that the time length between the current time and the time when the first operation is detected (or the time when the first animation starts) is less than a time length threshold. Exemplarily, the time length threshold can be 100 ms, 200 ms, 300 ms, or any other value. That is, within a time period corresponding to the time length threshold after the first animation is triggered, the phone 10 can not consider the number of buffers in the idle state in the buffer queue, but can perform frame insertion as long as there is an idle buffer in the buffer queue. In this way, since the need for computing resources of the thread or process for drawing, rendering, and synthesizing images can suddenly increase after an animation is triggered, performing frame insertion within a time period corresponding to the time length threshold after the first animation is triggered can avoid the subsequent image frames from being drawn in time and causing lag.

[0169] In some embodiments, the phone 10 can also adjust the buffer threshold according to the historical state of the buffers in the buffer queue, and determine whether the frame insertion condition is met based on the adjusted buffer threshold. For example, the phone 10 can decrease the buffer threshold when it is detected that the buffer queue is not full or the ratio of the full time length to the preset time length (or the ratio of the number of full Vsync periods to the preset number of Vsync periods) is less than a preset ratio for a continuous preset time length (or a preset number of Vsync periods, such as 3, 5, 6, or 7 Vsync periods (or any other value, which is not limited herein)). For another example, the phone 10 can increase the buffer threshold when it is detected that the buffer queue is full or the ratio of the full time length to the preset time length (or the ratio of the number of full Vsync periods to the preset number of Vsync periods) is greater than or equal to a preset ratio for a continuous preset time length (or a preset number of Vsync periods), so as to reserve enough idle buffers.

[0170] In some embodiments, the phone 10 can first determine whether the frame insertion condition is met and then adjust the buffer threshold according to the historical state of the buffers in the buffer queue, which is not limited herein.

[0171] In some embodiments, the phone 10 can determine whether the frame insertion condition is met by calling the aforementioned frame insertion decision module.

[0172] In some embodiments, the phone 10 can generate a software Vsync signal for drawing and rendering the (i+1)th image of the first animation based on the choreographer when the frame insertion condition is met, so that the phone 10 can draw and render the (i+1)th image of the first animation based on the software Vsync signal.

[0173] In some embodiments, no matter whether the drawing and rendering of the i-th image frame is longer than, shorter than or equal to a Vsync period, the interval between the animation time of the i-th image frame and the i+1-th image frame is a Vsync period. That is, the interval between the animation time of any i-th image frame and the i+1-th image frame in the first animation is the same value (e.g. a Vsync period). The specific implementation process can refer to the embodiment shown in FIG. 9 below, which will not be described here.

[0174] In some embodiments, when the interpolation condition is met, the mobile phone 10 can call the choreographer to trigger interpolation through the calling interface provided by the choreographer, so that the mobile phone 10 draws and renders the i+1-th image frame of the first animation.

[0175] In some embodiments, each application program (e.g. the main thread, the UI thread, the rendering thread, etc. of the application program) in the mobile phone 10 that needs to display content can interpolate based on the method of the foregoing step S703. For example, the main thread or the UI thread of the application program determines whether the interpolation condition is met and triggers interpolation, and the UI thread and the rendering thread draw and render the image frames of the first animation.

[0176] S704, when the i-th image frame is drawn and rendered and the interpolation condition is not met, a next Vsync signal is detected, and the i+1-th image frame of the first animation is drawn and rendered.

[0177] After the i-th image frame is drawn and rendered, the mobile phone 10 can draw and render the i+1-th image frame of the first animation when a next Vsync signal is detected without meeting the interpolation condition.

[0178] For example, if the buffer threshold is 1, for the situation shown in FIG. 5, after the mobile phone 10 draws and renders the frame F9 (as the i-th image frame) in the Vsync period P5, the number of idle buffers in the buffer queue is 1 (the buffer B6), which is equal to the buffer threshold (i.e. the interpolation condition is not met), and the mobile phone 10 can start drawing and rendering the frame F10 after the Vsync signal of the next Vsync period arrives.

[0179] Based on the above method, the mobile phone 10 can interpolate only when the above interpolation condition is met, which can avoid the total time of drawing and rendering of a frame of image exceeding a Vsync period due to the buffer queue being full.

[0180] The technical solution of the mobile phone 10 ensuring that the interval between the animation times of adjacent two image frames in the first animation is introduced below.

[0181] Exemplarily, FIG. 9 shows a flowchart of an image processing method according to some embodiments of the present application. As shown in FIG. 9, the flow includes the following steps:

[0182] S901, determining a first animation time point of the jth frame of image, and drawing and rendering the jth frame of image according to the first animation time point and a first animation curve corresponding to the first animation.

[0183] Exemplarily, the mobile phone 10 can determine the position of the animation element in the jth frame of image based on the animation time point of the jth frame of image and the first animation curve, and draw and render the jth frame of image according to the position of the animation element in the jth frame of image.

[0184] For example, for the case shown in FIG. 3A and FIG. 3D, when the jth frame of image is frame F1, the first animation time point is 2t0, the mobile phone 10 can determine the position of the animation element in frame F1 as y1=f(2t0) based on 2t0 and the aforementioned animation curve L1. Then, the mobile phone 10 can draw and render frame F1 when the animation element (e.g. the contact information shown in FIG. 3A) is located at position y1.

[0185] For example, for the case shown in FIG. 3A and FIG. 3D, when the jth frame of image is frame F4, the first animation time point is 5t0, the mobile phone 10 can determine the position of the animation element in frame F4 as y4=f(5t0) based on 5t0 and the aforementioned animation curve L1. Then, the mobile phone 10 can draw and render frame F4 when the animation element (e.g. the contact information shown in FIG. 3A) is located at position y4.

[0186] S902, detecting that the duration of drawing and rendering of the jth frame of image is greater than a Vsync period, setting the second animation time point of the j+1th frame of image as the sum of the first animation time point and the Vsync period.

[0187] After the mobile phone 10 completes the drawing and rendering of the jth frame of image, if it is detected that the duration of drawing and rendering of the jth frame of image is greater than a Vsync period (i.e. the jth frame of image is an overtime frame), the second animation time point of the j+1th frame of image can be set as the sum of the first animation time point and the Vsync period.

[0188] Exemplarily, for the case shown in FIG. 3A and FIG. 3D, when the mobile phone 10 completes the drawing and rendering of frame F4 (as the jth frame of image), it can determine that the duration of drawing and rendering of frame F4 (t0+Δt) is greater than a Vsync period (t0), and set the animation time point of frame F5 (as the j+1th frame of image) as the sum of the Vsync period (t0) and the first animation time point (5t0) (i.e. t0+5t0=6t0).

[0189] It should be noted that the Vsync period can be determined by the current refresh frequency of the display screen of the mobile phone 10. The current refresh frequency of the mobile phone 10 can be a refresh frequency set by a user or a refresh frequency adjusted by the mobile phone 10 according to its own running logic. For example, the refresh frequency of the mobile phone 10 can be 30 Hz, 60 Hz, 90 Hz, 120 Hz, 180 Hz, etc. Illustratively, if the current refresh frequency of the mobile phone 10 is 120 Hz, i.e., the Vsync period is 1 / 120 Hz = 8.3 ms. That is, the time difference between the second animation time and the first animation time is 8.3 ms.

[0190] In some embodiments, the mobile phone 10 can determine whether the time length of drawing and rendering of each frame image is greater than a Vsync period by the aforementioned animation management module, and in the case that the time length of drawing and rendering of the jthframe image is greater than a Vsync period, set the second animation time of the j+1thframe as the sum of the first animation time and the Vsync period.

[0191] It should be noted that in some embodiments, in the case that the time length of drawing and rendering of the jthframe image is less than or equal to a Vsync period, the mobile phone 10 also sets the second animation time of the j+1thframe as the sum of the first animation time and the Vsync period. Illustratively, in the case that the time length of drawing and rendering of the jthframe image is less than or equal to a Vsync period, the second animation time of the j+1thframe can be determined by a module (such as a scrolling module, an overscrolling module, etc.) in the operating system of the mobile phone 10 for managing interpolation, or can be determined by the aforementioned animation management module, which is not limited herein.

[0192] For example, for the case shown in FIG. 3D, corresponding to the jthframe image, the frames F1, F2, F3, F5, F6, F7, F8, F9, and F10, the mobile phone 10 can set the second animation time of the j+1thframe image as 3t0, 4t0, 5t0, 7t0, 8t0, 9t0, 10t0, 11t0, and 12t0, respectively. Then, the mobile phone 10 can draw and render the frames F2, F3, F4, F6, F7, F8, F9, F10, and F11 with the positions y2=f(3t0), y3=f(4t0), y4=f(5t0), y6=f(7t0), y7=f(8t0), y8=f(9t0), y9=f(10t0), y 10 =f(11t0), and y 11 =f(12t0) of the animation elements, respectively.

[0193] In some embodiments, the time length of drawing and rendering of a frame image can be the time length (or a part of the time length, or a sum of the time length and a fixed value) that the frame image is processed by the drawing and rendering thread, for example, the time length from starting to acquire the buffer corresponding to the frame image to completing drawing and rendering of the content of the frame image and storing into the acquired buffer.

[0194] In some embodiments, the time length of drawing and rendering of a frame image can also be the total time length (or a part of the total time length, or a sum of the total time length and a fixed value) that the frame image is processed by the main thread of the application and the drawing and rendering thread.

[0195] It should be noted that in other embodiments, the calculation manner of the time length of drawing and rendering of a frame image can also be defined by the operating system of the mobile phone 10, and the mobile phone 10 can acquire the time length of drawing and rendering of a frame image based on the calling interface provided by the operating system.

[0196] It should be noted that in some embodiments, the mobile phone 10 can also set a switch for starting or stopping the image processing method provided in the embodiments. When the switch is started, the mobile phone 10 performs the subsequent steps S903 to S903, otherwise, the image frames of the first animation are drawn and rendered according to the process (for example, the process defined by the foregoing scrolling and exceeding scrolling) of drawing and rendering of the first animation defined by the operating system of the mobile phone 10.

[0197] S903, drawing and rendering the j+1th frame image of the first animation according to the second animation time and the first animation curve.

[0198] After determining the second animation time of the second animation, the mobile phone 10 can draw and render the j+1th frame image of the first animation based on the second animation time and the first animation curve.

[0199] For example, the mobile phone 10 can determine the position of the animation element in the j+1th frame image based on the second animation time and the first animation curve, and draw and render the j+1th frame image according to the position of the animation element in the j+1th frame image.

[0200] For example, for the case shown in FIGS. 3A and 3D, after determining that the animation time of frame F5 (as the j+1th frame image) is 6t0, the mobile phone 10 can determine the position y5=f(6t0) of the animation element in frame F5 based on 6t0 and the first animation curve, and draw and render frame F5 with the animation element at the position y5.

[0201] In some embodiments, each application program (e.g., the main thread, the UI thread, the rendering thread, etc. of the application program) that needs to display content in the mobile phone 10 can determine the animation time of each frame image of the first animation based on the foregoing method and draw and render each frame image. For example, the main thread or the UI thread of the application program can determine whether the time length of drawing and rendering of a frame image is greater than the Vsync period, the main thread can determine the animation time of each frame image, and the UI thread and the rendering thread can actually draw and render the content of a frame image.

[0202] Based on the foregoing method, even if there is a timeout frame in the images of the first animation, in which the time length of drawing and rendering exceeds one Vsync period, the mobile phone 10 can set the animation time of the next frame image of the timeout frame as the sum of the animation time of the timeout frame and the Vsync period. In this way, it can be ensured that the animation times of each frame image of the first animation are in an arithmetic sequence, that is, the interval between the animation times of two adjacent frame images in the first animation is the same (e.g., one Vsync period), so that the position of the animation element in the first animation displayed by the mobile phone 10 changes over time according to the first animation curve, which is beneficial to improving the user experience.

[0203] In combination with the foregoing embodiments, the embodiments of the present application further provide an image processing method.

[0204] Exemplarily, FIG. 10 shows a flowchart of another image processing method according to some embodiments of the present application. The execution subject of the method is the mobile phone 10, as shown in FIG. 10, and the method includes the following steps:

[0205] S1001, detecting that a user triggers a first operation of a first animation.

[0206] The mobile phone 10 can trigger the image processing method provided by the embodiments of the present application when detecting that the user triggers the operation of the first animation. The specific manner in which the mobile phone 10 detects the first operation can refer to the content of the foregoing S701, which is not limited herein.

[0207] S1002, detecting a Vsync signal, determining the animation time of the i-th frame image of the first animation, and drawing and rendering the i-th frame image of the first animation according to the animation time and the first animation curve corresponding to the first animation.

[0208] After detecting the first operation of the user, the mobile phone 10 can determine the animation time of the i-th frame image of the first animation and draw and render the i-th frame image of the first animation according to the animation time and the first animation curve corresponding to the first animation every time a Vsync signal is detected. The i-th frame image can be the first frame image that has not been drawn and rendered among the plurality of continuous frame images corresponding to the first animation.

[0209] In some embodiments, the first animation time can be the sum of the animation time of the (i-1)th frame image of the first animation and a Vsync period when i is greater than 1; the first animation time can be any one of the time points in the first animation curve when i is 1.

[0210] S1003, when the i-th frame image is drawn and rendered and the interpolation condition is met, the second animation time of the (i+1)th frame is set to the sum of the first animation time and a Vsync period in the current Vsync period, and the (i+1)th frame image of the first animation is drawn and rendered according to the second animation time and the first animation curve corresponding to the first animation.

[0211] After the i-th frame image is drawn and rendered, the mobile phone 10 can set the second animation time of the (i+1)th frame to the sum of the first animation time and a Vsync period in the current Vsync period when the interpolation condition is met (i.e., the image can continue to be drawn and rendered in the current Vsync period), and draw and render the (i+1)th frame image of the first animation according to the second animation time and the first animation curve corresponding to the first animation.

[0212] In some embodiments, the interpolation condition can include at least one of the following conditions: the application in which the first animation is located has only one buffer queue, and the number of free buffers in the buffer queue is greater than the buffer threshold; the application in which the first animation is located includes a plurality of buffer queues, and the number of free buffers in each buffer queue of the plurality of buffer queues is greater than the buffer threshold; the time interval between the current time and the time when the first operation of the user is detected (or the time when the first animation starts) is less than the time threshold.

[0213] In some embodiments, the mobile phone 10 can also adjust the buffer threshold according to the historical state of the buffers in the buffer queue, and determine whether the interpolation condition is met based on the adjusted buffer threshold, or adjust the buffer threshold according to the historical state of the buffers in the buffer queue after determining whether the interpolation condition is met. The specific way in which the mobile phone 10 adjusts the buffer threshold can be referred to the related content of the foregoing step S703 and FIG. 3E, which will not be described here.

[0214] In some embodiments, when the interpolation condition is met, the mobile phone 10 sets the second animation time of the (i+1)th frame to the sum of the first animation time and a Vsync period, regardless of whether the duration of the drawing and rendering of the i-th frame image is greater than a Vsync period.

[0215] S1004, in a case where the image drawing and rendering of the i-th frame is completed and the interpolation condition is not met, in a case where a next Vsync period is detected, setting a second animation time of the i+1-th frame as a sum of the first animation time and the Vsync period, and drawing and rendering the i+1-th frame image of the first animation according to the second animation time and the first animation curve corresponding to the first animation.

[0216] In a case where the image drawing and rendering of the i-th frame is completed and the interpolation condition is not met, no interpolation is performed in the current Vsync period. The mobile phone 10 can set a second animation time of the i+1-th frame as a sum of the first animation time and the Vsync period in a case where a next Vsync period is detected, and draw and render the i+1-th frame image of the first animation according to the second animation time and the first animation curve corresponding to the first animation.

[0217] In this way, it can be ensured that the animation time corresponding to each image frame of the image frames of the first animation drawn and rendered by the mobile phone 10 is an arithmetic sequence (for example, an arithmetic sequence with a common difference of the Vsync period), that is, the time interval between the animation times of any two adjacent image frames in the multiple image frames of the first animation is equal. Based on the above method, when the mobile phone 10 displays the image frames of the first animation at a fixed refresh frequency, the position of the animation element in the displayed picture matches the first animation curve, which is beneficial to improving the user experience.

[0218] Embodiments of the present application also provide an image processing method. In the method, an electronic device can generate a first animation and interpolation frames based on the above-mentioned embodiments.

[0219] Exemplarily, FIG. 11 shows a flowchart of an image processing method according to some embodiments of the present application. The execution subject of the method is the mobile phone 10. As shown in FIG. 11, the method comprises the following steps:

[0220] S1101, displaying a first interface, the first interface comprising a first display element.

[0221] The mobile phone 10 displays a first interface, and the first interface can comprise a first display element. For example, the first display element can be the above-mentioned animation element.

[0222] S1102, detecting that a user triggers a first operation of a first animation.

[0223] The mobile phone 10 detects that a user triggers a first operation of a first animation. The first operation can be used to trigger the first animation, for example, the above-mentioned flicking operation of the user on the display screen of the mobile phone 10, or a hand gesture in a certain direction. For details, reference can be made to the above-mentioned step S601, which is not limited herein.

[0224] S1103, in response to the first operation, generating and displaying a first animation of movement of the first display element based on a first animation curve.

[0225] The mobile phone 10 can generate, render, synthesize and display the first animation of movement of the first display element based on the first animation curve in response to the first operation after detecting the first operation.

[0226] In some embodiments, during the generation of the first animation, the mobile phone 10 draws and renders each frame image of the first animation with the same interval of animation time points regardless of whether the duration of drawing and rendering of each frame image of the first animation is greater than the first duration. That is, the interval of the time points (i.e., animation time points) corresponding to the positions of the first display element in adjacent two frame images in the plurality of frame images of the first animation on the first animation curve is equal. For details, reference can be made to the embodiments shown in FIG. 9 and FIG. 10, which will not be described herein.

[0227] In some embodiments, during the generation of the first animation, the mobile phone 10 can also perform frame interpolation on each frame image of the first animation based on the aforementioned frame interpolation condition. For details, reference can be made to the embodiments shown in FIG. 7 and FIG. 10, which will not be described herein.

[0228] Based on the above method, it can be ensured that the animation time points corresponding to each image frame of the first animation drawn and rendered by the mobile phone 10 are an arithmetic sequence, that is, the animation time points of any two adjacent frame images in the plurality of frame images of the first animation are equal. Based on the above method, when the mobile phone 10 displays the image frames of the first animation at a fixed refresh frequency, the position of the animation element in the displayed screen matches the first animation curve, which is beneficial to improving the user experience.

[0229] The embodiments of the present application also provide a program product, which, when executed on an electronic device, can enable the electronic device to implement the image processing method provided by the above embodiments.

[0230] The embodiments of the present application also provide a readable storage medium, which stores one or more programs / instructions, which, when executed by an electronic device, enable the electronic device to implement the image processing method provided by the above embodiments.

[0231] Further, FIG. 12 shows a structural schematic diagram of a mobile phone 10 according to some embodiments of the present application.

[0232] As shown in FIG. 12, the mobile phone 10 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0233] The processor 110 can include one or more processing units, for example: the processor 110 can include a central processing unit (CPU), an AP, a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a microcontroller unit (MCU), a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), a field programmable gate array (FPGA), etc. In some embodiments, different processing units can be independent devices, or can be integrated in one or more processors.

[0234] The processor 110 can also be provided with a memory for storing one or more programs and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold one or more programs or data that the processor 110 has just used or recycled. When the processor 110 needs to use the one or more programs or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thus improving the efficiency of the system.

[0235] In some embodiments, the processor 110 can be configured to execute one or more programs / instructions corresponding to the image processing method provided in the foregoing embodiments.

[0236] The charging management module 140 is configured to receive charging input from a charger. The charging management module 140 can also supply power to the electronic device through the power management module 141 while charging the battery 142.

[0237] The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160.

[0238] The wireless communication function of the mobile phone 10 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0239] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals.

[0240] The mobile communication module 150 can provide a solution for wireless communication, including 2G / 3G / 4G / 5G, etc., applied to the mobile phone 10. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter, amplify, etc., the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can also amplify signals modulated by the modem processor and convert the signals into electromagnetic waves radiated by the antenna 1. In some embodiments, at least part of the function modules of the mobile communication module 150 can be arranged in the processor 110. In some embodiments, at least part of the function modules of the mobile communication module 150 and at least part of the modules of the processor 110 can be arranged in the same device.

[0241] The wireless communication module 160 can provide solutions for wireless communication, including wireless local area networks (WLAN) (e.g., wireless fidelity network), Bluetooth (BT), global navigation satellite system (GNSS), near field communication (NFC), infrared (IR) technology, etc., which are applied on the mobile phone 10. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be sent from the processor 110, frequency-modulate them, amplify them, and convert them into electromagnetic wave radiation via the antenna 2.

[0242] The mobile phone 10 implements display functions through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0243] The display screen 194 is used to display images.

[0244] The camera 193 is used to capture still images or videos.

[0245] The external memory interface 120 can be used to connect an external memory card. The external memory card communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, application data of an application program is saved in the external memory card.

[0246] The internal memory 121 can be used to store one or more programs and corresponding data. The internal memory 121 can include a program storage area and a data storage area. In some embodiments, the program storage area can store an operating system, application programs required by at least one function, etc., such as the programs / instructions corresponding to the image processing method provided in the foregoing embodiments. The data storage area can store data created during use of the mobile phone 10. In addition, the internal memory 121 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage, etc. The processor 110 executes various function applications of the mobile phone 10 by running one or more programs stored in the internal memory 121 and / or one or more programs stored in a memory disposed in the processor 110.

[0247] The mobile phone 10 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, the application processor, and the like. For example, music playing, and the like.

[0248] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals.

[0249] The speaker 170A, also referred to as a “loudspeaker”, is configured to convert an audio electrical signal into a sound signal.

[0250] The receiver 170B, also referred to as a “earpiece”, is configured to convert an audio electrical signal into a sound signal.

[0251] The microphone 170C, also referred to as a “microphone”, “sound transducer”, is configured to convert a sound signal into an electrical signal.

[0252] The earphone interface 170D is configured to connect a wired earphone.

[0253] The pressure sensor 180A is configured to sense a pressure signal, and to convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. The pressure sensor 180A can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The mobile phone 10 determines the intensity of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the mobile phone 10 detects the intensity of the touch operation according to the pressure sensor 180A. The mobile phone 10 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A.

[0254] In some embodiments, the mobile phone 10 can detect a first operation of a user triggering a first animation according to the electrical signal detected by the pressure sensor 180A.

[0255] The keys 190 include a power-on key, a volume key, and the like. The keys 190 can be mechanical keys. Alternatively, the keys 190 can be touch keys. The mobile phone 10 can receive key inputs, and generate key signal inputs related to user settings and function control of the mobile phone 10.

[0256] The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, and can be configured to indicate a charging state, a power change, and the like, and can also be configured to indicate a message, a notification, and the like.

[0257] The SIM card interface 195 is configured to connect a SIM card or an eSIM.

[0258] It is to be understood that the structure of the mobile phone 10 shown in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0259] In the drawings, some structural or methodological features can be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order can not be required. Rather, in some embodiments, these features can be arranged in a different manner and / or order than shown in the illustrative drawings. In addition, inclusion of structural or methodological features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments these features can not be included or can be combined with other features.

[0260] It should be noted that the relational terms herein, such as first and second, and the like, are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises" or "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

Claims

1. An image processing method applied to an electronic device, comprising: The method comprises: ​ displaying a first interface, the first interface comprising a first display element; detecting a first operation of a user on the first interface; in response to the first operation, generating and displaying a first animation of movement of the first display element based on an animation curve, wherein the first animation comprises a plurality of consecutive images, and positions of the first display element in adjacent two images of the plurality of consecutive images correspond to equal intervals of time at corresponding moments on the animation curve, and the plurality of consecutive images comprises a first image, and a drawing and rendering time length of the first image is greater than a first time length, the first time length being an inverse of a refresh frequency of a display screen of the electronic device.

2. The method of claim 1, wherein, The first operation comprises: an operation of pressing by the user at a first position in the first interface and lifting after sliding to a second position, wherein: a speed of sliding by the user at the second position is greater than a first speed, or an acceleration of sliding by the user at the second position is greater than a first acceleration.

3. The method of claim 1, wherein, The generating and displaying of the first animation of movement of the first display element based on the animation curve comprises: detecting a first vertical synchronization signal, and drawing and rendering a second image, wherein the second image is any one of the last image in the plurality of consecutive images; in a case where a second image drawing and rendering completion meets an interpolation condition, drawing and rendering a third image before detecting a second vertical synchronization signal, wherein the second vertical synchronization signal is a first vertical synchronization signal after the second image drawing and rendering completion, and the third image is a subsequent image of the second image in the plurality of consecutive images; in a case where the second image drawing and rendering completion does not meet the interpolation condition, drawing and rendering the third image after detecting the second vertical synchronization signal.

4. The method of claim 3, wherein, The interpolation condition comprises any one of the following conditions: a time length between a time of the second image drawing and rendering completion and a time of detecting the first operation is less than a second time length; a number of free buffers in a buffer queue used by an application program corresponding to the first display element to draw and render images is greater than a buffer threshold.

5. The method of claim 3, wherein, The drawing and rendering of the second image comprises: obtaining a first position corresponding to a first time in the animation curve, and drawing and rendering the first display element in the second image based on the first position, wherein: in a case where the second image is a first image in the plurality of consecutive images, the first time is any time in the animation curve, in a case where the second image is not the first image in the plurality of consecutive images, the first time is a sum of a second time corresponding to a position of the first display element in a previous image of the first image in the animation curve and the first time length.

6. The method of claim 5, wherein, The drawing and rendering of the third image comprises: obtaining a second position corresponding to a third time in the animation curve, and drawing and rendering the first display element in the third image based on the second position, wherein the third time is a sum of the first time and the first time length.

7. The method of claim 4, wherein, The method further comprises: adjust the buffer threshold based on a historical state of the buffers in the buffer queue.

8. The method of claim 7, wherein, The adjusting the buffer threshold based on the historical state of the buffers in the buffer queue comprises: decrease the buffer threshold in a case that the buffer queue is not occupied in a first number of vertical synchronization signal periods, or a ratio of a number of vertical synchronization signal periods in which the buffer queue is occupied in the first number of vertical synchronization signal periods to the first number is less than a first ratio, or the buffer queue is not occupied in a third time length, or a ratio of a time length in which the buffer queue is occupied in the third time length to the third time length is less than a second ratio; increase the buffer threshold in a case that the buffer queue is occupied in a second number of vertical synchronization signal periods, or a ratio of a number of vertical synchronization signal periods in which the buffer queue is occupied in the second number of vertical synchronization signal periods to the second number is greater than a third ratio, or the buffer queue is occupied in a fourth time length, or a ratio of a time length in which the buffer queue is occupied in the fourth time length to the fourth time length is greater than a fourth ratio.

9. An electronic device, comprising: comprise: a memory configured to store one or more programs; a processor configured to execute the one or more programs to cause the electronic device to implement the image processing method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises one or more programs, which when executed on an electronic device, cause the electronic device to implement the image processing method of any one of claims 1-8.

11. A computer program product, characterised in that, The computer program product, when executed on an electronic device, causes the electronic device to implement the image processing method of any one of claims 1-8.

Images