Display method, device, electronic equipment, storage medium and program product

Abstract

The present disclosure relates to a display method, device, electronic equipment, storage medium and program product. The method comprises: in response to a trigger interface switching operation, if a first picture is drawn, performing interpolation based on first picture data to obtain interpolation picture data, and drawing an interpolation picture based on the interpolation picture data, the first picture data representing the picture content of the first picture, and the first picture being an original picture associated with the interface switching operation; and displaying based on the drawn pictures, the drawn pictures including the drawn original picture and the drawn interpolation picture. After the interface switching operation is triggered, the interpolation operation is started to ensure that there are more drawn pictures available for display, thereby avoiding the occurrence of lag, ensuring the smoothness of the interface switching process, and improving the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of display technology, and in particular to a display method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] The smoothness of interface transitions directly impacts the user experience. Triple buffering technology is used to ensure smooth interface transitions. However, as application loads increase, in high-load scenarios such as those involving audio, video, and images, abnormal time consumption occurs during screen rendering, leading to severe stuttering and a poor user experience. Summary of the Invention

[0003] To overcome the problems existing in the related technologies, this disclosure provides a display method, apparatus, electronic device, storage medium, and program product.

[0004] According to a first aspect of the present disclosure, a display method is provided, comprising:

[0005] In response to the detection of a swipe operation and in response to the triggering of an interface switching operation, if the first screen is completed, frame interpolation is performed based on the first screen data to obtain interpolated frame screen data, and the interpolated frame screen is drawn based on the interpolated frame screen data. The first screen data is used to characterize the screen content of the first screen, and the first screen is the original screen associated with the interface switching operation.

[0006] The display is based on the completed image, which includes the original image and the interpolated image.

[0007] In some embodiments, if the first frame is completed, frame interpolation is performed based on the first frame data to obtain interpolated frame data, including:

[0008] If the first frame is completed and the first frame interpolation condition is met, frame interpolation is performed based on the first frame data to obtain the interpolated frame data;

[0009] The first frame interpolation condition refers to the current available capacity of the buffer satisfying the non-blocking rendering of the interpolated frame and the second frame, where the second frame is the original frame following the first frame.

[0010] In some embodiments, if the first frame is completed and the first frame interpolation condition is met, performing frame interpolation based on the first frame data to obtain the interpolated frame data includes:

[0011] If the first frame is completed and the first and second frame interpolation conditions are met, frame interpolation is performed based on the first frame data to obtain the interpolated frame data.

[0012] The second frame interpolation condition refers to the time interval between the current moment and the start time of the second frame being drawn being greater than the first threshold.

[0013] In some embodiments, the method further includes:

[0014] If the first frame is completed but the first frame interpolation condition and / or the second frame interpolation condition are not met, the second frame is drawn based on the second frame data, which is used to characterize the content of the second frame.

[0015] In some embodiments, the method further includes:

[0016] If the interpolated frame is completed, a second frame is drawn based on the second frame data. The second frame data is used to characterize the content of the second frame, and the second frame is the next frame of the original frame of the first frame.

[0017] In some embodiments, the step of interpolating frames based on the first frame data to obtain interpolated frame data includes:

[0018] Frame interpolation is performed based on the first image data and the interface switching speed to obtain the interpolated image data.

[0019] In some embodiments, the method further includes:

[0020] In response to the detection of frame dropping, the upper limit of the buffer capacity is increased. Frame dropping refers to the failure to draw a new frame within a preset time period.

[0021] Based on the increased capacity limit and the third frame data, a third frame is drawn. The third frame data is used to characterize the content of the third frame. The third frame is the next frame after the latest drawn frame.

[0022] In some embodiments, the method further includes:

[0023] In response to the launch operation of the target application, the maximum capacity of the buffer is read from the configuration information of the target application, and the buffer is configured based on the maximum capacity of the buffer.

[0024] In some embodiments, the display based on the completed drawing of the image includes:

[0025] Based on the time when each frame is completed and a first signal, the completed frames are displayed sequentially, whereby the first signal indicates the time when each frame is displayed.

[0026] In some embodiments, the interface switching operation includes an inertial swipe operation, which refers to a swipe operation triggered after a touch swipe operation has stopped.

[0027] According to a second aspect of the present disclosure, a display device is provided, comprising:

[0028] The frame interpolation module is configured to respond to a trigger interface switching operation. If the first screen is completed, it performs frame interpolation based on the first screen data to obtain interpolated frame screen data, and draws the interpolated frame screen based on the interpolated frame screen data. The first screen data is used to characterize the screen content of the first screen, and the first screen is the original screen associated with the interface switching operation.

[0029] The display module is configured to display based on a completed drawing screen, which includes the original drawing screen and the interpolated drawing screen.

[0030] In some embodiments, the frame interpolation module is configured to perform frame interpolation based on the first frame data if the first frame is completed and the first frame interpolation condition is met, to obtain the interpolated frame data.

[0031] The first frame interpolation condition refers to the current available capacity of the buffer satisfying the non-blocking rendering of the interpolated frame and the second frame, where the second frame is the original frame following the first frame.

[0032] In some embodiments, the frame interpolation module is configured to perform frame interpolation based on the first frame data if the first frame is completed and the first frame interpolation condition and the second frame interpolation condition are met, thereby obtaining the interpolated frame data.

[0033] The second frame interpolation condition refers to the time interval between the current moment and the start time of the second frame being drawn being greater than the first threshold.

[0034] In some embodiments, the apparatus further includes:

[0035] The capacity limit adjustment module is configured to increase the capacity limit of the buffer in response to the detection of frame dropping, wherein frame dropping refers to the failure to draw new frames within a preset time period.

[0036] The drawing module is configured to draw a third frame based on the increased capacity limit and the third frame data, wherein the third frame data is used to characterize the content of the third frame, and the third frame is the next frame after the latest drawn frame.

[0037] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0038] processor;

[0039] Memory used to store processor-executable instructions;

[0040] The processor is configured to perform the display method as described in the first aspect of the embodiments of this disclosure.

[0041] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the display method as described in the first aspect of the present disclosure.

[0042] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the display method as described in the first aspect of the present disclosure.

[0043] The method described above, as disclosed in this invention, has the following beneficial effects:

[0044] The method provided in this embodiment starts frame interpolation after triggering the interface switching operation to ensure that more drawn frames are available for display. Even if drawing frames takes a long time, there are still drawn frames available for display, thereby avoiding lag, ensuring the smoothness of the interface switching process, and improving the user experience.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0047] Figure 1 This is a flowchart illustrating a display method according to an exemplary embodiment.

[0048] Figure 2 This is a flowchart illustrating a display method according to an exemplary embodiment.

[0049] Figure 3 This is a schematic diagram illustrating a display process according to an exemplary embodiment.

[0050] Figure 4This is a flowchart illustrating a display method according to an exemplary embodiment.

[0051] Figure 5 This is a schematic diagram illustrating a smoothness comparison according to an exemplary embodiment.

[0052] Figure 6 This is a schematic diagram illustrating a smoothness comparison according to an exemplary embodiment.

[0053] Figure 7 This is a block diagram illustrating a display device according to an exemplary embodiment.

[0054] Figure 8 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0055] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0056] In related technologies, Triple Buffer technology is used to ensure the smoothness of interface switching. During screen display, the CPU (Central Processing Unit) is responsible for generating the screen, or generating screen data, while the GPU (Graphics Processing Unit) is responsible for drawing the screen and then displaying the drawn screen on the screen. Triple buffer technology sets up three buffers between the CPU, GPU, and screen to store the screen to be processed, the screen being drawn, and the screen that has been drawn and is ready to be displayed, respectively. These three buffers allow the CPU, GPU, and screen to work in parallel on different buffers, thereby improving overall processing efficiency and smoothness. However, as application loads increase, in high-load interface switching scenarios such as those involving audio, video, and images, more abnormal time-consuming situations occur during the screen drawing process, resulting in severe stuttering and a poor user experience.

[0057] To address the lag issue during interface switching, this disclosure provides a display method. In response to a triggered interface switching operation, if the first screen is completed, frame interpolation is performed based on the first screen data to obtain interpolated frame data. The interpolated frame data is then used to draw the interpolated frame data. The first screen data represents the content of the first screen, which is the original screen associated with the interface switching operation. The method displays the completed screen, which includes both the completed original screen and the completed interpolated frame data. Frame interpolation begins immediately after the interface switching operation is triggered to ensure that more completed and displayable screens are available. Even if screen drawing is time-consuming, completed screens can still be displayed, thus avoiding lag, ensuring smooth interface switching, and improving the user experience.

[0058] The display method provided in this disclosure is executed by an electronic device, which may specifically be a mobile phone, tablet computer, laptop computer, smart robot, smart wearable device, smart home device, vehicle terminal, etc. In addition, the electronic device also has various hardware resources and energy storage devices that provide power for the operation of these hardware resources.

[0059] Figure 1 This is a flowchart illustrating a display method according to an exemplary embodiment, performed by an electronic device. See also... Figure 1 The method includes the following steps:

[0060] In step S101, in response to the triggering of the interface switching operation, if the first screen is completed, frame interpolation is performed based on the first screen data to obtain interpolated frame screen data, and the interpolated frame screen is drawn based on the interpolated frame screen data. The first screen data is used to represent the screen content of the first screen, and the first screen is the original screen associated with the interface switching operation.

[0061] The interface switching operation can be triggered by the user on any display interface, or it can be a swipe operation by the user on the display interface of the target application. For example, the target application can be a video application, an instant messaging application, an audio playback application, or other applications that have a high load when switching interfaces.

[0062] In some embodiments, the interface switching operation can be triggered by actions such as swiping, clicking, or timed page turning. For example, on the homepage of a video application, in response to detecting a user's swipe action, an interface switching operation is determined to be triggered, thereby initiating the subsequent display process.

[0063] The first screen is the original screen associated with the interface switching operation. For example, the first screen can be the first original screen after the interface switching is triggered, the second original screen, or other original screens during the interface switching process. The original screen refers to the screen that was originally displayed when no frame interpolation was performed during the interface switching process. This original screen is drawn based on the acquired screen data, which can be obtained from a backend server, from the local cache of the electronic device, or through other means. The first screen can be drawn based on first screen data, which is used to represent the content of the first screen.

[0064] After the first frame is drawn, frame interpolation can be performed based on the first frame data to obtain interpolated frame data. This interpolated frame data is used to represent the content of the interpolated frame. Then, the interpolated frame can be drawn based on this interpolated frame data. This interpolated frame is the frame following the first frame. This embodiment of the disclosure does not limit the content of the original frame and the interpolated frame.

[0065] Optionally, if one interpolated frame is inserted between the first frame and the second frame, the interpolated frame drawn based on the interpolated frame data is the next frame of the first frame. If multiple interpolated frames are inserted between the first frame and the second frame, the interpolated frame drawn based on the interpolated frame data can be any interpolated frame between the first frame and the second frame. The second frame is the next original frame of the first frame.

[0066] It should be noted that, in this embodiment of the present disclosure, for each original frame, whenever an original frame is drawn, frame interpolation is performed based on the image data of the latest drawn original frame.

[0067] In some embodiments, a first frame is drawn based on a second signal and first frame data; that is, each original frame is drawn based on the second signal and the frame data of each original frame. The second signal indicates the time at which drawing each original frame begins, and this second signal can be a periodic signal. For example, the second signal is a Vsync (Vertical Synchronization) signal.

[0068] In some embodiments, frame interpolation algorithms such as motion compensation algorithms, optical flow methods, and deep learning models can be used to interpolate frames based on the first frame data to obtain interpolated frame data. This disclosure does not limit the frame interpolation algorithm used in the frame interpolation process.

[0069] It should be noted that the number of interpolated frames between two adjacent original frames can be one or more, and the number of interpolated frames can be set according to the actual display needs. For example, one interpolated frame can be inserted between two adjacent original frames.

[0070] Step S102: Display the completed image. The completed image includes the original image and the interpolated image.

[0071] During the display process, the completed frames are displayed sequentially based on the moment each frame is finished. For example, if an interpolated frame is inserted between the first and second frames, the first frame, the interpolated frame, and the second frame are displayed sequentially.

[0072] It should be noted that the frame interpolation process and the display are executed in parallel. While the image is being drawn, the completed image can be displayed. For example, if the original image 1, the interpolated image 1, and the original image 2 have been drawn, the original image 1 can be displayed while frame interpolation is being performed based on the image data of the original image 2 and the image data of the next frame, the original image 3.

[0073] The method provided in this embodiment starts frame interpolation after triggering the interface switching operation to ensure that more drawn frames are available for display. Even if drawing frames takes a long time, there are still drawn frames available for display, thereby avoiding lag, ensuring the smoothness of the interface switching process, and improving the user experience.

[0074] Figure 2 This is a flowchart illustrating a display method according to an exemplary embodiment, performed by an electronic device. See also... Figure 2 The method includes the following steps:

[0075] In step S201, in response to the startup operation of the target application, the maximum capacity of the buffer is read from the configuration information of the target application, and the buffer is configured based on the maximum capacity of the buffer.

[0076] To facilitate frame interpolation during subsequent display, the maximum capacity of the buffer can be read from the target application's configuration information upon startup, and the buffer can be configured based on this maximum capacity. The maximum buffer capacity can correspond to the load of the target application; if the target application has a high load, the maximum buffer capacity can be large, and if the target application has a low load, the maximum buffer capacity can be small.

[0077] The configuration information of the target application can be the target application's configuration file, which is used to configure the target application. In addition to the maximum capacity of the buffer, the configuration file can also include other information such as the target application's running parameters, log configuration, and function configuration.

[0078] In step S202, in response to the trigger interface switching operation, the first screen is drawn based on the second signal and the first screen data.

[0079] In some embodiments, a screen switching operation is triggered in response to the detection of an inertial swipe operation. That is, the screen switching operation includes an inertial swipe operation. In this case, the first screen can be a screen associated with the inertial swipe operation. For example, the first screen can be the first original frame after the inertial swipe operation is detected, the second original frame, or other original frames during the inertial swipe process.

[0080] Inertial swipe operation refers to a swipe operation triggered after a touch swipe operation has stopped. Optionally, a touch swipe operation refers to the operation of a user's finger (or stylus) touching the display interface and swiping within it, while an inertial swipe operation refers to the operation where, after the user's finger (or stylus) leaves the display interface, the elements on the display interface continue to maintain a certain swiping state due to the speed and inertia accumulated during the previous touch swipe operation.

[0081] Optionally, the touch-swipe operation process can be referred to as the touch-swipe phase, and the inertial swipe operation process can be referred to as the inertial swipe phase. See one example. Figure 3 The diagram shown illustrates the display process. No frame interpolation occurs during the DeliverInput stage (touch swipe stage), but frame interpolation begins after the Fling stage (inertial swipe stage) starts.

[0082] In some embodiments, in response to the detection of a click operation, an interface switching operation is triggered. For example, in a novel reading interface, the user clicks the screen to trigger a page-turning operation (i.e., an interface switching operation).

[0083] In some embodiments, a first screen is drawn based on a second signal and first screen data using a preset thread. The preset thread is used for screen drawing and can be a UI (User Interface) thread.

[0084] It should be noted that after the interface switching operation is triggered, each original frame is drawn based on the second signal. This embodiment only uses the first frame as an example for illustration.

[0085] Step S203: If the first frame is completed and the first frame interpolation condition is met, frame interpolation is performed based on the first frame data to obtain interpolated frame data, and the interpolated frame is drawn based on the interpolated frame data.

[0086] The first frame interpolation condition refers to the current available capacity of the buffer being sufficient for non-blocking rendering of the interpolated frame and the second frame, where the second frame is the next original frame after the first frame. In other words, the first frame interpolation condition means that the current available capacity of the buffer is sufficient for non-blocking rendering of the interpolated frame to be inserted after the current original frame and the next original frame after the current original frame. Non-blocking rendering means that there is sufficient available capacity to render the interpolated frame to be inserted and the next original frame after the current original frame, and there will be no problem of insufficient available capacity during the rendering process.

[0087] Meeting the first frame interpolation condition means that the current available capacity of the buffer can support the drawing of the interpolated frame and the second frame. Therefore, performing frame interpolation under the condition of meeting the first frame interpolation condition can avoid the problem of insufficient available capacity during the drawing process, ensure that the current interpolated frame and the next original frame can be drawn normally, and avoid stuttering during the drawing process.

[0088] In some embodiments, if the first frame is completed and the first and second interpolation conditions are met, interpolation is performed based on the first frame data to obtain interpolated frame data. The second interpolation condition refers to the interval between the current time and the start time of the second frame being drawn being greater than a first threshold. Specifically, the second interpolation condition refers to the interval between the current time and the start time of the next frame of the original frame being drawn being greater than the first threshold. The first threshold is a preset threshold, for example, 2ms or other values.

[0089] Optionally, the first threshold is related to the refresh rate of the electronic device; the higher the refresh rate, the lower the first threshold, and vice versa. For example, the display duration of each frame is determined based on the refresh rate of the electronic device, and the first threshold is less than that display duration.

[0090] Meeting the second frame interpolation condition means that there is still a considerable amount of time between the completion of the first frame drawing and the start of the second frame drawing. There is sufficient time for frame interpolation and the drawing of the interpolated frames before the second frame begins to be drawn. Therefore, performing the operation under the condition of meeting the first frame interpolation condition can avoid the problem of delaying the drawing of the next original frame before the interpolated frames have been drawn. This ensures that even if frame interpolation is performed, the next original frame can be drawn when the start of the next original frame drawing time is reached.

[0091] In some embodiments, frame interpolation based on first screen data to obtain interpolated frame data includes: frame interpolation based on the first screen data and the interface switching speed to obtain interpolated frame data. The interface switching speed characterizes the sliding speed of the screen content in the first screen during interface switching. A faster interface switching speed results in a greater change in the position of the screen content in the first screen on the display interface within a fixed time period; conversely, a slower interface switching speed results in a smaller change in the position of the screen content in the first screen on the display interface within a fixed time period. Therefore, based on the first screen data and the interface switching speed, the position of the screen content in the first screen after moving on the display interface according to the interface switching speed and then in the interpolated frame to be inserted can be predicted, thus obtaining the interpolated frame data.

[0092] Optionally, frame interpolation is performed based on the first frame data, the interface switching speed, and the first duration to obtain interpolated frame data. The first duration represents the time interval between the interpolated frame to be inserted and the first frame. Based on the interface switching speed and the first duration, the movement of the content in the first frame within the display interface during the first duration can be predicted, thereby determining the position of the content in the first frame within the interpolated frame to be inserted, thus obtaining the interpolated frame data.

[0093] Optionally, the interface switching direction can also be considered, that is, frame interpolation can be performed based on the first screen data, the interface switching speed, the first duration, and the interface switching direction to obtain interpolated frame data. For example, when the interface switching direction is to swipe upwards, the content in the first screen moves upwards; when the interface switching direction is to swipe downwards, the content in the first screen moves downwards.

[0094] In some embodiments, frame interpolation based on first frame data to obtain interpolated frame data includes: frame interpolation based on first frame data and second frame data to obtain interpolated frame data. That is, based on the frame data of the currently rendered original frame and the frame data of the next frame of the currently rendered original frame, the frame data of the interpolated frame to be displayed between the two original frames is predicted.

[0095] It should be noted that step S203 above applies to cases where the first and second interpolation conditions are met. If the first frame is drawn but the first and / or second interpolation conditions are not met, the second frame is drawn directly based on the second frame data. That is, if the first and / or second interpolation conditions are not met, no interpolation operation is performed. Optionally, the second frame is drawn based on the second signal and the second frame data, whereby the second frame data is used to represent the content of the second frame.

[0096] See one example. Figure 3With a buffer capacity limit of 5, when the occupied capacity is less than 5, interpolated frame 21 can be inserted after the original frame 11 is drawn, and interpolated frame 22 can be inserted after the original frame 12 is drawn. However, when the occupied capacity reaches 5, no interpolated frame is inserted after the original frame 13 is drawn; instead, the next original frame 14 is drawn directly. Furthermore, from... Figure 3 It can be seen that the start of the rendering of each original frame corresponds to a rising or falling edge of the second signal.

[0097] Another point to note is that after the interpolated frame is drawn, the second frame will continue to be drawn based on the second frame data. Then, after the second frame is drawn, an operation similar to step S203 will be performed, that is, frame interpolation will be performed after the second frame.

[0098] Step S204: In response to the detection of frame dropping, increase the upper limit of the buffer capacity.

[0099] Among them, frame dropping refers to the failure to render new frames within a preset time period.

[0100] In some embodiments, if no new frame is drawn at at least one time indicated by the second signal when the original frame is drawn, it is determined that a frame drop has occurred.

[0101] In some embodiments, if the available capacity of the buffer reaches its maximum capacity, it is determined that a frame drop has occurred.

[0102] See one example. Figure 3 Frame drops occurred when drawing the original frame 14. At this time, the maximum capacity of the buffer (Increase maxBufferCount) was increased to 6.

[0103] In addition, from Figure 3 As can be seen, since a large number of frames have already been drawn, even when frame drops occur, the already drawn frames can still be displayed. The frame drops during drawing do not affect the display and can still ensure the smoothness of the scrolling process.

[0104] In some embodiments, a preset capacity is set, and increasing the upper limit of the buffer capacity includes: adding the preset capacity to the upper limit of the buffer capacity. Of course, the degree to which the upper limit of the buffer capacity is increased can also be determined according to whether the current frame loss is severe. This disclosure does not limit the method of increasing the upper limit of the buffer capacity.

[0105] It should be noted that, in some embodiments, in response to the detection of a sliding operation, the upper limit of the buffer capacity can be increased before frame interpolation begins, so that more capacity is available during the frame interpolation process.

[0106] Step S205: Based on the increased capacity limit and the third screen data, draw the third screen.

[0107] The third frame is the next frame after the latest completed frame, and the third frame data is used to represent the content of the third frame.

[0108] In some embodiments, the third frame can be the original frame or an interpolated frame. If the third frame is an interpolated frame, the interpolation process of the third frame is similar to the interpolation process of the interpolated frame in the above embodiments, and will not be described again here.

[0109] It should be noted that if the third frame is the original frame, then after the third frame is drawn, an operation similar to step S203 above will continue to be performed. If the third frame is an interpolated frame, then after the third frame is drawn, an operation similar to step S202 above will continue to be performed.

[0110] See one example. Figure 3 After increasing the buffer's capacity limit to 6, the next frame (23) is drawn, and then the process of drawing the original frame and inserting frames is repeated until the occupied capacity reaches 6, at which point the buffer's capacity limit can be increased again. Figure 3 The subsequent process is not explained in the text.

[0111] It should be noted that steps S204 and S205 are optional. If no frame loss occurs, steps S204 and S205 do not need to be executed.

[0112] Step S206: Based on the time when each frame is completed and the first signal, the completed frames are displayed sequentially.

[0113] The first signal is used to indicate the time when each frame is displayed, and each rising and falling edge of the first signal indicates the time when one frame is displayed. In one example, from Figure 3 It can be seen that the display time of each completed frame corresponds to a rising or falling edge of the first signal. Furthermore, from... Figure 3 It can also be seen that when frame drops occur during the rendering process, since multiple frames have already been rendered, the rendered frames are displayed sequentially according to the first signal, and the display can still be displayed normally without any stuttering.

[0114] It should be noted that step S206 is executed in parallel with steps S202-S205 above. During the frame interpolation and drawing process, the completed drawing screen will be displayed.

[0115] Another point to note is that, in response to the completion of the interface switching or exiting the interface switching process, the frame interpolation operation and the operation of increasing the upper limit of the buffer capacity can be stopped.

[0116] The method provided in this embodiment begins frame interpolation after triggering an interface switching operation. This ensures that more frames are drawn and available for display, even if drawing frames takes a long time, as long as some frames are already drawn, thus avoiding stuttering, ensuring smooth interface switching, and improving the user experience. Furthermore, when frame drops occur, the upper limit of the buffer capacity can be dynamically adjusted to increase the number of drawn frames available for display, thereby enhancing anti-stuttering capabilities.

[0117] Figure 4 This is a flowchart illustrating a display method according to an exemplary embodiment, performed by an electronic device. See also... Figure 4 The method includes the following steps:

[0118] In step S401, in response to the startup operation of the target application, the maximum capacity of the buffer is read from the configuration information of the target application, and the buffer is configured based on the maximum capacity of the buffer.

[0119] Step S402: Enter the inertial sliding stage.

[0120] Step S403: Based on the second signal and the image data of the original image to be drawn, draw the original image to be drawn.

[0121] In step S404, in response to the completion of the original frame drawing, it is determined whether the first frame interpolation condition and the second frame interpolation condition are met. If the first frame interpolation condition and the second frame interpolation condition are met, step S405 is executed. If the first frame interpolation condition and / or the second frame interpolation condition are not met, step S406 is executed.

[0122] Step S405: Perform frame interpolation based on the image data of the latest drawn original image to obtain interpolated frame image data, and draw the interpolated frame image based on the interpolated frame image data, and then execute step S406.

[0123] Step S406: Based on the second signal and the image data of the next frame of the latest drawn original image, draw the next frame of the latest drawn original image, and then execute step S407.

[0124] Step S407: Detect whether frame dropping occurs. If frame dropping occurs, proceed to step S408. If frame dropping does not occur, proceed to step S404.

[0125] Step S408: Increase the upper limit of the buffer capacity and proceed to step S404.

[0126] See one example. Figure 5 The smoothness comparison diagram shown shows that the first electronic device uses... Figure 4 During the display process shown, the smoothness of applications 1, 2, 3, 4, and 5 installed on the first electronic device was tested. The smoothness scores of these five applications were 88.22, 98.54, 86.92, 99.39, and 98.02, respectively. When the first electronic device uses a method that interpolates frames only when stuttering occurs, and does not consider the buffer capacity limit during frame interpolation, the smoothness of applications 1, 2, 3, 4, and 5 was tested again. The smoothness scores of these five applications were 86.35, 97.3, 83.02, 98.22, and 78.98, respectively. From... Figure 5 It can be seen that compared to the method of only interpolating frames when stuttering occurs, and not considering the maximum capacity of the buffer during the interpolation process, Figure 4 The method shown in the embodiment can better improve smoothness.

[0127] In another example, see Figure 6 The smoothness comparison diagram shown indicates that the second electronic device uses... Figure 4 During the display process shown, the smoothness of applications 1, 2, 3, 4, and 5 installed on the first electronic device was tested. The smoothness scores for these five applications were 82.85, 99.28, 70.66, 93.96, and 76.49, respectively. During the display process on the second electronic device, which interpolates frames only when stuttering occurs and does not dynamically adjust the upper limit of the buffer capacity during frame interpolation, the smoothness of applications 1, 2, 3, 4, and 5 was tested. The smoothness scores for these five applications were 76.67, 98.5, 54.55, 90.38, and 52.33, respectively. Figure 6 It can be seen that compared to the method of only interpolating frames when stuttering occurs, and not dynamically adjusting the upper limit of the buffer capacity during the interpolation process, Figure 4 The method shown in the embodiment can better improve smoothness.

[0128] Therefore, in application interface switching scenarios, frame interpolation is triggered when the inertial sliding phase begins. This frame interpolation process is drawn at intervals with the original screen. At the same time, the buffer is configured based on the upper limit of the buffer capacity in the configuration information, which can ensure that there are enough drawn screens for display. Furthermore, if a relatively serious frame drop phenomenon still occurs, the upper limit of the buffer capacity can be dynamically increased, thereby greatly improving the anti-lag capability and improving the user experience.

[0129] Figure 7 This is a block diagram illustrating a display device according to an exemplary embodiment, configured in an electronic device. See also... Figure 7 The device includes:

[0130] The frame interpolation module 701 is configured to respond to a trigger interface switching operation. If the first screen is completed, it performs frame interpolation based on the first screen data to obtain interpolated frame data, and draws the interpolated frame based on the interpolated frame data. The first screen data is used to represent the screen content of the first screen, and the first screen is the original screen associated with the interface switching operation.

[0131] The display module 702 is configured to display based on a completed drawing screen, which includes the original drawing screen and the interpolated drawing screen.

[0132] In some embodiments, the frame interpolation module 701 is configured to:

[0133] If the first frame is completed and the first frame interpolation condition is met, frame interpolation is performed based on the first frame data to obtain the interpolated frame data;

[0134] The first interpolation condition refers to the current available capacity of the buffer being sufficient for non-blocking rendering of the interpolated frame and the second frame, where the second frame is the original frame following the first frame.

[0135] In some embodiments, the frame interpolation module 701 is configured to perform frame interpolation based on the first frame data if the first frame is completed and the first frame interpolation condition and the second frame interpolation condition are met, so as to obtain frame interpolation frame data.

[0136] The second frame interpolation condition refers to the time interval between the current moment and the start time of the second frame being drawn being greater than the first threshold.

[0137] In some embodiments, the apparatus further includes:

[0138] The drawing module is configured to draw a second screen based on the second screen data if the first screen is completed but the first interpolation condition and / or the second interpolation condition are not met. The second screen data is used to represent the screen content of the second screen.

[0139] In some embodiments, the apparatus further includes:

[0140] The drawing module is configured to draw a second frame based on the second frame data after the interpolated frame is drawn. The second frame data is used to represent the content of the second frame, which is the next frame of the original frame of the first frame.

[0141] In some embodiments, the frame interpolation module 701 is configured to perform frame interpolation based on the first screen data and the interface switching speed to obtain interpolated screen data.

[0142] In some embodiments, the apparatus further includes:

[0143] The capacity limit adjustment module is configured to increase the capacity limit of the buffer in response to the detection of frame dropping. Frame dropping refers to the failure to draw new frames within a preset time period.

[0144] The drawing module is configured to draw the third frame based on the increased capacity limit and the third frame data. The third frame data is used to represent the content of the third frame, which is the next frame after the latest drawn frame.

[0145] In some embodiments, the apparatus further includes:

[0146] The configuration module is configured to read the maximum capacity of the buffer from the target application's configuration information in response to the target application's startup operation, and configure the buffer based on the maximum capacity of the buffer.

[0147] In some embodiments, the display module 702 is configured to:

[0148] Based on the time when each frame is completed and the first signal, the completed frames are displayed sequentially. The first signal is used to indicate the time when each frame is displayed.

[0149] In some embodiments, the interface switching operation includes an inertial swipe operation, which is a swipe operation triggered after a touch swipe operation has stopped.

[0150] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0151] This disclosure also provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the display method described in the above embodiments.

[0152] Figure 8 This is a block diagram of an electronic device 800 according to an exemplary embodiment.

[0153] Reference Figure 8 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0154] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0155] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0156] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0157] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0158] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0159] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0160] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0161] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0162] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0163] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0164] This disclosure also provides a non-transitory computer-readable storage medium, wherein when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the display method described in the above embodiments.

[0165] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the display method described in the above embodiments.

[0166] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0167] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A display method, characterized in that, include: In response to the triggering of the interface switching operation, if the first screen is completed, frame interpolation is performed based on the first screen data to obtain interpolated frame screen data, and the interpolated frame screen is drawn based on the interpolated frame screen data. The first screen data is used to represent the screen content of the first screen, and the first screen is the original screen associated with the interface switching operation. The display is based on the completed image, which includes the original image and the interpolated image.

2. The display method according to claim 1, characterized in that, If the first frame is completed, frame interpolation is performed based on the first frame data to obtain interpolated frame data, including: If the first frame is completed and the first frame interpolation condition is met, frame interpolation is performed based on the first frame data to obtain the interpolated frame data; The first frame interpolation condition refers to the current available capacity of the buffer satisfying the non-blocking rendering of the interpolated frame and the second frame, where the second frame is the original frame following the first frame.

3. The display method according to claim 2, characterized in that, If the first frame is completed and the first frame interpolation condition is met, frame interpolation is performed based on the first frame data to obtain the interpolated frame data, including: If the first frame is completed and the first and second frame interpolation conditions are met, frame interpolation is performed based on the first frame data to obtain the interpolated frame data. The second frame interpolation condition refers to the time interval between the current moment and the start time of the second frame being drawn being greater than the first threshold.

4. The display method according to claim 3, characterized in that, The method further includes: If the first frame is completed but the first frame interpolation condition and / or the second frame interpolation condition are not met, the second frame is drawn based on the second frame data, which is used to characterize the content of the second frame.

5. The display method according to claim 1, characterized in that, The method further includes: If the interpolated frame is completed, a second frame is drawn based on the second frame data. The second frame data is used to characterize the content of the second frame, and the second frame is the next frame of the original frame of the first frame.

6. The display method according to claim 1, characterized in that, The process of interpolating frames based on the first frame data to obtain interpolated frame data includes: Frame interpolation is performed based on the first frame data and the interface switching speed to obtain the interpolated frame data.

7. The display method according to claim 1, characterized in that, The method further includes: In response to the detection of frame dropping, the upper limit of the buffer capacity is increased. Frame dropping refers to the failure to draw a new frame within a preset time period. Based on the increased capacity limit and the third frame data, a third frame is drawn. The third frame data is used to characterize the content of the third frame. The third frame is the next frame after the latest drawn frame.

8. The display method according to claim 7, characterized in that, The method further includes: In response to the launch operation of the target application, the maximum capacity of the buffer is read from the configuration information of the target application, and the buffer is configured based on the maximum capacity of the buffer.

9. The display method according to claim 1, characterized in that, The display based on the completed drawing includes: Based on the time when each frame is completed and a first signal, the completed frames are displayed sequentially, whereby the first signal indicates the time when each frame is displayed.

10. The display method according to any one of claims 1 to 9, characterized in that, The interface switching operation includes an inertial swipe operation, which is a swipe operation triggered after the touch swipe operation stops.

11. A display device, characterized in that, include: The frame interpolation module is configured to respond to a trigger interface switching operation. If the first screen is completed, it performs frame interpolation based on the first screen data to obtain interpolated frame screen data, and draws the interpolated frame screen based on the interpolated frame screen data. The first screen data is used to characterize the screen content of the first screen, and the first screen is the original screen associated with the interface switching operation. The display module is configured to display based on a completed drawing screen, which includes the original drawing screen and the interpolated drawing screen.

12. The display device according to claim 11, characterized in that, The frame interpolation module is configured to perform frame interpolation based on the first frame data if the first frame is completed and the first frame interpolation condition is met, so as to obtain the interpolated frame data. The first frame interpolation condition refers to the current available capacity of the buffer satisfying the non-blocking rendering of the interpolated frame and the second frame, where the second frame is the original frame following the first frame.

13. The display device according to claim 12, characterized in that, The frame interpolation module is configured to perform frame interpolation based on the first frame data if the first frame is completed and the first and second frame interpolation conditions are met, thereby obtaining the interpolated frame data. The second frame interpolation condition refers to the time interval between the current moment and the start time of the second frame being drawn being greater than the first threshold.

14. The display device according to claim 11, characterized in that, The device further includes: The capacity limit adjustment module is configured to increase the capacity limit of the buffer in response to the detection of frame dropping, wherein frame dropping refers to the failure to draw new frames within a preset time period. The drawing module is configured to draw a third frame based on the increased capacity limit and the third frame data, which is used to characterize the content of the third frame, and the third frame is the next frame after the latest drawn frame.

15. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the display method as described in any one of claims 1-10.

16. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the display method as described in any one of claims 1-10.

17. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the display method as described in any one of claims 1-10.

Images