Image display method and electronic device

Abstract

Provided are an image display method and an electronic device, which help to reduce the likelihood of frame loss during rotation transition animation, thereby making display content less likely to freeze. The method comprises: when a display screen of an electronic device is in a first state, processing image data on the basis of Vsync-app at a first frequency and Vsync-sf at the first frequency, and performing display submission processing on a processed image; then, while the display screen switches from the first state to the second state, performing drawing and rendering processing on content layer data of a video application on the basis of Vsync-app at a second frequency, performing drawing and rendering processing on rotation layer data and system layer data of a first application on the basis of Vsync-app at the first frequency, then performing compositing processing on rendered image data on the basis of Vsync-sf at the first frequency, and performing display submission processing on a composited image.

Description

Image display methods and electronic devices Technical Field

[0001] This application relates to the field of display technology, and in particular to an image display method and electronic device. Background Technology

[0002] With the development of display and sensor technologies in electronic devices, more and more electronic devices support rotation functionality, allowing the content displayed on the screen to rotate along with the device. For example, a video playing in portrait mode can switch to landscape mode as the device is rotated. The process of the displayed content changing from landscape to portrait or vice versa can be called a rotation animation.

[0003] In rotation animations, frame drops may occur, causing stuttering in the displayed content. Therefore, reducing frame drops in rotation animations is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides an image display method and electronic device that, in rotation animation, helps reduce the probability of frame drops, thereby reducing the probability of display content stuttering.

[0005] In a first aspect, embodiments of this application provide an image display method, which can be applied to an electronic device, the electronic device including a display screen. The method may include: upon receiving a first operation from a user, responding to the first operation by launching a first application, so that the display screen of the electronic device displays an image of the first application;

[0006] When the display screen is in the first state, the content layer data and system layer data in the first image are rendered based on the first rendering signal to obtain the first rendered image. The generation frequency of the first rendering signal is the first frequency. The first rendered image is composited based on the first composite signal to obtain the first image. The generation frequency of the first composite signal is the first frequency. The first image is then sent for display.

[0007] During the process of the display screen switching from the first state to the second state, the content layer data in the second image data is drawn and rendered based on the second rendering signal, and the rotation layer data and system layer data in the second image data are drawn and rendered based on the third rendering signal to obtain the second drawn and rendered image; the second drawn and rendered image is synthesized based on the second compositing signal to obtain the second image; and the second image is sent for display.

[0008] The second rendering signal is generated at a second frequency, the third rendering signal is generated at a first frequency, and the second compositing signal is generated at a first frequency. The second frequency is lower than the first frequency to allow more time for compositing the content layer data.

[0009] The first image data is image data from the first application, and the second image data is image data from the first application. The first state is landscape mode, and the second state is portrait mode; or, the first state is portrait mode, and the second state is landscape mode. Switching from the first state to the second state can be a switch from landscape mode to portrait mode, or a switch from portrait mode to landscape mode.

[0010] It is evident that during the screen switching between portrait and landscape modes (i.e. during rotation animation), reducing the generation frequency of rendering signals for content layer data allows more time for content layer data synthesis, which helps reduce the probability of frame drops and thus reduces the probability of content stuttering.

[0011] In conjunction with the method provided in the first aspect, in some embodiments, the first rendering signal, the second rendering signal, and the third rendering signal are Vsync-app signals, and the first compositing signal and the second compositing signal are Vsync-sf signals. The Vsync-app signal is used to trigger the drawing and rendering process, and the Vsync-sf signal is used to trigger the compositing process.

[0012] In conjunction with the method provided in the first aspect, in some embodiments, after the display screen has switched to the second state, i.e., after the rotation animation effect has ended, the content layer data and system layer data in the third image data are rendered based on the fourth rendering signal to obtain a third rendered image; wherein, the third image data is the image data of the first application; the generation frequency of the fourth rendering signal is the first frequency; based on the third composite signal, the third rendered image is composited to obtain a third image; the frequency of the third composite signal is the first frequency; and the third image is sent for display. The fourth rendering signal is the Vsync-app signal, and the third composite signal is the Vsync-sf signal.

[0013] As can be seen, after the rotation animation ends, the generation frequency of the Vsync-sf signal is restored to the first frequency to ensure that the first application can be displayed normally on the screen and to ensure the display effect.

[0014] In conjunction with the method provided in the first aspect, in some embodiments, in response to the display screen switching to the second state, a fourth rendering signal is generated at a first frequency; based on the fourth rendering signal, content layer data and system layer data in the third image data are rendered to obtain a third rendered image. Generating the fourth rendering signal at the first frequency may be achieved by a SurfaceFlinger (SF) interface compositor generating the fourth rendering signal at the first frequency.

[0015] In conjunction with the method provided in the first aspect, in some embodiments, upon receiving a second operation from the user, the display screen is switched from a first state to a second state in response to the second operation; wherein the second operation is an operation on a first control in the first image. The first control may be an entry full-screen control, an exit full-screen control, a full-screen viewing control, etc., used to trigger the display screen to switch from the first state to the second state.

[0016] Alternatively, if the electronic device's sensors detect a change in the display orientation of the screen from a first orientation to a second orientation, the display screen will be switched from the first state to the second state. The sensor detects whether the display orientation has changed from the first orientation to the second orientation, and in response, the electronic device switches the display screen from the first state to the second state. If the first orientation is landscape and the second orientation is portrait, then switching from the first state to the second state is equivalent to switching from landscape to portrait mode; if the first orientation is portrait and the second orientation is landscape, then switching from the first state to the second state is equivalent to switching from portrait to landscape mode.

[0017] It is evident that a second operation input by the user can trigger the display screen to switch from the first state to the second state, or a change in the orientation of the electronic device can trigger the display screen to switch from the first state to the second state.

[0018] In conjunction with the method provided in the first aspect, in some embodiments, during the process of the display screen switching from the first state to the second state, a second rendering signal of the second frequency and a third rendering signal of the first frequency are generated; the content layer data in the second image data is rendered based on the second rendering signal, and the rotation layer data and system layer data in the second image data are rendered based on the third rendering signal.

[0019] In conjunction with the method provided in the first aspect, in some embodiments, the above-mentioned electronic device further includes a Window Manager Service (WMS) and a SF;

[0020] The first application responds to the received landscape / portrait screen switching command by sending a rotation animation start command to WMS;

[0021] In response to the rotation animation start command, WMS sends a first notification message to SF, which indicates the identifier of the first application;

[0022] In response to the first notification message, SF generates a second rendering signal.

[0023] In the first application, when the first control receives the screen orientation switching instruction, it sends a rotation animation start instruction to the WMS, which triggers the WMS to send a first notification message to the SF, so that the SF can generate a second rendering signal for the content layer data of the first application, thereby reducing the probability of frame drops in the rotation animation and thus reducing the probability of content stuttering.

[0024] In conjunction with the method provided in the first aspect, in some embodiments, the electronic device further includes a rendering module;

[0025] The rendering module performs rendering processing on the content layer data in the second image data based on the second rendering signal, and performs rendering processing on the rotation layer data and system layer data in the second image data based on the third rendering signal.

[0026] In conjunction with the method provided in the first aspect, in some embodiments, upon receiving a third operation from the user, the first application is closed in response to the third operation, meaning the screen of the electronic device no longer displays the image of the first application. Upon receiving a fourth operation from the user, the second application is launched in response to the fourth operation. The third operation is used to close the first application, and the fourth operation is used to launch the second application. The first application is a video application, and the second application is a non-video application.

[0027] With the display screen in the first state, the content layer data and system layer data in the fourth image data are rendered based on the fifth rendering signal to obtain the fourth rendered image; the fourth rendered image is composited based on the fourth composite signal to obtain the fourth image; and the fourth image is sent for display.

[0028] During the process of the display screen switching from the first state to the second state, the content layer data, rotation layer data and system layer data in the fifth image data are rendered based on the sixth rendering signal to obtain the fifth rendered image; the fifth rendered image is composited based on the fifth composite signal to obtain the fifth image; and the fifth image is sent for display.

[0029] Specifically, the generation frequency of the fifth rendering signal, the generation frequency of the fourth composite signal, the generation frequency of the sixth rendering signal, and the generation frequency of the fifth composite signal are all the same. The fourth image data and the fifth image data are both image data from the second application. Both the fifth and sixth rendering signals are Vsync-app signals, while the fourth and fifth composite signals are Vsync-sf signals.

[0030] As can be seen, for rotation effects in non-video applications, the generation frequency of the Vsync-app signal is not changed. Since the data content involved in rotation effects in non-video applications is less than that involved in rotation effects in video applications, the display effect of the rotation effect will not be affected even if the generation frequency of the Vsync-app signal is not changed.

[0031] Secondly, embodiments of this application provide another image display method, which can be applied to an electronic device, the electronic device including a display screen. The method may include: upon receiving a first operation from a user, responding to the first operation by launching a first application, so that the display screen of the electronic device displays an image of the first application;

[0032] When the display screen is in the first state, the sixth image data of the first application is rendered based on the rendering signal (i.e., the Vsync-app signal) to obtain the sixth rendered image; the sixth rendered image is composited based on the compositing signal (i.e., the Vsync-sf signal) to obtain the sixth image; and the sixth image is sent for display.

[0033] During the process of the display screen switching from the first state to the second state, the seventh image data of the first application is rendered based on the rendering signal to obtain the seventh rendered image. The seventh rendered image includes the first rendering data, the second rendering data, and the third rendering data. The first rendering data is the rendering data of the target layer in the seventh rendered image, the second rendering data is the rendering data of the rotation layer in the seventh rendered image, and the third rendering data is the rendering data of the system layer in the seventh rendered image. The second rendering data and the third rendering data are combined based on the compositing signal to obtain the seventh image. The seventh image is then sent for display.

[0034] The generation frequencies of the rendering signal and the synthesized signal can be the same or different.

[0035] The first state is landscape mode, and the second state is portrait mode; or, the first state is portrait mode, and the second state is landscape mode. Switching from the first state to the second state can be a switch from landscape mode to portrait mode, or a switch from portrait mode to landscape mode.

[0036] It is evident that during the screen switching between portrait and landscape modes (i.e., during rotation animation), by not compositing the drawing and rendering data of the target layer during the compositing process, the compositing load can be reduced, thereby reducing the probability of frame drops and the probability of display content stuttering.

[0037] In conjunction with the method provided in the second aspect, in some embodiments, when the display screen has been switched to the second state, the eighth image data of the first application is rendered based on the rendering signal to obtain a fifth rendered image; the eighth rendered image includes fourth rendering data, fifth rendering data and sixth rendering data, the fourth rendering data is the rendering data of the target layer in the eighth rendered image, the fifth rendering data is the rendering data of the rotation layer in the eighth rendered image, and the sixth rendering data is the rendering data of the system layer in the eighth rendered image; the fourth rendering data, fifth rendering data and sixth rendering data are combined based on the synthesis signal to obtain the eighth image; the eighth image is then sent for display processing.

[0038] As can be seen, after the rotation animation ends, the compositing process of the drawing and rendering data of the target layer is restored to ensure that the first application can be displayed normally on the screen and to ensure the display effect.

[0039] In conjunction with the method provided in the second aspect, in some embodiments, the target layer is the content layer of the first application. That is, in the rotation animation, the drawing and rendering data of the content layer is not composited during compositing, or the drawing and rendering data of the content layer is filtered out during compositing.

[0040] In conjunction with the method provided in the second aspect, in some embodiments, the above-mentioned electronic device further includes WMS and SF;

[0041] The first application responds to the received landscape / portrait screen switching command by sending a rotation animation start command to WMS;

[0042] In response to the start command of the rotation animation, WMS sends a third notification message to SF, which indicates the identifier of the target layer;

[0043] In response to the third notification message, SF performs composite processing on the second and third rendering data based on the composite signal to obtain the seventh image.

[0044] In the first application, when the first control receives the screen orientation switching instruction, it sends a rotation animation start instruction to the WMS, which triggers the WMS to send a third notification message to the SF, so that the SF filters the first rendering data during compositing to reduce the compositing load, thereby reducing the probability of frame drops and the probability of display content stuttering.

[0045] Thirdly, embodiments of this application provide an electronic device including one or more processors and one or more memories; wherein the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when the one or more processors execute the computer instructions, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof, or to perform the method described in the second aspect and any possible implementation thereof.

[0046] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform a method as described in the first aspect and any possible implementation thereof, or to perform a method as described in the second aspect and any possible implementation thereof.

[0047] Fifthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof, or to perform the method described in the second aspect and any possible implementation thereof.

[0048] Sixthly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are configured to invoke computer instructions to cause the electronic device to perform a method as described in the first aspect and any possible implementation thereof, or to perform a method as described in the second aspect and any possible implementation thereof.

[0049] Understandably, the electronic device provided in the third aspect, the computer storage medium provided in the fourth aspect, the computer program product provided in the fifth aspect, and the chip system provided in the sixth aspect are all used to execute the methods provided in the first or second aspect of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0050] Figure 1 is an example diagram of the image drawing, rendering, compositing, and display process;

[0051] Figure 2 is an example of the process of changing the content displayed on the screen from landscape to portrait mode.

[0052] Figure 3 is an example of the layers involved in the rotation animation;

[0053] Figure 4 is an example of frame loss during the rendering-compositing-display process;

[0054] Figure 5 is an example diagram of the software architecture of electronic device 100;

[0055] Figure 6 is a schematic diagram of the internal interaction flow of an image display method provided in an embodiment of this application;

[0056] Figures 7A to 7C are example diagrams of the display process before the rotation animation starts, during the rotation animation, and after the rotation animation ends;

[0057] Figure 8 is a schematic diagram of the internal interaction flow of another image display method provided in an embodiment of this application;

[0058] Figure 9 is an example diagram of an image drawing, rendering, compositing, and display process provided in an embodiment of this application;

[0059] Figure 10 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0061] The terms "first," "second," "third," etc., used in the embodiments of this application are to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, it may include a series of steps or units, or optionally, steps or units not listed, or other steps or units inherent to these processes, methods, products, or devices. The terms "one embodiment" or "some embodiments," etc., mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of the embodiments of this application, do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0062] Furthermore, "at least one" refers to one or more, while "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0063] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0064] 1. Vertical Synchronization (Vsync) Signal

[0065] The Vsync signal is primarily used to coordinate the workflow between the Graphics Processing Unit (GPU) and the display screen to prevent screen tearing and ensure that electronic devices update images at the correct times. The Vsync signal is generated by the GPU. After the GPU finishes rendering the current frame, it sends a Vsync signal to notify the display screen to update to the next frame. The Vsync signal is periodic, and its period can be set according to the refresh rate. For example, at a refresh rate of 60Hz, the Vsync signal period can be 16.6ms, meaning the electronic device generates a control signal every 16.6ms to trigger the Vsync signal periodically.

[0066] Vsync signals can be divided into hardware Vsync signals and software Vsync signals. Hardware Vsync signals, often abbreviated as Vsync-hw signals, are used to trigger the display refresh process. Vsync-hw signals can be understood as the tearing effect (TE) signal. Software Vsync signals include Vsync-app and Vsync-sf signals. The Vsync-app signal triggers the rendering process, and the Vsync-sf signal triggers the compositing process. For ease of description, Vsync-hw, Vsync-app, and Vsync-sf signals will be abbreviated as Vsync-hw, Vsync-app, and Vsync-sf, respectively.

[0067] The generation of Vsync-hws is related to the display's refresh rate; Vsync-hws are generated based on the display's refresh rate. For example, if the display's refresh rate is 60Hz, and the display can refresh and display 60 frames per second, then 60 Vsync-hws can be generated per second to display 60 frames per second.

[0068] The generation of Vsync-app and Vsync-sf is related to Vsync-hw; Vsync-app and Vsync-sf are generated based on Vsync-hw. In some embodiments, DispSyncSource.cpp in the SurfaceFlinger (SF) virtualizes Vsync-hw into Vsync-app and Vsync-sf. Specifically, DispSyncSource.cpp samples Vsync-hw, and the sampling result is input into the software Vsync model. DispSyncSource.cpp runs the software Vsync model to output Vsync-app and Vsync-sf.

[0069] Vsync-hw, Vsync-app, and Vsync-sf typically appear as pulses. While some tools may perceive these signals as rising and falling rectangular waves, this is merely for ease of observation and analysis and does not imply that these signals are actually rectangular waves. In reality, these signals represent an instantaneous action, with the rising or falling edge indicating the signal's arrival.

[0070] The frequency at which the application (APP) renders images is controlled by Vsync-app distributed by SF. The APP sends the rendered images to SF for compositing. The compositing frequency on the SF side is controlled by Vsync-sf. In essence, the APP is the producer (image producer), and the SF is the consumer (image compositer). Typically, the frequencies of Vsync-app and Vsync-sf are consistent.

[0071] For example, the image rendering-compositing-display process can be seen in Figure 1. The electronic device uses Vsync-app, Vsync-sf, and Vsync-hw for rendering, compositing, and display. In chronological order, the images displayed by the electronic device correspond to frame 1 and frame 2 respectively. Taking the display of frame 1 as an example, the APP, under the action of a Vsync-app, completes the rendering of frame 1 and sends the rendered frame 1 to SF. SF, under the action of a Vsync-app, composites the rendered frame 1. After compositing, SF sends the composited frame 1 to the display driver to display the content corresponding to frame 1 on the screen. The display driver, under the action of a Vsync-hw, sends the composited frame 1 to the display screen for display. The process of rendering and displaying frame 2 is the same as that of rendering and displaying frame 1.

[0072] Figure 1 illustrates an example where there is no time interval between Vsync-app, Vsync-sf, and Vsync-hw; that is, when a Vsync-hw arrives, a Vsync-app and a Vsync-sf also arrive simultaneously. In practical applications, there may be a certain time interval between the three, but their frequencies are the same. Same frequency means the same period.

[0073] II. Buffer and Transaction

[0074] A buffer typically refers to a temporary storage area used to store image content, responsible for temporarily storing image content passed down from the application layer. A buffer can be divided into a front buffer and a back buffer. The front buffer contains the currently displayed image content, while the back buffer contains the image content to be displayed in the next frame. This double-buffering mechanism effectively reduces screen tearing. The image content refers to the visual elements included in the image, such as animals, portraits, and landscapes.

[0075] A transaction typically refers to the submission and processing of a series of operations (such as setting layer attributes and updating image content) as a whole during image rendering. During development, the system initiates a transaction based on the developer's instructions, which usually involves creating a transaction object or marking the start of a transaction. After the transaction is started, the developer can set layer attributes such as position, size, and transparency. These settings are temporarily stored and do not take effect immediately. After the necessary operations are set, the developer calls the transaction's submission method to submit the transaction. A transaction can include layer attribute information and image content information for a frame of image. In this embodiment, the transaction serves as a transfer object between the rendering module and SF, and also as a processing object for SF compositing. The rendering module passes the transaction, including layer attribute information and image content information, to SF's Transaction Queue, and SF reads the transaction from the Transaction Queue for compositing.

[0076] By encapsulating layer attribute information and image content within a single transaction, it's ensured that this information and content can be applied to the layer, guaranteeing data consistency and integrity and preventing loss or corruption due to unforeseen circumstances such as system crashes. However, transactions containing a large amount of attribute information or complex calculations may prolong rendering and compositing time.

[0077] III. Client and Device Composition

[0078] Currently, SF supports two compositing methods: Client compositing and Device compositing. After collecting all buffers in the visible layer, SF queries the Hardware Composer (HWC) for instructions on how to perform the compositing. HWC is a Hardware Abstraction Layer module in the Android system used for window compositing and display, providing hardware support for SF.

[0079] Client compositing renders the content of each layer into a temporary buffer using the GPU, and then sends this buffer to the display hardware (such as the display device, display driver, and screen). This temporary buffer can be called an FBTarget, and each display device has its own FBTarget. Client compositing can also be called GLES compositing or GPU compositing. Client compositing is performed using the RenderEngine thread. During client compositing, the RenderEngine thread selects version 2.0 based on the OpenGL ES version supported by the GPU (if version 2.0 is not available, then version 1.1 / 1.0 is used). The RenderEngine thread renders the content of each layer into the temporary buffer (FBTarget) by calling the OpenGL ES API, and then sends this temporary buffer to the display hardware for display.

[0080] Device compositing transmits all the content from each layer to the display hardware and instructs it to read different portions of the display screen from different buffers. HWC is an abstraction of Device compositing.

[0081] Device compositing combines multiple buffers using hardware capabilities, typically resulting in better performance and lower power consumption. However, when the compositing process becomes more complex and exceeds the hardware's capacity, client compositing is required. This involves first overlaying some buffers in SF (Software-Defined) mode, and then handing the resulting layer over to the hardware for further processing. Here, "hardware" refers to HWC (Hardware-Wide Container Registry), and the hardware's capabilities are equivalent to those of HWC.

[0082] IV. Rotational Animation

[0083] Rotation effects refer to the process of the content displayed on the screen changing from landscape to portrait or vice versa. For example, see Figure 2, which illustrates the process of the content displayed on the screen changing from landscape to portrait. Figure 2 uses a video application as an example, showing the video being played changing from portrait to landscape. In one implementation, the electronic device receives a click operation on the "Enter Fullscreen" control 101 in Figure 2, and in response to the click operation, changes the video being played by the video application from portrait to landscape, that is, from the interface shown in Figure 2A to the interface shown in Figure 2B. Alternatively, the electronic device receives a click operation on the "Exit Fullscreen" control 102 in Figure 2B, and in response to the click operation, changes the video being played by the video application from landscape to portrait, that is, from the interface shown in Figure 2B to the interface shown in Figure 2A. For ease of distinction, the interface shown in Figure 2A is referred to as the video application's playback interface, and the interface shown in Figure 2B is referred to as the video application's fullscreen playback interface. The term "enter full-screen control" in Figure 2 is used as an example; it could also be called a rotation control, full-screen switching control, or full-screen viewing control, etc. For instance, in short video applications, the control that switches from portrait to landscape mode is called a full-screen viewing control. In another implementation, when the "auto-rotate" function of the electronic device is enabled, the electronic device detects that its orientation has changed from portrait to landscape, and then automatically changes the video being played by the video application from portrait to landscape mode.

[0084] In this embodiment, the display screen has two states: landscape mode and portrait mode. Landscape mode refers to the content displayed on the screen being displayed in landscape mode, as shown in Figure 2B; portrait mode refers to the content displayed on the screen being displayed in portrait mode, as shown in Figure 2A.

[0085] The above rotation animation mainly involves three types of layers, namely:

[0086] (1) Rotation and mask layers can be understood as the front texture and back mask of the rotation animation;

[0087] (2) APP content layers, such as video interfaces that play video and bullet comments during the rotation of horizontal and vertical screens;

[0088] (3) Other layers, such as the status bar and notification bar, will also have a fade-in and fade-out effect during rotation, and the notification content may also change.

[0089] For example, see Figure 3 for an example of the layers involved in the rotation animation. Assuming Figure 3 represents the layers involved in video application A during the landscape / portrait switching process, the numbers in Figure 3 represent layers: 1 represents PointerLocation-display0#290, the pointer position display layer, which can be understood as the front layer of the rotation animation; 2 represents UnknownDisplay, the unknown layer, which can be understood as the back layer of the rotation animation; 3 to 7 represent the content layers of video application A, which may include the video content layer played by video application A and the bullet screen layer; 8 represents the status bar layer; and 9 represents the BackColor layer. Among these, layers 1, 2, and 9 represent rotation and mask layers; layers 3 to 7 represent the app content layers; and layer 8 represents other layers.

[0090] For ease of description, in this application embodiment, the rotated and masked layers are referred to as rotated layers, the APP content layers are referred to as content layers, and other layers are referred to as system layers.

[0091] For example, frame dropping during the image rendering-compositing-display process can be seen in Figure 4. Taking video application A as an example, the process shown in Figure 4, from top to bottom, can be divided into three processes, which are as follows:

[0092] 1. APP drawing and rendering.

[0093] For video application A that does not have rotation effects, when Vsync-app arrives, the APP draws and renders image frames based on the refresh of the content layer of video application A.

[0094] For a video application A that has a rotation animation, when a Vsync-app arrives, it triggers a refresh of the content layer, rotation layer, and system layer of video application A. The application then draws and renders image frames based on the refreshed content. For example, when one Vsync-app arrives, the application draws and renders frame 1 based on the refreshed content; when the next Vsync-app arrives, the application draws and renders frame 2 based on the refreshed content, and so on.

[0095] Rendering involves three types of animations: InsetsAnimation, wmshell.anim, and android.anim. InsetsAnimation refers to animation of inserted areas, typically used to handle animation effects of inserted areas at the screen edges (such as the status bar). It can achieve smooth appearance, disappearance, and resizing of inserted areas, enhancing the visual appeal and interactivity of the user interface. wmshell.anim provides animation effects for WMS. When an application window is opened or closed, or when switching between different windows, it can produce smooth transition animations, making interface transitions more natural and aesthetically pleasing. android.anim is the animation-related package in the Android system. Classes and resources in this package can be used to implement various animation effects, such as property animations and tween animations. Tween animations can include AlphaAnimation for changing the transparency of a view, RotateAnimation for rotating a view, ScaleAnimation for scaling a view, and TranslateAnimation for moving a view. These tween animations allow a view to smoothly transition from one state to another over a period of time.

[0096] For rotation effects, taking frame 1 as an example, after frame 1 has been drawn and rendered, the app uses Binder to transfer the drawn and rendered frame 1 to SF's TransactionQueue across processes. Binder is an important inter-process communication mechanism in electronic device systems. Binder allows for rapid data exchange and method calls between different processes, thus enabling the transfer of drawn and rendered content to SF across processes via Binder.

[0097] 2. SF synthesis.

[0098] For rotation effects, when a Vsync-sf arrives, SF (Streaming Functions) retrieves a transaction from the TransactionQueue for consumption to composite the already rendered frames. SF uses a Client-based compositing method and calls the RenderEngine thread for compositing. During the rotation effect process, when SF retrieves the corresponding transaction from the TransactionQueue under the influence of a Vsync-sf, it can obtain the transaction of the content layer, the rotation layer, and the system layer of video application A. SF then performs compositing based on these three transactions.

[0099] For images without rotation effects, the composite content only involves the content layer of video application A. However, for images with rotation effects, the composite content involves not only the content layer of video application A, but also the rotation layer and the system layer. In other words, rotation effects involve more or more complex composite content.

[0100] For rotation animation effects, during the compositing process using Client compositing, when a Vsync-sf arrives, because the compositing content corresponding to the Vsync-sf transition is large or complex, the RenderEngine thread takes a long time to process. As a result, when the next Vsync-sf arrives, SF cannot compose and display the image frame corresponding to the Vsync-sf in time, thus dropping frames when displaying image frames.

[0101] For example, in Figure 4, assume that the Vsync-sf triggering the composition of frame 2 arrives at time t1, and the Vsync-sf triggering the composition of frame 3 arrives at time t2. The composition content of frame 2 is extensive or complex. At time t1, SF reads the already rendered transaction of frame 2 and performs composition based on this transaction. Because the composition content of frame 2 is extensive or complex, when the Vsync-sf arrives at time t2, the SF's composition of frame 2 may not have finished. Therefore, SF continues to compose frame 2, and thus at time t2, SF will not perform composition of frame 3, resulting in frame 3 not being composed.

[0102] In Figure 4, the frequency of Vsync-app is consistent with the frequency of Vsync-sf. The arrival of Vsync-app triggers the content refresh and rendering of the rotation animation-related layers, while the arrival of Vsync-sf triggers SF to perform compositing. Thus, Vsync-app and Vsync-sf maintain a rhythmic producer-consumer model.

[0103] 3. Display screen.

[0104] After the rendered frames are composited, SF sends the composited frames to the display driver to display the corresponding content on the screen. When Vsync-hw arrives, the display driver sends the composited frames to the screen for display.

[0105] In the example above, the synthesis of frame 2 is completed after time t2, while the synthesis of frame 3 is not completed. As a result, under the action of Vsync-hw, the display of frame 2 is prolonged, and frame 3 will not be displayed, resulting in frame dropping when displaying image frames.

[0106] Before and during the rotation animation, the image drawing, rendering, compositing, and display process is the same. However, because the rotation animation involves many layers and has a lot of complex content, using the Client compositing method during the rotation animation process can lead to a heavy load on the RenderEngine thread and excessive processing time. This can result in frame drops during the display of the rotation animation, which may cause noticeable stuttering for the user and affect the user experience.

[0107] In view of this, embodiments of this application provide an image display method and electronic device. In rotation animation, by reducing the frequency of Vsync-app or filtering some transactions, the load on the RenderEngine thread is reduced, thereby reducing the probability of frame drops and thus reducing the probability of display content stuttering. When reducing the frequency of Vsync-app or filtering some transactions, the rotation layer is not processed to ensure the rotation effect.

[0108] Electronic devices are not limited to mobile phones and tablets, but can also include desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices, and / or smart city devices. This application does not impose any special limitations on the specific type of electronic device.

[0109] Figure 5 illustrates the software architecture of electronic device 100. The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to illustrate the software structure of electronic device 100.

[0110] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the Hardware Abstraction Layer (HAL), and the kernel layer.

[0111] The application layer may include a series of application packages. As shown in Figure 5, the application packages may include applications such as camera, gallery, video, music, navigation, calendar, map, and WLAN. In this embodiment, the application layer also includes a video application. The content played by the video application can switch between landscape and portrait modes as the electronic device 100 rotates, or switch from landscape to portrait mode or vice versa in response to a click on the full-screen control 101. The video application may include, but is not limited to, short video applications and long video applications. The content played by the video application may include video content, bullet comments, and advertising content.

[0112] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 5, the application framework layer may include Window Manager Service (WMS), a phone manager, a content provider, a notification manager, a resource manager, a view system, etc.

[0113] WMS is used to manage windows, such as handling tasks like window creation, layout, display, and interaction. WMS also coordinates the display of windows on the screen, passes input events to the appropriate applications, and collaborates with other components. Collaborating with other components, such as SF in the system library, ensures that the window content is correctly rendered on the screen. In this embodiment, WMS can identify the start and end times of a rotation animation. When the start of a rotation animation is detected, it calls an API to notify SF to reduce the frequency of Vsync-app or filter some transactions; when the end of a rotation animation is detected, it calls an API to notify SF to restore the frequency of Vsync-app or resume processing of that part of the transaction.

[0114] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0115] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0116] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build the display interface of an application. The display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0117] The Android Runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries.

[0118] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0119] The system library can include multiple functional modules, such as SF (Surface Manager), Surface Manager, Media Libraries, and graphics processing libraries (e.g., OpenGL or OpenGL ES). SF is responsible for compositing and displaying image frames and is a crucial component of the system library. The Surface Manager manages the display subsystem and provides 2D and 3D layer blending for multiple applications. The Media Libraries support playback and recording of various common audio and video formats, as well as still image files. The Media Libraries support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The graphics processing libraries are used for graphics drawing, image rendering, compositing, and layer processing.

[0120] In this embodiment, when the rotation animation starts, the SF receives a notification message from the WMS, and in response to the notification message, reduces the frequency of Vsync-app or filters some transactions. When the rotation animation ends, the SF receives a notification message from the WMS, and in response to the notification message, restores the frequency of Vsync-app or resumes the processing of that part of the transaction.

[0121] The Hardware Abstraction Layer (HAL) provides standard interfaces for controlling hardware operations. HAL comprises multiple library modules, each implementing a set of interfaces for a specific type of hardware component, such as the WLAN module and HWC. HWC is the HAL module used for layer compositing and display in Android, providing hardware support for SF.

[0122] The kernel layer is the foundation of the Android system. For example, ART relies on the kernel layer to execute low-level functions, such as threads and low-level memory management. The kernel layer is the layer between hardware and software. At a minimum, the kernel layer includes display drivers, camera drivers, audio drivers, sensor drivers, and GPU drivers.

[0123] The image display method provided in this application embodiment will be described in conjunction with the software architecture shown in Figure 5. The following description uses a video application A currently running on the electronic device as an example. The video played by video application A can be switched from portrait to landscape mode, or vice versa.

[0124] Please refer to Figure 6, which is an example diagram of the internal interaction flow of an image display method provided in an embodiment of this application. The method may include, but is not limited to, the following steps:

[0125] 601, The drawing and rendering module performs drawing and rendering based on the first frequency to obtain the first drawing and rendering image.

[0126] Before the rotation animation begins, the image data of video application A undergoes rendering, compositing, and display processing, and is finally displayed on the screen. In the embodiment shown in Figure 6, the image data of the video played by video application A before the rotation animation begins is collectively referred to as the first image data.

[0127] The rendering module renders the first image data based on a first frequency to obtain a first rendered image. The first image data includes content layer data and system layer data, i.e., the content layer data and system layer data of video application A. The rendering module may include a drawing thread and a rendering thread. When video application A receives a Vsync-app, it calls the drawing thread to draw a piece of the first image data and calls the rendering thread to render the drawn image to obtain a first rendered image. The frequency of the Vsync-app is the first frequency, for example, 90Hz. The period of the Vsync-app is 11.1ms, meaning the time interval between two adjacent Vsync-apps is 11.1ms, and a first rendered image can be generated every 11.1ms.

[0128] Calling the drawing thread and rendering thread can be done either by the application framework layer calling the drawing and rendering functions, or by the application framework layer calling the drawing and rendering functions in the graphics processing library.

[0129] Before step 601, the electronic device receives a first operation from the user, responds to the first operation, and launches video application A. The first operation could be, for example, clicking the icon of video application A, or launching video application A via voice.

[0130] 602, The drawing and rendering module sends the first drawing and rendering image to SF.

[0131] The rendering module sends the first rendered image to SF. The rendering module can pass the first rendered image to SF across processes via Binder.

[0132] SF can store the transaction of the first drawn and rendered image in the Transaction Queue. The transaction of the first drawn and rendered image may include transactions of the content layer and transactions of the system layer.

[0133] 603, SF synthesis yields the first image.

[0134] Under the control of Vsync-sf, SF reads the transaction of the first rendered image and uses the Client compositing method to composite it into the first image. When using the Client compositing method, the RenderEngine thread is invoked for compositing.

[0135] The frequency of Vsync-sf is the same as that of Vsync-app, both being the first frequency. At the same time, Vsync-app and Vsync-sf may arrive simultaneously, or only Vsync-app or Vsync-sf may arrive.

[0136] 604, SF sends the first image to the display driver.

[0137] The SF can perform display processing on the first image. For example, the display process may include the SF sending the first image to the display driver via the HWC.

[0138] 605, Display driver controls the display screen to display the first image.

[0139] Under the action of Vsync-hw, the display driver drives the display screen to display the first image. The frequency of Vsync-hw is the same as the frequency of Vsync-sf. Steps 604 and 605 can be understood as two steps in the display sending process, which may also involve other steps. The frequency of Vsync-app can also be described as the generation frequency of Vsync-app, and the frequency of Vsync-sf can also be described as the generation frequency of Vsync-sf.

[0140] Steps 601 to 605 describe the process of rendering, compositing, and displaying the image of video application A before the rotation animation begins. During this process, the frequency of Vsync-app or Vsync-sf is the same.

[0141] 606, Video application A receives a command to switch between portrait and landscape modes.

[0142] In one implementation, a click operation on the full-screen control 101 is received on the playback interface of video application A. In response to this click operation, a landscape / portrait switching instruction is received, which triggers the video being played by video application A to switch from portrait to landscape mode. Alternatively, a click operation on the exit full-screen control 102 is received on the playback interface of video application A. In response to this click operation, a landscape / portrait switching instruction is received, which triggers the video being played by video application A to switch from landscape to portrait mode. In this embodiment, the full-screen control 101 and the exit full-screen control 102 can be referred to as the first control. That is, the first control is the control that triggers the display state to switch from landscape to portrait mode, or from portrait to landscape mode. The click operation on the full-screen control 101 and the click operation on the exit full-screen control 102 can be referred to as the user's second operation.

[0143] 607, Video application A sends a rotation animation start command to WMS.

[0144] In response to the screen orientation switch command, video application A sends a rotation animation start command to WMS so that WMS can recognize that the rotation animation has started.

[0145] WMS can also identify the start of a rotational animation in one or more of the following ways:

[0146] Method 1: The WMS (Wind Management System) recognizes the start of a rotation animation based on event notifications from sensors. Electronic devices are typically equipped with sensors such as gravity sensors, orientation sensors, and gyroscopes, which can detect changes in the orientation and rotation of the electronic device. When a sensor detects that the display orientation changes from a first orientation to a second orientation (e.g., from landscape to portrait or from portrait to landscape), it sends an event notification to the WMS. Based on the event notification, the WMS determines whether the electronic device has rotated; if rotation is detected, it can recognize the start of the rotation animation.

[0147] Method 2: WMS detects the start of rotation animations based on the activity's configuration change event. When an electronic device rotates, the Android system triggers the activity's configuration change event. For example, when an electronic device switches from portrait to landscape or vice versa, the activity's onConfigurationChanged() method is called. WMS can understand the electronic device's rotation state by listening to the activity's configuration change event, and when it detects the event, it detects the start of rotation animations.

[0148] The prerequisite for methods 1 and 2 above is that the "auto-rotation" function of the electronic device is enabled.

[0149] Method 3: WMS detects the start of the rotation animation based on the window icon. WMS manages the window layout and display; the window icon for landscape playback of a video application differs from that for portrait playback. When WMS detects a switch in the window icon from full-screen mode to non-full-screen mode, or vice versa, it detects the start of the rotation animation. In this method, the electronic device's "auto-rotate" function is enabled, or a click operation to enter the full-screen control 101 is received on the video application's playback interface, or a click operation to exit the full-screen control 102 is received on the video application's full-screen playback interface.

[0150] Method 4: WMS detects the start of a rotation animation based on system services or broadcast events. The Android system provides several system services, such as the DisplayManagerService, with which WMS can interact to obtain information about the display status of the electronic device. When the electronic device rotates, the DisplayManagerService notifies WMS, which can then detect the start of the rotation animation. The Android system can also send broadcast events to notify applications and system components of changes in system state. For example, when an electronic device rotates, the system sends a configuration change broadcast. WMS can register to listen for these broadcast events to stay informed about the rotation status of the electronic device. When WMS detects a configuration change broadcast, it recognizes the start of the rotation animation.

[0151] The above implementation methods are for illustrative purposes only and do not constitute a limitation on the embodiments of this application.

[0152] 608. WMS sends a first notification message to SF. The first notification message is used to indicate the identifier of video application A.

[0153] WMS detects the start of the rotation animation and sends a first notification message to SF. This first notification message identifies the video application A. Video application A is the currently running video application on the electronic device, i.e., the application that needs to be load-reduced. In other words, the first notification message identifies the target application, which can be the currently running video application (i.e., the one currently playing a video) or another currently running application. The currently running application refers to the application currently running in the foreground. The target application can also be described as the target process, which is the currently running foreground process.

[0154] In addition to indicating the identifier of the target application, the first notification message can also indicate the load reduction method. That is, the first notification message indicates that a load reduction method should be used to reduce the load on the target application in the rotation animation. In the embodiment shown in Figure 6, the load reduction method is to reduce the frequency of the target application's Vsync-app, that is, to reduce the frequency of the Vsync-app of the target application's content layer data.

[0155] 609, SF responds to the first notification message by reducing the frequency of the Vsync-app of video application A from the first frequency to the second frequency.

[0156] In response to the first notification message, SF reduces the frequency of the Vsync-app in video application A from a first frequency to a second frequency to extend the compositing time of SF in rotational animations. For example, if the first frequency is 90Hz, the Vsync-app frequency can be reduced from 90Hz to 60Hz to extend the compositing time of SF in rotational animations. Reducing the frequency of the Vsync-app in video application A from the first frequency to the second frequency also means reducing the frequency of the Vsync-app for the content layer data of video application A from the first frequency to the second frequency.

[0157] Optionally, the first frequency can be 1.5 times the second frequency, for example, the first frequency is 90Hz and the second frequency is 60Hz. Optionally, the first frequency can be 2 times the second frequency, for example, the first frequency is 120Hz and the second frequency can be 60Hz. Optionally, the first frequency can be 120Hz and the second frequency can be 90Hz.

[0158] In SF's DispSyncSource.cpp, Vsync-hw is virtualized into Vsync-app and Vsync-sf. Typically, with a fixed frequency for Vsync-hw, the frequencies of Vsync-app and Vsync-sf are also fixed, and they are identical. This can lead to dropped frames in rotational animations. However, in the embodiment shown in Figure 6, SF can reduce the frequency of Vsync-app for video application A during rotational animations, while keeping the frequency of Vsync-sf unchanged (i.e., the frequency of Vsync-sf applies to all layers involved in the rotational animation). Therefore, when the next Vsync-sf transition occurs, the composition content corresponding to video application A may not exist, eliminating the need for SF to perform client compositing. This allows SF more time to composite the content of video application A, preventing dropped frames in the displayed image of video application A.

[0159] Optionally, SF reduces the frequency of Vsync-app for video application A, while leaving the frequencies of Vsync-app for rotation layers and system layers unchanged at the highest frequency to ensure basic rotation effects. In other words, the Vsync-app frequency for rotation layer data and system layer data remains the highest frequency.

[0160] 610, the rendering module receives Vsync-app at the second frequency.

[0161] The second frequency of Vsync-app is Vsync-app for the content layer data of video application A.

[0162] 611, The drawing and rendering module performs drawing and rendering based on the second frequency to obtain the second drawing and rendering image.

[0163] The rendering module renders the content layer data in the second image data based on the second frequency, and renders the rotation layer data and system layer data in the second image data based on the first frequency, to obtain the second rendered image. In the embodiment shown in Figure 6, the image data of the video played by video application A after the rotation animation starts is collectively referred to as the second image data. Video application A receives the Vsync-app at the second frequency, and the rendering module renders the content of video application A under the action of the Vsync-app at the second frequency.

[0164] 612, The drawing and rendering module sends the second drawing and rendering image to SF.

[0165] The rendering module sends the second rendered image to SF. The rendering module can also pass the second rendered image to SF via Binder across processes.

[0166] SF can store the transaction of the second drawn and rendered image in the Transaction Queue.

[0167] 613, SF synthesis yields the second image.

[0168] Under the influence of Vsync-sf, SF reads the transaction of the second rendered image and performs compositing using the Client compositing method to obtain the second image. When performing compositing using the Client compositing method, the RenderEngine thread is invoked for compositing. The frequency of Vsync-sf in step 613 is the same as the frequency of Vsync-sf in step 603.

[0169] 614, SF sends a second image to the display driver.

[0170] SF can send a second image to the display driver via HWC.

[0171] 615, the display driver controls the display screen to show the second image.

[0172] The display driver, under the control of Vsync-hw, drives the display to show a second image. The frequency of Vsync-hw is the same as the frequency of Vsync-sf.

[0173] Steps 606 to 615 describe the process of rendering, compositing, and displaying the image of video application A after the rotation animation begins (i.e., during the rotation animation process). During this process, the frequency of Vsync-app for video application A differs from the frequency of Vsync-app for the rotation layer and the system layer.

[0174] For example, see Figures 7A to 7C, which are example diagrams of the display process before the start of the rotation animation, during the rotation animation, and after the rotation animation ends. In Figures 7A to 7C, the frequency of Vsync-app1 is the same as the frequency of Vsync-sf, and the frequency of Vsync-app1 is twice the frequency of Vsync-app2.

[0175] In Figure 7A, the display screen is in its first state before the rotation animation begins. Taking frame 1 as an example, the rendering module performs rendering processing on the layer data of frame 1 based on Vsync-app1. This layer data includes content layer data and system layer data, but does not involve rotation layer data. The rendering module then passes the rendered frame 1 to SF. Under the action of Vsync-sf, SF performs composite processing on the rendered frame 1 and passes the composited frame 1 to the display driver, so that the display driver, under the action of Vsync-hw, drives the display screen to display frame 1.

[0176] In Figure 7B, the rotation animation occurs during the transition from the first state to the second state of the display screen. Assume frames 5, 6, and 7 in Figure 7B represent the first three frames of video application A after the rotation animation begins. Upon receiving the first Vsync-app2, the rendering module renders the content layer data in frame 5 based on Vsync-app2, and renders the rotation layer data and system layer data in frame 5 based on Vsync-app1, then passes the rendered frame 1 to SF. In Figure 7B, frame 5-1 represents the content layer data in frame 5, and frame 5-2 represents the rotation layer data and system layer data in frame 5. When the first Vsync-app1 arrives, the rendering module does not render frame 5-2; instead, it renders frame 5-2 when the second Vsync-app1 arrives. In Figure 7B, the rendering of frames 6-1 and 6-2 occurs simultaneously. In other words, when a Vsync-app1 arrives, the rendering module may render the rotation layer data and system layer data, or it may render the rotation layer data and system layer data when the next Vsync-app1 arrives. Upon receiving the third Vsync-sf, the SF composites the rendered frame 5 and passes the composited frame 5 to the display driver, so that the display driver, under the action of Vsync-hw, drives the display to show frame 5. Upon receiving the second Vsync-app2, the rendering module renders the content layer data in frame 6 based on Vsync-app2, and renders the rotation layer data and system layer data in frame 6 based on Vsync-app1, and passes the rendered frame 6 to the SF. Upon receiving the fourth Vsync-sf, the SF does not have the composited content of the video application A corresponding to this Vsync-sf, so no compositing is needed. Therefore, upon receiving the fifth Vsync-sf, the SF composites the rendered frame 6 and passes the composited frame 6 to the display driver, so that the display driver, under the action of Vsync-hw, drives the display to show frame 6. In this way, even if there is a lot of composite content for frame 5, it will not cause frame drops in frames 5 and 6. That is to say, by reducing the frequency of Vsync-app in the rotation animation, SF can have more time to composite the content of video application A, thereby avoiding frame drops in the displayed image of video application A.

[0177] In Figure 7C, after the rotation animation ends, the display is in the second state. Taking frame 8 as an example, the rendering module performs rendering processing on the layer data of frame 8 based on Vsync-app1. This layer data includes content layer data and system layer data, but does not involve rotation layer data. The rendering module then passes the rendered frame 8 to SF. Under the action of Vsync-sf, SF performs composite processing on the rendered frame 8 and passes the composited frame 8 to the display driver, so that the display driver, under the action of Vsync-hw, drives the display to show frame 8.

[0178] 616, WMS detected the end of the rotation animation.

[0179] In one implementation, WMS detects that the window has switched from full-screen mode to non-full-screen mode, or from non-full-screen mode to full-screen mode, and recognizes that the rotation animation has ended.

[0180] In another implementation, WMS detects that the rotation of the layer, the app content layer, and the system layer has been completed, thus recognizing the end of the rotation animation.

[0181] 617. WMS sends a second notification message to SF. The second notification message is used to indicate a return to the first frequency.

[0182] WMS detects the end of the rotation animation and sends a second notification message to SF. The second notification message is used to instruct the Vsync-app frequency of video application A to be restored to the first frequency.

[0183] 618, SF responds to the second notification message by restoring the frequency of the Vsync-app of video application A to the first frequency.

[0184] 619, The rendering module receives the first frequency of Vsync-app.

[0185] The first frequency of Vsync-app is for the content layer data of video application A, and also for system layer data.

[0186] 620, the drawing and rendering module performs drawing and rendering based on the first frequency to obtain the third drawing and rendering image.

[0187] The rendering module renders the content layer data and system layer data in the third image data based on the first frequency to obtain the third rendered image. In the embodiment shown in Figure 6, the image data of the video played by video application A after the rotation animation ends is collectively referred to as the third image data. Video application A receives the Vsync-app at the first frequency, enabling the rendering module to render the content layer and system layer of video application A under the Vsync-app at the first frequency.

[0188] 621, The drawing and rendering module sends the third drawing and rendering image to SF.

[0189] 622, SF synthesis yields the third image.

[0190] 623, SF sends a third image to the display driver.

[0191] 624, the display driver controls the display screen to show the third image.

[0192] The processes of steps 621 to 624 are the same as those of steps 602 to 605, and will not be described again here.

[0193] Steps 616 to 624 describe the process of rendering, compositing, and displaying the image of video application A after the rotation animation effect ends. During this process, the frequency of Vsync-app or Vsync-sf is the same.

[0194] In the embodiment shown in Figure 6, when the rotation animation starts, SF reduces the frequency of the Vsync-app of video application A, and restores the frequency of the Vsync-app of video application A when the rotation animation ends, in order to reduce or avoid frame drops in the displayed image of video application A during the rotation animation.

[0195] As an optional embodiment, after step 624 or step 615, upon receiving a third operation from the user, the video application A is closed in response to the third operation; that is, the display screen of the electronic device no longer displays the image of video application A. Upon receiving a fourth operation from the user, a non-video application B is launched in response to the fourth operation. The third operation is used to close video application A, and the fourth operation is used to launch non-video application B.

[0196] With the display screen in the first state, based on the first frequency of Vsync-app, the content layer data and system layer data in the fourth image data of non-video application B are rendered to obtain the fourth rendered image; based on the first frequency of Vsync-sf, the fourth rendered image is composited to obtain the fourth image; the fourth image is sent for display so that the display screen displays the fourth image.

[0197] During the process of the display screen switching from the first state to the second state, based on the first frequency of Vsync-app, the content layer data, rotation layer data and system layer data in the fifth image data of non-video application B are rendered to obtain the fifth rendered image; based on the first frequency of Vsync-sf, the fifth rendered image is composited to obtain the fifth image; the fifth image is sent for display so that the display screen displays the fourth image.

[0198] In other words, for rotation effects in non-video applications, the frequency of Vsync-app is not changed within the rotation effect itself. Since rotation effects in non-video applications involve less data than those in video applications, the display effect of the rotation effect will not be affected even if the frequency of Vsync-app is not changed.

[0199] Please refer to Figure 8, which is an example diagram of the internal interaction flow of another image display method provided in an embodiment of this application. The method may include, but is not limited to, the following steps:

[0200] 801, the drawing and rendering module performs drawing and rendering to obtain the sixth drawing and rendering image.

[0201] The drawing and rendering module draws and renders the sixth image to be processed, so as to obtain the sixth drawn and rendered image.

[0202] 802, The rendering module sends the sixth rendering image to SF.

[0203] The sixth image was obtained by 803 and SF synthesis.

[0204] 804, SF sends the sixth image to the display driver.

[0205] 805, Display driver controls the display screen to display the sixth image.

[0206] Steps 801 to 805 describe the process of drawing, rendering, compositing, and displaying the image of video application A before the rotation animation begins. Steps 801 to 805 are the same as steps 601 to 605, and will not be described again here.

[0207] 806, Video application A receives a command to switch between portrait and landscape modes.

[0208] 807, Video application A sends a rotation animation start command to WMS.

[0209] Steps 806 and 807 can be found in the detailed description of steps 606 and 607 in the embodiment shown in Figure 6, and will not be repeated here.

[0210] 808, WMS sends a third notification message to SF. The third notification message is used to indicate the identifier of the target layer.

[0211] The target layer is the content layer of the currently running video application, which can be all content layers of the target application. For example, if the target application is video application A, the target layer could be all content layers of video application A. The identifier of the target layer is its name.

[0212] In addition to indicating the identifier of the target layer, the third notification message can also indicate the load reduction method. That is, the third notification message indicates that a load reduction method should be used to reduce the load in the rotation animation. In the embodiment shown in Figure 8, the load reduction method is a filtering transaction.

[0213] 809, SF filters the transaction of the target layer in the seventh rendered image.

[0214] In response to the third notification message, the SF filters the transactions of the target layer in the seventh rendered image. The seventh rendered image is the rendered image received by the SF from the rendering module after receiving the third notification message. In the embodiment shown in Figure 8, the rendered images received by the SF from the rendering module after receiving the third notification message are collectively referred to as the seventh rendered image. Filtering the transactions of the target layer means that during compositing, the transactions of the target layer are not considered; that is, no compositing processing is performed on the transactions of the target layer. The seventh rendered image includes the transactions of the target layer, the system layer, and the rotated layer. During compositing, the SF performs compositing processing based on the transactions of the system layer and the rotated layer.

[0215] For example, SF removes the target layer's transaction from the TransactionQueue, so that when retrieving transactions from the TransactionQueue, the target layer's transaction will not be retrieved. This reduces the compositing load when SF performs compositing via the Client compositing method, thus helping to reduce or avoid frame drops.

[0216] When filtering transactions on the target layer, SF does not filter transactions on the rotated layer to ensure the rotation effect.

[0217] SF filters transactions of the target layer in the fourth rendering image, which can also be described as SF filtering the rendering data of the target layer in the fourth rendering image. In other words, rendering data can be used as transactions.

[0218] 810, SF synthesis yields the seventh image.

[0219] After filtering the transaction of the target layer in the seventh rendering image, SF performs composite processing on the transaction of the system layer and the transaction of the rotation layer in the seventh rendering image to obtain the fourth image.

[0220] 811, SF sends the seventh image to the display driver.

[0221] 812, the display driver controls the display screen to show the seventh image.

[0222] Steps 811 and 812 can be found in the detailed description of steps 604 and 605 in the embodiment shown in Figure 6, and will not be repeated here.

[0223] For example, see Figure 9, which is an example diagram of an image rendering-compositing-display process provided by an embodiment of this application. In Figure 9, the frequencies of Vsync-sf, Vsync-app, and Vsync-hw remain unchanged. Assume that frames 1 to 5 in Figure 9 are the first five frames of video application A after the rotation animation begins. Taking the display of frame 1 as an example, after the rendering module completes the rendering of frame 1, it sends the rendered frame 1 to SF. The rendered frame 1 includes the rendering data of the target layer, the rendering data of the rotation layer, and the rendering data of the system layer. Under the action of Vsync-sf, when SF reads the content of frame 1, it does not read the rendering data of the target layer of frame 1. Therefore, it does not consider the transaction of the target layer of frame 1 during compositing, but only the transaction of the rotation layer and the transaction of the system layer. This reduces the time for SF to compose frame 1, thereby reducing the load on the RenderEngine thread. In this way, under the action of Vsync-hw, the display driver can drive the display screen to display frame 1. Because filtering the transaction of the target layer in the rotation animation reduces the load on the RenderEngine thread, frame drops in the displayed image of video application A can be avoided.

[0224] 813, WMS detected that the rotation animation effect had ended.

[0225] Step 813 can be found in the detailed description of step 616 in the embodiment shown in Figure 6, and will not be repeated here.

[0226] 814, WMS sends a fourth notification message to SF. The fourth notification message is used to instruct the resumption of transaction processing on the target layer.

[0227] 815, SF reads the transaction of the target layer in the eighth rendering image and composites it to obtain the eighth image.

[0228] In response to the fourth notification message, the SF reads the transaction of the target layer in the eighth rendering image and resumes the processing of the target layer's transaction. The eighth rendering image is the rendering image received by the SF from the rendering module after receiving the fourth notification message. In the embodiment shown in Figure 8, the rendering images received by the SF from the rendering module after receiving the fourth notification message are collectively referred to as the eighth rendering image. Resuming the processing of the target layer's transaction means considering the target layer's transaction during compositing; that is, performing compositing processing on the target layer's transaction. Specifically, during compositing, the SF performs compositing processing on the target layer's transaction, the rotated layer's transaction, and the system layer's transaction.

[0229] The content to be synthesized in step 815 is the same as the content to be synthesized in step 804.

[0230] 816, SF sends the eighth image to the display driver.

[0231] 817, the display driver controls the display screen to show the eighth image.

[0232] Steps 816 and 817 can be found in the detailed descriptions of steps 604 and 605 in the embodiment shown in Figure 6, and will not be repeated here.

[0233] In the embodiment shown in Figure 8, SF filters the transaction of the target layer at the start of the rotation animation and resumes the processing of the transaction of the target layer at the end of the rotation animation, so as to reduce or avoid frame drops in the rotation animation.

[0234] As an optional implementation, for rotation effects in non-video applications, SF does not filter the transaction of the target layer in the rotation effect. Since the rotation effects in non-video applications involve less data than those in video applications, not filtering the transaction of the target layer will not affect the display effect of the rotation effect.

[0235] Figure 10 illustrates a schematic diagram of the hardware structure of the electronic device 100.

[0236] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0237] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0238] Processor 110 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0239] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0240] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0241] In some embodiments, processor 110 may include one or more interfaces.

[0242] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0243] The charging management module 140 receives charging input from a charger, which can be either a wireless or wired charger. The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, among other components. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance).

[0244] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0245] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. Mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on electronic device 100. The modem processor can include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through display screen 194. The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. In some embodiments, antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and antenna 2 is coupled to the wireless communication module 160, enabling the electronic device 100 to communicate with networks and other devices via wireless communication technologies.

[0246] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0247] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In this embodiment, the display screen 194 can display rotational animation effects, and during the display of rotational animation effects, frame drops can be reduced or avoided.

[0248] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0249] The ISP is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The camera 193 is used to capture still images or videos. An object passes through the lens, generating an optical image that is projected onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0250] A digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals.

[0251] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs.

[0252] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0253] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0254] Random access memory (RAM) can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and double data rate synchronous dynamic random-access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM). Non-volatile memory can include disk storage devices and flash memory.

[0255] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0256] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0257] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0258] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage. During video playback, the speaker connected to the speaker 170A or headphone jack 170D can support the playback of audio from the video.

[0259] Electronic device 100 can implement audio functions through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, such as music playback and recording. Audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. Audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, audio module 170 can be located in processor 110, or some functional modules of audio module 170 can be located in processor 110.

[0260] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make phone calls through the speaker 170A.

[0261] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0262] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also achieve noise reduction.

[0263] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0264] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch intensity can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0265] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). In embodiments of this application, the gyroscope sensor can be used to determine whether the electronic device 100 is in landscape or portrait orientation.

[0266] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0267] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover.

[0268] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0269] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0270] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects.

[0271] An ambient light sensor 180L is used to sense the ambient light intensity. Electronic device 100 can adaptively adjust the brightness of display screen 194 according to the sensed ambient light intensity.

[0272] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0273] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature processing strategy.

[0274] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0275] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords.

[0276] In this embodiment, the accelerometer 180E and the gyroscope 180B are used to detect the physical state of the folding display screen.

[0277] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0278] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback.

[0279] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0280] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and detach from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, and other SIM cards.

[0281] The term "user interface (UI)" used in the specification, claims, and drawings of this application refers to the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form acceptable to the user. The user interface of an application is source code written in a specific computer language such as Java or Extensible Markup Language (XML). This source code is parsed and rendered on the terminal device, ultimately presenting user-recognizable content such as images, text, and buttons. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, images, and text. The attributes and content of controls in the interface are defined using tags or nodes, such as XML tags. <textview> 、 <imgview> 、 <videoview>Nodes define the controls contained in the interface. A node corresponds to a control or property in the interface, and after parsing and rendering, the node is presented as the content visible to the user. In addition, many applications, such as hybrid applications, often contain web pages within their interfaces. A web page, also known as a webpage, can be understood as a special control embedded in the application interface. Web pages are source code written in a specific computer language, such as Hypertext Markup Language (GTML), Cascading Style Sheets (CSS), JavaScript (JS), etc. Web page source code can be loaded and displayed as user-readable content by a browser or a web page display component with browser-like functionality. The specific content contained in a webpage is also defined through tags or nodes in the webpage source code; for example, GTML uses tags or nodes to define the content. 、 、 <video> 、 <canvas>Used to define the elements and attributes of a webpage.

[0282] The most common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0283] As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the listed items. As used in the above embodiments, depending on the context, the term "when" can be interpreted as meaning "if..." or "after..." or "in response to determining..." or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining..." or "in response to determining..." or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)."

[0284] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0285] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. An image display method, characterized in that, The method is applied to an electronic device, the electronic device including a display screen, and the method includes: Receive the user's first action; In response to the first operation, launch the first application; When the display screen is in a first state, the content layer data and system layer data in the first image data are rendered based on the first rendering signal to obtain a first rendered image; wherein, the generation frequency of the first rendering signal is a first frequency; The first rendered image is synthesized based on the first synthesized signal to obtain a first image; wherein the generation frequency of the first synthesized signal is the first frequency; The first image is then processed for display. During the process of the display screen switching from the first state to the second state, the content layer data in the second image data is rendered based on the second rendering signal, and the rotation layer data and system layer data in the second image data are rendered based on the third rendering signal to obtain the second rendered image; wherein, the generation frequency of the second rendering signal is the second frequency, and the generation frequency of the third rendering signal is the first frequency; The second rendered image is synthesized based on the second synthesized signal to obtain the second image; wherein the generation frequency of the second synthesized signal is the first frequency; The second image is then processed for display. Wherein, the first image data is the image data of the first application, and the second image data is the image data of the first application; the second frequency is less than the first frequency; the first state is a landscape state, and the second state is a portrait state; or, the first state is a portrait state, and the second state is a landscape state.

2. The method as described in claim 1, characterized in that, The method further includes: In response to the display screen switching to the second state, the content layer data and system layer data in the third image data are rendered based on the fourth rendering signal to obtain a third rendered image; wherein, the third image data is the image data of the first application; and the generation frequency of the fourth rendering signal is the first frequency; Based on the third synthesized signal, the third rendered image is synthesized to obtain a third image; the frequency of the third synthesized signal is the first frequency. The third image is then processed for display.

3. The method as described in claim 2, characterized in that, In response to the display screen switching to the second state, the content layer data and system layer data in the third image data are rendered based on the fourth rendering signal to obtain a third rendered image, including: In response to the display screen switching to the second state, a fourth rendering signal is generated at the first frequency; Based on the fourth rendering signal, the content layer data and system layer data in the third image data are rendered to obtain the third rendered image.

4. The method as described in claim 1, characterized in that, The method further includes: Receive a second operation from the user, the second operation being an operation on a first control in the first image; in response to the second operation, switch the display screen from the first state to the second state; or, When the sensors of the electronic device detect that the display orientation of the screen has changed from a first orientation to a second orientation, the display screen is switched from the first state to the second state.

5. The method according to any one of claims 1-4, characterized in that, During the process of the display screen switching from a first state to a second state, the content layer data in the second image data is rendered based on a second rendering signal, and the rotation layer data and system layer data in the second image data are rendered based on a third rendering signal, including: During the process of the display screen switching from a first state to a second state, a second rendering signal of a second frequency is generated and a third rendering signal of the first frequency is generated. The content layer data in the second image data is rendered based on the second rendering signal, and the rotation layer data and system layer data in the second image data are rendered based on the third rendering signal.

6. The method as described in claim 5, characterized in that, The electronic device also includes a window management service (WMS) and an interface synthesizer (SF); During the process of the display screen switching from a first state to a second state, generating a second rendering signal of a second frequency for the content layer data of the first application includes: In response to the received landscape / portrait screen switching command, the first application sends a rotation animation start command to the WMS; In response to the rotation animation start command, the WMS sends a first notification message to the SF, the first notification message indicating the identifier of the first application; The SF generates the second rendering signal in response to the first notification message.

7. The method as described in claim 5, characterized in that, The electronic device also includes a rendering module; The step of rendering the content layer data in the second image data based on the second rendering signal, and rendering the rotation layer data and system layer data in the second image data based on the third rendering signal, includes: The drawing and rendering module performs drawing and rendering processing on the content layer data in the second image data based on the second rendering signal, and performs drawing and rendering processing on the rotation layer data and system layer data in the second image data based on the third rendering signal.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Receive the user's third operation; In response to the third operation, close the first application; Receive the user's fourth operation; In response to the fourth operation, launch the second application; When the display screen is in the first state, the content layer data and system layer data in the fourth image data are rendered based on the fifth rendering signal to obtain the fourth rendered image; wherein, the generation frequency of the fifth rendering signal is the first frequency; The fourth rendered image is synthesized based on the fourth synthesized signal to obtain the fourth image; wherein the generation frequency of the fourth synthesized signal is the first frequency; The fourth image is then processed for display. During the process of the display screen switching from the first state to the second state, the content layer data, rotation layer data and system layer data in the fifth image data are rendered based on the sixth rendering signal to obtain the fifth rendered image; wherein, the generation frequency of the sixth rendering signal is the first frequency; The fifth rendered image is synthesized based on the fifth synthesized signal to obtain the fifth image; wherein the generation frequency of the fifth synthesized signal is the first frequency; The fifth image is then processed for display. The fourth image data is image data of the second application, and the fifth image data is image data of the second application; the first application is a video application, and the second video is a non-video application.

9. An image display method, characterized in that, The method is applied to an electronic device, the electronic device including a display screen, and the method includes: Receive the user's first action; In response to the first operation, launch the first application; When the display screen is in the first state, the sixth image data of the first application is rendered based on the rendering signal to obtain the sixth rendered image. The sixth rendered image is synthesized based on the synthesized signal to obtain the sixth image. The sixth image is then processed for display. During the process of the display screen switching from the first state to the second state, the seventh image data of the first application is drawn and rendered based on the rendering signal to obtain a seventh drawn and rendered image; the seventh drawn and rendered image includes first rendering data, second rendering data and third rendering data, the first rendering data is the drawn and rendered data of the target layer in the seventh drawn and rendered image, the second rendering data is the drawn and rendered data of the rotation layer in the seventh drawn and rendered image, and the third rendering data is the drawn and rendered data of the system layer in the seventh drawn and rendered image; Based on the synthesized signal, the second rendering data and the third rendering data are synthesized to obtain the seventh image; The seventh image is then processed for display. Wherein, the first state is landscape mode and the second state is portrait mode; or, the first state is portrait mode and the second state is landscape mode.

10. The method as described in claim 9, characterized in that, The method further includes: In response to the display screen switching to the second state, the eighth image data of the first application is rendered based on the rendering signal to obtain a fifth rendered image; the eighth rendered image includes fourth rendering data, fifth rendering data and sixth rendering data, the fourth rendering data is the rendering data of the target layer in the eighth rendered image, the fifth rendering data is the rendering data of the rotation layer in the eighth rendered image, and the sixth rendering data is the rendering data of the system layer in the eighth rendered image; The fourth rendering data, the fifth rendering data, and the sixth rendering data are synthesized based on the synthesized signal to obtain the eighth image; The eighth image is then processed for display.

11. The method as described in claim 9 or 10, characterized in that, The target layer is the content layer of the first application.

12. An electronic device, characterized in that, The electronic device includes a processor for invoking a computer program in memory to perform the method as described in any one of claims 1-11.

13. A chip, characterized in that, The chip includes a processor for calling a computer program in memory to perform the method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-11.

Images