Image display method and related device

Abstract

The embodiment of the invention provides an image display method and a related device, and is applied to the technical field of terminals. The first application and the second application are applications running in the foreground of the electronic equipment, and determining a first rendering priority of the first application according to a first initial rendering priority of the first application, whether the first application corresponds to a focus window and / or an application switching type of the first application; determining a second rendering priority of the second application according to a second initial rendering priority of the second application, whether the second application corresponds to a focus window and / or an application switching type of the second application; the first rendering priority of the first application is compared with the second rendering priority of the second application, when the first rendering priority is lower than the second rendering priority, the sending frequency of the Vsync is reduced or the sending of the Vsync is stopped for the first application, and the original sending frequency of the Vsync is kept for the second application, so that the fluency of the foreground application with the higher rendering priority is improved, and the user experience is improved. And user experience is improved.

Description

Image display method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 21, 2022, with application number 202210868302.5 and application name “Image Display Method and Related Devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to an image display method and related devices. Background Art

[0003] With the advancement of technology, electronic devices are supporting increasingly diverse functions. For example, more and more electronic devices support multiple foreground applications. Multiple foreground application scenarios include application switching, floating window display, or split-screen display. At the same time, application developers will also iterate more application features, such as picture-in-picture features in video applications.

[0004] However, multiple foreground applications will bring heavy rendering tasks to the system, causing excessive system load, lag, and affecting user experience.

[0005] Summary of the Invention

[0006] The present invention provides an image display method and related apparatus for electronic devices. By reducing the frequency of Vsync signals or stopping the transmission of Vsync signals for foreground applications with lower rendering priorities, the smoothness of foreground applications with higher rendering priorities is improved, thereby enhancing the user experience.

[0007] In a first aspect, an embodiment of the present application proposes an image display method, which is applied to an electronic device, and the method includes: determining a first rendering priority of a first application based on a first initial rendering priority of the first application, whether the first application corresponds to a focus window and / or an application switching type of the first application; wherein the first rendering initial priority is related to the application type of the first application and / or the rendering load of the first application, and the application switching type includes switching out of the application or switching in of the application; determining a second rendering priority of the second application based on a second initial rendering priority of the second application, whether the second application corresponds to a focus window and / or the application switching type of the second application, wherein the second rendering initial priority is related to the application type of the second application and / or the rendering load of the interface to be displayed of the second application; wherein the first application and the second application are applications running in the foreground of the electronic device; when the first rendering priority is lower than the second rendering priority, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; and sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application and the image rendered in the second application to obtain a target image; and displaying the target image.

[0008] In this way, by comparing the first rendering priority of the first application and the second rendering priority of the second application, for the foreground application with lower rendering priority, the sending frequency of the Vsync signal is reduced or the sending of the Vsync signal is stopped; for the foreground application with higher rendering priority, the original sending frequency of the Vsync signal is maintained, thereby improving the smoothness of the foreground application with higher rendering priority and enhancing the user experience.

[0009] Optionally, the electronic device is in a scenario where the foreground application is switched, the first application is the switching-out application, and the second application is the switching-in application; determining the first rendering priority of the first application includes: determining the first rendering priority of the first application based on the first rendering initial priority of the first application and the application switching type of the first application; determining the second rendering priority of the second application includes: determining the second rendering priority of the second application based on the second rendering initial priority of the second application and the application switching type of the second application.

[0010] In this way, when the electronic device is in a scenario where the foreground application is switched, the first rendering priority of the first application is determined based on the first rendering initial priority of the first application and the application switching type of the first application; the second rendering priority of the second application is determined based on the second rendering initial priority of the second application and the application switching type of the second application, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0011] Optionally, the first rendering priority of the first application is determined based on the first rendering initial priority of the first application and the application switching type of the first application, including: based on the first application being a switched-out application, lowering the first rendering initial priority of the first application to obtain the first rendering priority of the first application; and the second rendering priority of the second application is determined based on the second rendering initial priority of the second application and the application switching type of the second application, including: based on the second application being a switched-in application, increasing the second rendering initial priority of the second application to obtain the second rendering priority of the second application.

[0012] In this way, in order to improve the smoothness of applications switching from the background to the foreground, based on the first application being the switched-out application, the first rendering initial priority of the first application is lowered to obtain the first rendering priority of the first application; based on the second application being the switched-in application, the second rendering initial priority of the second application is increased to obtain the second rendering priority of the second application, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0013] Optionally, the electronic device is in a split-screen display scenario, and the first application and the second application are respectively displayed in two different windows; determining the first rendering priority of the first application includes: determining the first rendering priority of the first application based on the first rendering initial priority of the first application and whether the first application corresponds to the focus window; determining the second rendering priority of the second application includes: determining the second rendering priority of the second application based on the second rendering initial priority of the second application and whether the second application corresponds to the focus window.

[0014] In this way, based on the first rendering initial priority of the first application, the first rendering priority of the first application is determined according to whether the first application corresponds to the focus window; based on the second rendering initial priority of the second application, the second rendering priority of the second application is determined according to whether the second application corresponds to the focus window, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0015] Optionally, the first rendering priority of the first application is determined based on the first rendering initial priority of the first application and whether the first application corresponds to the focus window, including: based on the first application corresponding to the non-focus window, lowering the first rendering initial priority of the first application to obtain the first rendering priority of the first application; based on the second rendering initial priority of the second application and whether the second application corresponds to the focus window, determining the second rendering priority of the second application, including: based on the second application corresponding to the focus window, increasing the second rendering initial priority of the second application to obtain the second rendering priority of the second application.

[0016] In this way, the focus window is a window that the user pays more attention to. In order to improve the fluency of the focus window, based on the non-focus window corresponding to the first application, the first rendering initial priority of the first application is lowered to obtain the first rendering priority of the first application; based on the focus window corresponding to the second application, the second rendering initial priority of the second application is increased to obtain the second rendering priority of the second application.

[0017] Optionally, the electronic device is in a scenario where a floating window is displayed; determining a first rendering priority of a first application, including: determining the first rendering priority of the first application based on the first rendering initial priority of the first application and whether the first application corresponds to a focus window; determining a second rendering priority of a second application, including: determining the second rendering priority of the second application based on the second rendering initial priority of the second application and whether the second application corresponds to a focus window.

[0018] In this way, based on the first rendering initial priority of the first application, the first rendering priority of the first application is determined according to whether the first application corresponds to the focus window; based on the second rendering initial priority of the second application, the second rendering priority of the second application is determined according to whether the second application corresponds to the focus window, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0019] Optionally, the first rendering priority of the first application is determined based on the first rendering initial priority of the first application and whether the first application corresponds to a focus window, including: based on the first application corresponding to a non-focus window, lowering the first rendering initial priority of the first application to obtain the first rendering priority of the first application; and the second rendering priority of the second application is determined based on the second rendering initial priority of the second application and whether the second application corresponds to a focus window, including: based on the second application corresponding to a focus window, increasing the second rendering initial priority of the second application to obtain the second rendering priority of the second application.

[0020] In this way, the focus window is a window that the user pays more attention to. In order to improve the fluency of the focus window, based on the non-focus window corresponding to the first application, the first rendering initial priority of the first application is lowered to obtain the first rendering priority of the first application; based on the focus window corresponding to the second application, the second rendering initial priority of the second application is increased to obtain the second rendering priority of the second application.

[0021] Optionally, before determining the first rendering priority of the first application and determining the second rendering priority of the second application, it also includes: obtaining the rendering area of ​​the first application and the rendering area of ​​the second application; determining the first rendering priority of the first application and the second rendering priority of the second application, including: when the rendering area of ​​the first application and the rendering area of ​​the second application are both greater than a first preset value, determining the first rendering priority of the first application and determining the second rendering priority of the second application.

[0022] In this way, when the rendering area of ​​the first application and the rendering area of ​​the second application are both greater than the first preset value, the first rendering priority of the first application and the second rendering priority of the second application are determined, and the rendering of the first application and the second application are controlled according to the first rendering priority and the second rendering priority, thereby reducing the rendering load of the system and improving the fluency of the application.

[0023] Optionally, before determining the first rendering priority of the first application and determining the second rendering priority of the second application, it also includes: obtaining window information of the application running in the foreground and / or switching information of the application running in the foreground; determining that the application running in the foreground includes the first application and the second application based on the window information of the application running in the foreground and / or switching information of the application running in the foreground.

[0024] In this way, it is possible to determine whether the foreground running applications include the first application and the second application based on the window information of the foreground running application and / or the foreground running application switching information. When it is determined that the foreground running applications include the first application and the second application, the first rendering priority of the first application and the second rendering priority of the second application are compared, and then rendering control of the first application and the second application is performed to improve the smoothness of the application. When it is determined that the foreground running applications include the first application but not the second application, it is not necessary to obtain the first rendering priority of the first application, thereby saving system resources.

[0025] In a second aspect, an embodiment of the present application proposes an image display method, which is applied to an electronic device, and the method includes: displaying a multitasking interface, the multitasking interface including a first application and a second application; receiving a first operation from a user; the first operation is used to switch the first application in and out of the second application; in response to the first operation, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; in response to the first operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the second application to obtain a target image; and displaying the target image.

[0026] Among them, the first operation can be a click operation in Figure 12, the first application can be a video application in Figure 12, and the second application can be a social application in Figure 12.

[0027] In this way, for the incoming application, the Vsync signal is sent to it at the original frequency, and for the outgoing application, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the incoming application can have sufficient rendering resources and improve the fluency of the incoming application.

[0028] In a third aspect, an embodiment of the present application proposes an image display method, which is applied to an electronic device, and the method includes: displaying a first interface, the first interface including a first application; receiving a second operation of the user for split-screen display of the first application and the second application; displaying the first application in the first window and the second application in the second window in response to the second operation; receiving a third operation on the second application in the second window; sending a Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application in response to the third operation; sending a Vsync signal to the second application at a second frequency in response to the second operation; wherein the first frequency is less than the second frequency; synthesizing the rendered image in the first application and the rendered image in the second application to obtain a target image; and displaying the target image.

[0029] Among them, the first application can be the word application as shown in Figure 14, the second application can be the ppt application as shown in Figure 14, the first window can be the window corresponding to the word application, the second window can be the window corresponding to the ppt application, and the third operation can be the operation of the user making a ppt in the ppt application corresponding to the second window.

[0030] In this way, for the application in the focus window, the Vsync signal is sent to it at the original frequency, and for the application in the non-focus window, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the application in the focus window can have sufficient rendering resources, thereby improving the smoothness of the application in the focus window.

[0031] In a fourth aspect, an embodiment of the present application proposes an image display method, which is applied to an electronic device, and the method includes: displaying a first application in full screen; receiving a fourth operation for triggering a second application; in response to the fourth operation, displaying the second application in full screen, and displaying the first application in a floating window on the second application; receiving a fifth operation on the second application; in response to the fifth operation, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; in response to the fifth operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the second application to obtain a target image; and displaying the target image.

[0032] Among them, the first application can be a video application as shown in a in Figure 15, the second application can be a social application as shown in b in Figure 15, the fourth operation can be the operation of clicking on the message reminder of the social application as shown in a in Figure 15, and the fifth operation can be the operation of replying to the message as shown in b in Figure 15.

[0033] In this way, for the application in the focus window, the Vsync signal is sent to it at the original frequency, and for the application in the non-focus window, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the application in the focus window can have sufficient rendering resources, thereby improving the smoothness of the application in the focus window.

[0034] In a fifth aspect, an embodiment of the present application proposes an image display method, which is applied to an electronic device, and the method includes: receiving a sixth operation of a user returning from a first application to a desktop application; in response to the sixth operation, stopping sending a Vsync signal to the first application, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending a Vsync signal to the first application; in response to the sixth operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the desktop application to obtain a target image; and displaying the target image.

[0035] Among them, the first application can be a social application as shown in a in Figure 13, the second application can be a desktop application as shown in b in Figure 13, and the sixth operation can be an operation of swiping up to return to the desktop as shown in a in Figure 13.

[0036] In this way, for the incoming application, the Vsync signal is sent to it at the original frequency, and for the outgoing application, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the incoming application can have sufficient rendering resources and improve the fluency of the incoming application.

[0037] In a sixth aspect, an embodiment of the present application provides an electronic device, which includes: a touch screen, wherein the touch screen includes a touch panel and a display screen; one or more processors; a memory; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device performs the following steps: determining a first rendering priority of the first application based on the first initial rendering priority of the first application, whether the first application corresponds to the focus window and / or the application switching type of the first application; wherein the first rendering initial priority is related to the application type of the first application and / or the rendering load of the first application, and the application switching type includes switching out of the application or switching in of the application; ... second application The second rendering priority of the second application is determined based on the second initial rendering priority, whether the second application corresponds to the focus window and / or the application switching type of the second application, wherein the second rendering initial priority is related to the application type of the second application and / or the rendering load of the interface to be displayed of the second application; wherein the first application and the second application are applications running in the foreground of the electronic device; when the first rendering priority is lower than the second rendering priority, a vertical synchronization signal Vsync signal is sent to the first application at a first frequency, or the Vsync signal is stopped from being sent to the first application; and a Vsync signal is sent to the second application at a second frequency; wherein the first frequency is less than the second frequency; the rendered image in the first application is synthesized with the rendered image in the second application to obtain a target image; and the target image is displayed.

[0038] In the seventh aspect, an embodiment of the present application provides an electronic device, which includes: a touch screen, wherein the touch screen includes a touch panel and a display screen; one or more processors; a memory; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device performs the following steps: displaying a multi-tasking interface, the multi-tasking interface includes a first application and a second application; receiving a first operation of the user; the first operation is used to switch the first application in and out of the second application; in response to the first operation, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; in response to the first operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the second application to obtain a target image; and displaying the target image.

[0039] In an eighth aspect, an embodiment of the present application provides an electronic device, comprising: a touch screen, wherein the touch screen comprises a touch panel and a display screen; one or more processors; a memory; a plurality of applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, cause the electronic device to perform the following steps: displaying a first interface, wherein the first interface includes a first application; receiving a second operation of a user for splitting the screen display of the first application and the second application; displaying the first application in the first window and the second application in the second window in response to the second operation; receiving a third operation on the second application in the second window; sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application in response to the third operation; sending a Vsync signal to the second application at a second frequency in response to the second operation; wherein the first frequency is less than the second frequency; synthesizing the rendered image in the first application with the rendered image in the second application to obtain a target image; and displaying the target image.

[0040] In a ninth aspect, an embodiment of the present application provides an electronic device, comprising: a touch screen, wherein the touch screen comprises a touch panel and a display screen; one or more processors; a memory; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, cause the electronic device to perform the following steps: displaying a first application in full screen; receiving a fourth operation for triggering a second application; in response to the fourth operation, displaying the second application in full screen, and displaying the first application in a floating window on the second application; receiving a fifth operation for the second application; in response to the fifth operation, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; in response to the fifth operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the second application to obtain a target image; and displaying the target image.

[0041] In the tenth aspect, an embodiment of the present application provides an electronic device, which includes: a touch screen, wherein the touch screen includes a touch panel and a display screen; one or more processors; a memory; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device performs the following steps: receiving a sixth operation of the user returning from the first application to the desktop application; in response to the sixth operation, stopping sending the Vsync signal to the first application, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; in response to the sixth operation, sending the Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the desktop application to obtain a target image; and displaying the target image.

[0042] In the eleventh aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the processor being used to call a program in the memory so that the electronic device executes the method provided in the first, second, third, fourth or fifth aspect above.

[0043] In the twelfth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method provided in the first, second, third, fourth or fifth aspect above.

[0044] In the thirteenth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is run, it enables the computer to execute the method provided in the first, second, third, fourth or fifth aspect above.

[0045] The beneficial effects of the possible implementation methods of the sixth to thirteenth aspects mentioned above can be referred to the beneficial effects brought about by the first aspect mentioned above, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0047] FIG2 is a block diagram of the software structure of a terminal device provided in an embodiment of the present application;

[0048] FIG3 is a schematic diagram of module interaction provided in an embodiment of the present application;

[0049] FIG4 is a schematic diagram of module interaction provided in an embodiment of the present application;

[0050] FIG5 is a schematic diagram of an electronic device interface display processing flow provided by an embodiment of the present application;

[0051] FIG6 is a schematic diagram of an electronic device interface display processing flow provided by an embodiment of the present application;

[0052] FIG7 is a schematic diagram of an electronic device interface display processing flow provided by an embodiment of the present application;

[0053] FIG8 is a schematic diagram of an electronic device interface display processing flow provided by an embodiment of the present application;

[0054] FIG9 is a flow chart of an image display method provided in an embodiment of the present application;

[0055] FIG10a is a schematic diagram of a rendering control for a foreground application provided by an embodiment of the present application;

[0056] FIG10b is a schematic diagram of an electronic device interface display processing flow provided by an embodiment of the present application;

[0057] FIG11a is a schematic diagram of a rendering control for a foreground application provided by an embodiment of the present application;

[0058] FIG11b is a schematic diagram of an electronic device interface display processing flow provided by an embodiment of the present application;

[0059] FIG12 is a schematic diagram of a first scenario applicable to an image display method provided in an embodiment of the present application;

[0060] FIG13 is a schematic diagram of a second scenario applicable to an image display method provided in an embodiment of the present application;

[0061] FIG14 is a schematic diagram of a third scenario applicable to an image display method provided in an embodiment of the present application;

[0062] FIG15 is a schematic diagram of a fourth scenario applicable to an image display method provided in an embodiment of the present application;

[0063] FIG16 is a schematic flow chart of an image display method provided in an embodiment of the present application;

[0064] FIG17 is a schematic structural diagram of an image display device provided in an embodiment of the present application;

[0065] FIG18 is a schematic diagram of the hardware structure of an image display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] To facilitate a clear description of the technical solutions of the embodiments of the present application, some of the terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0067] 1) System Applications: These are applications developed by the manufacturer of an electronic device to solve various practical problems and improve the user experience. For example, system applications include the system desktop and the system gallery.

[0068] 2) Third-party applications: In addition to applications developed by electronic device manufacturers, there are also applications developed by individuals or companies for specific electronic devices. These are called third-party applications. For example, third-party applications include WeChat and Toutiao.

[0069] 3) Frame: This refers to the smallest unit of a single image in an interface display. A frame can be understood as a still image. Displaying multiple frames in rapid succession can create the illusion of motion.

[0070] 4) Frame rate: refers to the number of images that can be generated in 1 second, measured in frames per second (fps).

[0071] 5) Refresh rate: This refers to the number of frames per second that refresh an image. It can also be understood as the number of times per second that the graphics processor in an electronic device refreshes the screen. A higher frame rate produces smoother and more realistic animation. The more frames per second, the smoother the displayed motion. The unit is hertz (Hz).

[0072] It should be noted that before the interface displays a frame, it usually needs to go through processes such as drawing, rendering, and synthesis.

[0073] 6) Frame drawing: This refers to the image drawing of the display interface. The display interface can be composed of one or more views. Each view can be drawn by the visual controls of the view system. Each view is composed of subviews. A subview corresponds to a small widget in the view. For example, a subview corresponds to a symbol in the image view.

[0074] 7) Frame rendering: This is to shade the drawn view or add 3D effects, etc. For example, 3D effects can be lighting effects, shadow effects, and texture effects.

[0075] 8) Frame synthesis: It is the process of synthesizing multiple or more rendered views into a display interface.

[0076] 9) Other terms

[0077] In the embodiments of this application, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the terms "first chip" and "second chip" are used solely to distinguish between different chips and do not define their order. Those skilled in the art will understand that terms such as "first" and "second" do not define the quantity or execution order, and do not necessarily define differences.

[0078] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0079] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0080] With the development of technology, the display functions of electronic devices are becoming increasingly diverse. For example, more and more electronic devices support multiple foreground applications. Multiple foreground application scenarios include application switching, floating window display, or split-screen display. At the same time, application developers will also iterate more application features, such as picture-in-picture features in video applications.

[0081] When multiple applications are in the foreground of an electronic device, multiple rendering tasks must run concurrently, posing significant challenges to hardware such as the central processing unit (CPU) and graphics processing unit (GPU). Because resources are limited, when foreground applications follow a defined rendering pipeline, they frequently compete for resources. This can cause foreground applications to fail to complete rendering and / or compositing tasks within the expected Vsync period, resulting in application interface lag and a poor user experience.

[0082] In view of this, the image display method provided in the embodiment of the present application, when there are multiple foreground applications, the electronic device will control the rendering according to the rendering priority of each foreground application, reduce the overall load of the system, improve the fluency of the application interface, and thus enhance the user experience.

[0083] The image display method provided in the embodiment of the present application can be applied to electronic devices that support multiple foreground applications.

[0084] Electronic devices include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of the present application do not limit the specific technologies and specific device forms adopted by the electronic devices. In order to better understand the embodiments of the present application, the structure of the electronic devices in the embodiments of the present application is introduced below:

[0085] 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0086] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0088] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0089] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0090] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0091] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is for illustrative purposes only and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from those in the above embodiments, or a combination of multiple interface connection methods.

[0092] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device 100 via the power management module 141.

[0093] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0094] Electronic device 100 implements display functionality through a graphics processing unit (GPU), display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0095] Display screen 194 is used to display images, videos, and receive sliding operations. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0096] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0097] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0098] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0099] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0100] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100. Figure 2 is a block diagram of the software structure of the electronic device of the embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, which are, from top to bottom, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library and the kernel layer.

[0101] The application layer can include a series of application packages. As shown in Figure 2, the application package can include desktop launcher Launcher, system user interface SystemUI, phone, camera, calendar, WeChat and other applications.

[0102] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0103] As shown in FIG2 , the application framework layer may include a system server, a rendering orchestrator, a graphics system, a view system, etc., wherein the system server may include a window manager, a package manager, an activity manager, an input system, etc.

[0104] The graphics system is used to manage the graphics data of each application, including frame buffers, graphics drawing pipelines, etc.

[0105] The window manager service (WMS) manages windowed programs. It can obtain the display size, determine whether a status bar is present, lock the screen, and take screenshots.

[0106] The package manager is used for program management within the system, such as application installation, uninstallation, and upgrades.

[0107] The Activity Manager Service (AMS) is a core service of Android. It is responsible for starting, switching, and scheduling multiple important components in the system, managing and scheduling application processes, and tracking and managing the switching of foreground applications.

[0108] The input system is used to process input events, for example, receiving user touch operations.

[0109] The rendering orchestrator is used to provide a unified logic update and callback processing mechanism for the application, and is responsible for requesting and receiving APP-Vsync signals, providing a stable operation mechanism for the application.

[0110] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0111] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0112] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0113] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0114] The system library may include multiple functional modules, such as a surface manager, a 3D graphics processing library (e.g., OpenGL ES), a graphics synthesis module, a compositor client, and a graphics rendering management module.

[0115] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0116] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0117] The graphics synthesis module includes a connection manager, a signal simulator, a graphics synthesizer, and a synthesis controller.

[0118] The connection manager is used to establish a connection between the application and the graphics synthesis module when the application is started, receive layer information related to the application, and send a Vsync signal to the application.

[0119] The signal simulator is used to modulate the software Vsync signal based on the hardware Vsync signal.

[0120] The graphics compositor is used to synthesize one or more rendered views into a display interface. For example, in a multi-window display, the images of multiple windows are synthesized after rendering.

[0121] The compositor client is used to encapsulate the external functional interface of the graphics compositor, including layer creation, destruction, rearrangement, event distribution, rendering control, etc.

[0122] The graphics rendering management module is responsible for recording and managing the rendering status of applications in the system, and controlling the rendering of each application according to the rendering control strategy.

[0123] The hardware abstraction layer can contain multiple library modules, such as the hardware configuration module (hwcomposer, HWC), the camera library module, etc. The Android system can load the corresponding library module for the device hardware, thereby enabling the application framework layer to access the device hardware.

[0124] The kernel layer is the layer between hardware and software. It drives the hardware, making it work. The kernel layer includes at least the CPU driver, sensor driver, display driver, audio driver, camera driver, backlight driver, and graphics processor driver.

[0125] Hardware can be audio devices, Bluetooth devices, LCD screens, cameras, etc.

[0126] The display process of the electronic device 100 is described below.

[0127] Taking the interface display in the WeChat app as an example, the WeChat app uses the view system in the application framework layer to call the image drawing module in the system library to draw the image, and then calls the image rendering module to render the drawn image. The WeChat app sends the rendered image to the graphics synthesis module. The graphics synthesis module calls the image synthesis module in the system library to synthesize the rendered image. The graphics synthesis module sends the synthesized image to the hardware abstraction layer (HWC). The HWC sends the synthesized image to the LCD / LED driver in the core layer. The LCD / LED driver drives the screen to refresh the displayed image.

[0128] It should be noted that in order to ensure the smoothness of display and avoid display freezes and other phenomena, electronic devices generally display based on Vsync signals to synchronize the processes of image drawing, rendering, synthesis and screen refresh display.

[0129] It can be understood that the Vsync signal is a periodic signal, and the Vsync period can be set according to the screen refresh frame rate. For example, when the screen refresh frame rate is 60Hz, the Vsync period can be 16.6ms, that is, the electronic device generates a control signal every 16.6ms to trigger the Vsync period.

[0130] For example, the modules involved in the image display method provided in the embodiments of the present application may include an application, a graphics synthesis module, a graphics rendering management module, a system server, and a rendering orchestrator. Figures 3 and 4 are interaction diagrams of the various modules. As shown in Figures 3 and 4, the system server may include a window manager and an activity manager, the graphics rendering management module may include a rendering load manager and a rendering controller, and the graphics synthesis module may include a connection manager, a signal simulator, and a synthesis controller.

[0131] The application receives and responds to a preset operation, establishing a Vsync signal transmission channel with the graphics synthesis module through the connection manager, wherein the preset operation is used to instruct to open an application or switch applications.

[0132] The system server monitors changes in the foreground application, records the foreground application's window information in the window manager, records the foreground application's activity information in the activity manager, and sends the foreground application's window information and activity information to the rendering load manager. Changes in the foreground application include adding a foreground application or switching the foreground application.

[0133] The foreground application creates a rendering thread, sends rendering attribute information to the rendering load manager, and registers a rendering callback function with the rendering orchestrator. The rendering attribute information includes information such as the initial rendering priority.

[0134] The rendering controller obtains window information and activity information of the foreground application from the rendering load manager, and determines the number of foreground applications based on the window information and activity information of the foreground application.

[0135] It should be noted that there is a communication connection between the rendering load manager and the rendering controller.

[0136] When the number of foreground applications is greater than 1, the rendering controller obtains the initial rendering priority from the rendering load manager, determines the rendering priority of each foreground application based on the window information of the foreground application, the activity information of the foreground application and the initial rendering priority, obtains the rendering control strategy of multiple foreground applications, and sends the rendering control strategy to the synthesis controller.

[0137] The signal simulator obtains the rendering control policy from the synthesis controller and sends corresponding Vsync signals to multiple foreground applications based on the rendering control policy and the established Vsync signal transmission channel. The Vsync signal can be a software Vsync signal (for example, APP-Vsync), which is a Vsync signal modulated by the signal simulator based on the hardware Vsync signal.

[0138] It can be understood that when the electronic device is an LCD / LED screen, the hardware Vsync signal is the HW-Vsync signal generated by the LCD / LED driver. The LCD / LED driver sends the HW-Vsync signal to the graphics synthesis module through the HWC. The signal simulator in the graphics synthesis module modulates the software Vsync signal based on the hardware Vsync signal.

[0139] It should be noted that there is a communication connection between the connection manager, the signal simulator and the synthesis controller.

[0140] Multiple foreground applications call the rendering callback function in the rendering orchestrator according to the corresponding Vsync signal to perform rendering operations.

[0141] For example, when there are three foreground applications (for example, application 1, application 2, and application 3), and the rendering priority of application 1, application 2, and application 3 is the same, you can refer to the rendering control method in Figure 3. The rendering control method can normally send an APP-Vsync signal to each foreground application. After receiving the APP-Vsync signal, application 1, application 2, and application 3 can all be rendered according to the APP-Vsync signal.

[0142] For another example, when the rendering priorities of application 1, application 2, and application 3 are different, and the rendering priority of application 1 is higher than the rendering priority of application 2 and application 3, the interaction diagram of each module is shown in Figure 4. The rendering control method may be to send the APP-Vsync signal to application 1 normally, and stop sending the APP-Vsync signal to application 2 and application 3. Specifically, after receiving the APP-Vsync signal, application 1 can render according to the APP-Vsync signal; however, after application 2 and application 3 no longer receive the APP-Vsync signal, the display interface will no longer be refreshed.

[0143] The following describes the generation and synchronization process of Vsync in a possible implementation.

[0144] It should be noted that the Vsync signal in electronic devices can be divided into software Vsync signal and hardware Vsync signal. Software Vsync signal includes APP-Vsync signal and SF-Vsync signal. Among them, APP-Vsync signal is used to trigger the drawing and rendering process, and SF-Vsync signal is used to trigger the synthesis process. Hardware Vsync signal (HW-Vsync signal) is used to trigger the screen display refresh process.

[0145] Typically, the software Vsync signal and the hardware Vsync signal maintain periodic synchronization. For example, if the HW-Vsync signal switches from 60Hz to 120Hz, the APP-Vsync signal and the SF-Vsync signal change synchronously, switching from 60Hz to 120Hz.

[0146] Exemplarily, the application sends a Vsync request to the graphics synthesis module, the graphics synthesis module sends an HW-Vsync request to the HWC, the HWC sends an HW-Vsync request to the LCD / LED driver, the LCD / LED driver generates an HW-Vsync signal, feeds the HW-Vsync signal back to the HWC, the HWC feeds the HW-Vsync signal back to the graphics synthesis module, and the graphics synthesis module sends the HW-Vsync signal to the application.

[0147] The application draws and renders the image according to the received HW-Vsync signal; the graphics synthesis module synthesizes the image according to the HW-Vsync signal; and the HWC controls the LCD / LED screen to refresh the display image through the LCD / LED driver of the core layer according to the HW-Vsync signal.

[0148] Because the LCD / LED driver generates a HW-Vsync signal and feeds it back to the HWC, electronic devices consume significant power. To reduce power consumption, the graphics synthesis module can generate a software Vsync signal to control image drawing, rendering, and synthesis processes.

[0149] Specifically, after receiving the HW-Vsync signal from the HWC, the graphics synthesis module collects the HW-Vsync signal timestamp. If the graphics synthesis module receives the HW-Vsync signal multiple times in a row, it generates a software Vsync signal based on the recorded HW-Vsync signal period and sends the software Vsync signal to the application to synchronize the image drawing, rendering, and synthesis processes.

[0150] For ease of understanding, an example display process of an electronic device is given for reference.

[0151] Figure 5 is a schematic diagram of an electronic device interface display processing flow in a possible implementation. Exemplarily, in chronological order, the content displayed by the electronic device corresponds to frame 1, frame 2, frame 3, frame 4, and frame 5.

[0152] It can be understood that when the screen refresh rate is 120Hz, the Vsync period is 8.3ms, when the screen refresh rate is 60Hz, the Vsync period is 16.6ms, and so on. In the embodiments of the present application, there is no specific limitation on the specific values ​​of the screen refresh rate and the Vsync period.

[0153] Specifically, when application 1 is rendering, taking frame 2 as an example, the application of the electronic device renders frame 2 through the view system of the application framework layer. After the rendering of frame 2 is completed, the application of the electronic device sends the rendered frame 2 to the graphics synthesis module (for example, surfaceflinger), and the graphics synthesis module synthesizes the rendered frame 2. After the synthesis of frame 1 is completed, the electronic device can start the display driver by calling the kernel layer to display the content corresponding to frame 2 on the screen. Frames 3 and 4 are also synthesized and displayed in a similar process to frame 2, which will not be repeated here.

[0154] As shown in Figure 5, during the five Vsync cycles, the interface of application 1 is displayed as frame 1, frame 2, frame 3, and frame 4 in sequence. The interface changes continuously and smoothly without any lag.

[0155] It should be noted that the time it takes for an application to draw a rendered image and the time it takes to synthesize an image are related to the system load and other states of the electronic device. When each step in the rendering pipeline, including operations such as drawing, rendering, and synthesis, can be completed within a fixed Vsync cycle, the display interface of the electronic device changes smoothly. When the system load is large (for example, two applications are rendering at the same time), the time it takes for an application to draw a rendered image and / or the time it takes to synthesize an image increases, which may cause a certain frame of image to not be displayed in time, resulting in frame loss, causing the electronic device to display stuck and reducing the user experience.

[0156] For example, when Application 1 and Application 2 are rendering simultaneously, with Application 2 rendering timed out as an example, Figure 6 illustrates a possible implementation of an electronic device interface display processing flow. The electronic device renders, synthesizes, and displays Frames 2, 3, and 4 in Figure 6, respectively. For details, please refer to the corresponding description in Figure 5.

[0157] Unlike Figure 5, Application 1 and Application 2 are running in the foreground at the same time. Due to insufficient resources and other reasons, the rendering time of Application 2's frame 3 increases, and the rendering cannot be completed within the Vsync period. When synthesizing frame 3, Application 1's frame 3 can participate in the synthesis work normally, but Application 2's frame 3 cannot participate in the synthesis of frame 3 normally due to rendering timeout. As a result, when synthesizing frame 3, Frame 2 in Application 2 participates in the synthesis of frame 3. As a result, when the electronic device displays Application 1's interface, it displays Frame 1, Frame 2, Frame 3, and Frame 4 in sequence, while Application 2's interface displays Frame 1, Frame 2, Frame 2, and Frame 3 in sequence. The increased display time of Frame 2 in Application 2 causes the display of Application 2 to freeze, reducing the user experience.

[0158] It should be noted that the application's rendering thread includes rendering operations in the CPU and rendering in the GPU. The rendering operation will handle some events, including: responding to the event, calling the DoFrame function to layout the application interface, and sending various attribute information of the interface to the GPU through rendering instructions. The GPU performs specific rendering operations according to the rendering instructions.

[0159] For example, when Application 1 and Application 2 are rendering simultaneously, using synthesis timeout as an example, Figure 7 is a schematic diagram of an electronic device interface display processing flow in a possible implementation. The electronic device renders, synthesizes, and displays Frames 2, 3, and 4 in Figure 7, respectively. For details, please refer to the corresponding description in Figure 5.

[0160] Unlike Figure 5, Application 1 and Application 2 are running in the foreground at the same time. Due to insufficient resource supply and other reasons, the time consumed in synthesizing Frame 3 increases, and the synthesis cannot be completed within the Vsync period. As a result, when the electronic device is displayed, the interface of Application 1 is displayed as Frame 1, Frame 2, Frame 2 and Frame 3 in sequence, and the interface of Application 2 is displayed as Frame 1, Frame 2, Frame 2 and Frame 3 in sequence. The display duration of Frame 2 in Application 1 and Application 2 is increased, resulting in display freezes of Application 1 and Application 2, which reduces the user experience.

[0161] For example, using the example of both rendering and compositing timeouts, Figure 8 illustrates a possible implementation of an electronic device interface display process. In chronological order, the content displayed by the electronic device corresponds to frames 1, 2, 3, 4, and 5. The electronic device renders, composites, and displays frames 2, 3, and 4 in Figure 8, respectively. For details, please refer to the corresponding description in Figure 5.

[0162] Unlike Figure 5, Application 1 and Application 2 are running in the foreground simultaneously. Due to insufficient resource supply and other reasons, the rendering time of frame 3 in Application 2 increases, and due to insufficient resource supply and other reasons, the synthesis time of frame 3 increases. Therefore, it cannot be rendered or synthesized within the Vsync cycle. As a result, when the electronic device displays, the interface of Application 1 is displayed as frame 1, frame 2, frame 2, and frame 3 in sequence, while the interface of Application 2 is displayed as frame 1, frame 2, frame 2, and frame 2 in sequence. The display length of frame 2 in both Application 1 and Application 2 increases, causing the display of Application 1 and Application 2 to be stuck, which reduces the user experience.

[0163] As can be seen from Figures 5 to 8, when multiple applications are rendering simultaneously, a heavy rendering load may occur, and the application may not be able to complete the rendering and / or synthesis work within the Vsync cycle as expected, causing the application interface to freeze and affecting the user experience.

[0164] The image display method of the embodiment of the present application is described in detail below through specific embodiments. The following embodiments can be combined with each other or implemented independently, and the same or similar concepts or processes may not be repeated in some embodiments.

[0165] FIG9 is a flow chart of an image display method provided by an embodiment of the present application. The method shown in FIG9 can be executed by the electronic device 100 shown in FIG1 . As shown in FIG9 , the method includes:

[0166] S901. The application receives and responds to a preset operation. The system server monitors changes in the foreground application, records the window information and activity information of the foreground application in the system server, and sends the window information and activity information of the foreground application to the graphics rendering management module.

[0167] The preset operation is used to instruct opening an application or switching applications.

[0168] In the embodiment of the present application, the foreground application can be understood as an application that is visible and running on the desktop taskbar of the terminal device, for example, a running game application.

[0169] In the embodiments of the present application, window information can be understood as the corresponding window interface of each application during operation, and each window interface has its own corresponding information and data. This information and data may include attribute information such as window size, window position, and window layout. For example, WeChat has a corresponding WeChat window interface during operation, and WeChat window information may include information such as the position of each button and the size of the dialog box.

[0170] In the embodiments of the present application, activity information can be understood as information generated and stored in the activity manager when an application on a terminal device is launched or the foreground application switches, so as to facilitate tracking and managing the switching of foreground applications. For example, when the foreground application switches from a social application to a game application, activity information will be generated and recorded in the activity manager.

[0171] In an embodiment of the present application, the system server includes a window manager and an activity manager. Window information of foreground applications is recorded in the window manager, and activity information of foreground applications is recorded in the activity manager. A graphics rendering management module includes a rendering load manager. The graphics rendering management module records the acquired information in the rendering load manager. The system server sends the foreground application window information and activity information to the graphics rendering management module, which is then stored in the rendering load manager.

[0172] For example, when the application receives a preset operation for instructing to open the application or switch the application, the system server monitors the changes of the foreground application and generates window information and activity information.

[0173] For example, a social application and a game application have been pre-launched on the electronic device, the social application runs in the foreground, and the game application runs in the background. When the user switches the foreground application of the electronic device from the social application to the game application, the foreground application is switched, and window information and activity information are generated.

[0174] S902: The foreground application sends rendering attribute information to the graphics rendering management module based on the created rendering thread, wherein the rendering attribute information includes information such as the initial rendering priority.

[0175] In an embodiment of the present application, after the foreground application sends the rendering attribute information to the graphics rendering management module, it is also stored in the rendering load manager.

[0176] In an embodiment of the present application, different applications may have different initial rendering priorities. Exemplarily, the initial rendering priority may be divided into high-medium-low. For example, the initial rendering priority may be set to 0-2 in order from high to low, with 0 corresponding to high initial rendering priority, 1 corresponding to medium initial rendering priority, and 2 corresponding to low initial rendering priority. Of course, the initial rendering priority may also be set to 0-2 in order from low to high, with 2 corresponding to high initial rendering priority, 1 corresponding to medium initial rendering priority, and 0 corresponding to low initial rendering priority. In an embodiment of the present application, there is no specific limitation on the division of the initial rendering priority.

[0177] In one possible implementation, the initial rendering priority can be set based on the application type. Specifically, the initial rendering priority of system applications can be set higher than the initial rendering priority of third-party applications. For example, system applications may include desktop launchers, system user interfaces, etc., while third-party applications may include WeChat, Toutiao, etc.

[0178] In another possible implementation, the initial rendering priority can be set based on the rendering load of the application. Specifically, the initial rendering priority of an application with a large rendering load can be set higher than the initial rendering priority of an application with a small rendering load. For example, if the rendering load of a gaming application is greater than that of a social networking application, the initial rendering priority of the gaming application can be set higher than that of the social networking application.

[0179] In another possible implementation, the initial rendering priority can be set based on the application type and the rendering load of the application. Specifically, when multiple applications have the same application type, the initial rendering priority can be set based on the rendering load of the application, and the initial rendering priority of the application with a large rendering load is set higher than the initial rendering priority of the application with a small rendering load. For example, when the two foreground applications are a third-party game application and a third-party social application, when the rendering load of the third-party game application is greater than the rendering load of the third-party social application, the initial rendering priority of the third-party game application is set higher than the initial rendering priority of the third-party social application.

[0180] For example, before performing rendering, the foreground application creates a rendering thread and sends rendering attribute information such as the initial rendering priority in the rendering thread to the rendering load manager.

[0181] S903: The graphics rendering management module may determine the number of foreground applications according to the window information of the foreground applications and the activity information of the foreground applications.

[0182] Exemplarily, the graphics rendering management module can determine whether the foreground application has changed based on the window information of the foreground application and the activity information of the foreground application, such as the switching or addition of the foreground application, and obtain which applications are in the foreground based on the window information of the foreground application and the activity information of the foreground application, thereby determining the number of foreground applications.

[0183] It is understandable that the embodiment of the present application may also use any other method to determine the number of foreground applications, and the embodiment of the present application does not make any specific limitation.

[0184] S904. When the number of foreground applications is greater than 1, the graphics rendering management module determines the rendering priority of each foreground application based on the window information of the foreground application, the activity information of the foreground application, and the initial rendering priority, obtains the rendering control strategy corresponding to the multiple foreground applications, and sends the rendering control strategy to the graphics synthesis module.

[0185] In an embodiment of the present application, the rendering controller determines the rendering priority of each foreground application based on the window information of the foreground application, the activity information of the foreground application, the initial rendering priority and other information stored in the rendering load manager, and obtains the rendering control strategy corresponding to multiple foreground applications.

[0186] In an embodiment of the present application, the rendering priority can be used to control the rendering of multiple foreground applications to varying degrees. For example, when rendering multiple foreground applications, the higher the rendering priority of the foreground application, the more computing resources are given, which can improve the smoothness of the foreground application with a high rendering priority.

[0187] For example, the rendering priority can be divided into high-medium-low. For example, the rendering priority can be set to 0-2 in order from high to low, where high corresponds to 0, medium corresponds to 1, and low corresponds to 2. Of course, the rendering priority can also be set to 0-2 in order from low to high, where high corresponds to 2, medium corresponds to 1, and low corresponds to 0.

[0188] It should be noted that in the embodiment of the present application, there is no specific limitation on the division of rendering priorities.

[0189] In one possible implementation, the graphics rendering management module determines the rendering control strategies corresponding to multiple foreground applications based on the rendering priorities of each foreground application, which may include: the rendering controller obtains the initial rendering priority from the rendering load manager, determines the rendering priority of each foreground application based on the window information of the foreground application, the activity information of the foreground application and the initial rendering priority, and obtains the rendering control strategies corresponding to multiple foreground applications.

[0190] Exemplarily, the window information also includes a rendering area of ​​a foreground application. When the number of foreground applications is greater than 1, whether to perform rendering control on each foreground application is determined based on the rendering area of ​​each foreground application.

[0191] Specifically, when the rendering area of ​​each foreground application does not exceed a certain threshold, rendering control is not performed on each foreground application; when the rendering area of ​​each foreground application exceeds a certain threshold, rendering control is performed on each foreground application, and the rendering priority of each foreground application is determined based on the window information of the foreground application, the activity information of the foreground application, and the initial rendering priority, to obtain the rendering control strategy for multiple foreground applications.

[0192] It should be noted that the threshold can be set according to actual conditions or experience, for example, it can be set to half of the screen area, etc. The specific value of the threshold is not limited in the embodiment of the present application.

[0193] Exemplarily, the window information may also include the z-order of the windows corresponding to each foreground application, and whether the window corresponding to each foreground application is the focus window. The activity information includes the switching information of the foreground application. The rendering controller may obtain the rendering priority of each foreground application based on the z-order of the windows corresponding to each foreground application, whether the window corresponding to each foreground application is the focus window, the switching information of the foreground application and / or the initial rendering priority of the foreground application.

[0194] It should be noted that in order to ensure the smoothness of applications switching from the background to the foreground, low-level rendering priority is given to applications that are about to switch from the foreground to the background (cut out), and high-level rendering priority is given to applications that are about to switch from the background to the foreground (cut in).

[0195] The window z-order can be understood as the order of windows stacked on top of each other along an imaginary line perpendicular to the screen. Applications with windows near the top of the z-order are given a higher rendering priority, while applications with windows near the bottom of the z-order are given a lower rendering priority.

[0196] The application corresponding to the focus window is given a high level of rendering priority, and the application corresponding to the non-focus window is given a low level of rendering priority. The focus window can be determined based on the user's operation. For example, when two foreground applications are displayed in a split screen, the user performs a touch operation on the upper half of the screen. The foreground application displayed in the upper half of the screen is the focus window, and the foreground application displayed in the lower half of the screen is the non-focus window. When the user performs a touch operation, the window manager will also receive corresponding information, so that the foreground application corresponding to the focus window can be determined based on the window information.

[0197] In one possible implementation, the initial rendering priority is set to 0-2 in order from high to low. Based on the initial rendering priority of each application, the rendering priority is increased by 1 for the application to be switched out, and the rendering priority is reduced by 1 for the application to be switched in; the rendering priority is reduced by 1 for the application whose window z-order is moved forward, and the rendering priority is increased by 1 for the application whose window z-order is moved backward; the rendering priority is reduced by 1 for the application corresponding to the focus window, and the rendering priority is increased by 1 for the application corresponding to the non-focus window.

[0198] In an embodiment of the present application, the rendering control policy may be related to the rendering priority of each application.

[0199] For example, the rendering control strategy can be determined based on the rendering priority, such as stopping sending Vsync signals to foreground applications with lower rendering priorities, or reducing the frequency of sending Vsync signals to foreground applications with lower rendering priorities, or first stopping sending Vsync signals to foreground applications with lower rendering priorities, and then adjusting to reduce the frequency of sending Vsync signals to foreground applications with lower rendering priorities if it is identified that the load of multiple foreground applications is reduced to a first threshold based on the window information of the foreground applications and the activity information of the foreground applications.

[0200] Exemplarily, Figures 10a and 11a are two rendering control diagrams for foreground applications, wherein Figure 10a stops sending Vsync signals to foreground applications with lower rendering priority, and Figure 11a reduces the sending frequency of Vsync signals to foreground applications with lower rendering priority.

[0201] As shown in FIG10 a , during the rendering control duration, the graphics synthesis module stops sending Vsync signals to foreground applications with lower rendering priorities, and sends Vsync signals to foreground applications with higher rendering priorities as normal according to the original Vsync cycle.

[0202] For example, Figure 10b is a schematic diagram of an electronic device interface display processing flow in a possible implementation. As shown in Figure 10b, after the foreground application with a lower rendering priority renders frame 1, when the second Vsync signal arrives, the graphics synthesis module stops sending Vsync signals to the foreground application with a lower rendering priority, and continues to send Vsync signals to the foreground application with a higher rendering priority according to the original Vsync cycle. The foreground application interface with a higher rendering priority renders frames 1, 2, 3, 4, and 5 according to the original Vsync cycle.

[0203] For example, after stopping sending the Vsync signal to the foreground application with a lower rendering priority, in the synthesized picture, frames 2, 3, 4, and 5 rendered by the foreground application with a higher rendering priority are synthesized and displayed in sequence according to the original Vsync cycle, and the last frame 1 rendered by the foreground application with a lower rendering priority is synthesized and displayed according to the original Vsync cycle. The synthesized picture is still displayed in the order of frame 1, frame 2, and frame 3. When displayed, the foreground application interface with a higher rendering priority is refreshed according to the original Vsync cycle, while the foreground application interface with a lower rendering priority stays at the last frame before rendering control.

[0204] As shown in Figure 11a, during the rendering control period, the graphics synthesis module reduces the frequency of sending Vsync signals for foreground applications with lower rendering priority, for example, reducing it to half of the original Vsync signal sending frequency, and sends Vsync signals to foreground applications with higher rendering priority normally according to the original Vsync cycle.

[0205] For example, FIG11b is a schematic diagram of an electronic device interface display processing flow in a possible implementation. As shown in FIG11b, when the first Vsync signal arrives, the graphics synthesis module reduces the Vsync signal transmission frequency of the foreground application with a lower rendering priority to half of the original Vsync signal transmission frequency, and the foreground application interface with a lower rendering priority renders frames 1, 3, and 5 according to the reduced Vsync cycle, and sends Vsync signals to the foreground application with a higher rendering priority according to the original Vsync cycle. The foreground application interface with a higher rendering priority renders frames 1, 2, 3, 4, and 5 according to the original Vsync cycle.

[0206] For example, after stopping sending the Vsync signal to the foreground application with a lower rendering priority, in the synthesized picture, frames 2, 3, 4, and 5 rendered by the foreground application with a higher rendering priority are synthesized and displayed in sequence according to the original Vsync cycle, and frames 1, 3, and 5 rendered by the foreground application with a lower rendering priority are synthesized and displayed in sequence according to the reduced Vsync cycle. The synthesized picture is still displayed in the order of frame 1, frame 2, and frame 3. When displayed, the foreground application interface with a higher rendering priority is refreshed according to the original Vsync cycle, while the foreground application interface with a lower rendering priority has a lower rendering frequency and the picture refreshes more slowly. When the foreground application interface with a higher rendering priority refreshes 2 frames, the foreground application interface with a lower rendering priority can only refresh 1 frame. The refresh frequency of the foreground application interface with a lower rendering priority is one-half slower than the refresh frequency of the foreground application interface with a higher rendering priority.

[0207] Exemplarily, when the number of foreground applications is greater than 1, the graphics rendering management module obtains the rendering priority of each foreground application and determines the rendering control strategy corresponding to multiple foreground applications, which can be specifically: for foreground applications with lower rendering priority, stop sending Vsync signals and / or reduce the sending frequency of Vsync signals until the load of the foreground application is identified or the number of foreground applications is reduced to the corresponding threshold; for foreground applications with higher priority, send Vsync signals normally according to the original Vsync cycle.

[0208] S905 . The graphics synthesis module performs rendering control on each foreground application according to the rendering control strategies corresponding to the multiple foreground applications.

[0209] Among them, the graphics synthesis module includes a synthesis controller, a signal simulator, and a connection manager.

[0210] In an embodiment of the present application, the graphics synthesis module stores the rendering control strategy sent by the rendering controller in the synthesis controller, and the synthesis controller sends the rendering control strategy to the signal simulator. The signal simulator sends the corresponding Vsync signal to each foreground application according to the rendering control strategy and based on the established Vsync signal transmission channel.

[0211] The Vsync signal transmission channel is a channel established between the graphics synthesis module and the foreground application through the connection manager, so that the Vsync signal generated by the signal simulator can be sent to the foreground application.

[0212] In one possible implementation, when the rendering control method is to stop sending Vsync signals to foreground applications with lower rendering priorities, the signal simulator stops sending Vsync signals to foreground applications with lower rendering priorities within the rendering control duration based on the established Vsync signal transmission channel, and sends Vsync signals to foreground applications with higher priorities normally according to the original Vsync cycle.

[0213] For example, when the foreground applications include application 1 and application 2, the rendering priority of application 1 is higher than that of application 2, the signal simulator stops sending the Vsync signal to application 2, application 2 stops rendering within the rendering control time, the screen of application 2 stays at the last frame before the rendering control, and, according to the original Vsync cycle, sends the Vsync signal to application 1, and application 1 can render normally.

[0214] In another possible implementation, when the rendering control method is to reduce the sending frequency of the Vsync signal for the foreground application with a lower rendering priority, the signal simulator reduces the sending frequency of the Vsync signal for the foreground application with a lower rendering priority within the rendering control period based on the established Vsync signal transmission channel (for example, reducing it to half or one third of the original Vsync signal sending frequency, etc.), and sends the Vsync signal to the foreground application with a higher priority normally according to the original Vsync cycle.

[0215] For example, when the foreground applications include Application 1 and Application 2, the original Vsync signal sending frequency of Application 1 and Application 2 is 60Hz, and the rendering priority of Application 1 is higher than that of Application 2. The original Vsync signal sending frequency of Application 2 is reduced to 30Hz, and the sending frequency of Application 1 is maintained at 60Hz. Then, within the rendering control time, when the screen of Application 1 refreshes 2 frames, the screen of Application 2 can only refresh 1 frame. The screen refresh frequency of Application 2 is half slower than that of Application 1.

[0216] It should be noted that when executing a rendering control policy, the number of foreground applications may decrease and / or the load of the foreground applications may decrease. Thus, the rendering control policy can be stopped when the number of foreground applications decreases to one or the load of the foreground applications falls below a certain threshold. It is understood that the changes in the foreground applications are uncertain, and therefore the rendering control duration of executing the rendering control policy may also be uncertain.

[0217] For example, when executing the rendering control strategy, the number of foreground applications can be identified based on the window information and activity information of the foreground applications. When it is identified that the number of foreground applications has decreased to 1, the rendering control method can be stopped. Alternatively, the load of multiple foreground applications can be identified based on the window information and activity information of the foreground applications. When it is identified that the load of multiple foreground applications has decreased to a certain threshold, the rendering control method can be stopped.

[0218] The image display method provided in the embodiment of the present application, when there are multiple foreground applications, the electronic device will control the rendering according to the rendering priority of each foreground application, reduce the overall load of the system, improve the fluency of applications with higher rendering priority, and thus enhance the user experience.

[0219] Exemplarily, the image display method provided in the embodiment of the present application can be applied to a multi-window display scenario, and the multi-window display can include interface display, split-screen display, or floating window display during the switching process of the foreground application.

[0220] Specifically, when the graphics rendering management module determines the rendering control strategies corresponding to multiple foreground applications according to the rendering priority of each foreground application, three scenarios are explained, namely, foreground application switching, split-screen display, and floating window display.

[0221] It should be noted that when the rendering area of ​​each foreground application meets the rendering control conditions, the rendering control of each foreground application is triggered; when the rendering area of ​​each foreground application does not meet the rendering control conditions, each foreground application is rendered according to the original Vsync signal.

[0222] Among them, the multi-window display scenario can be a scenario of switching the foreground application, for example, the foreground application returns to the desktop from a third-party application or other system application, opens a third-party application or other system application, switches the foreground application in / out from a third-party application or other system application, etc.

[0223] Exemplarily, when two applications other than desktop applications are switched in / out, the specific process includes: displaying a multitasking interface on the screen of the electronic device, the multitasking interface including a first application and a second application; receiving a first operation from the user, the first operation being used to switch in the first application and to switch out the second application; in response to the first operation, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; in response to the first operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the second application to obtain a target image; and displaying the target image.

[0224] For example, taking the first scenario of switching in / out of a video application and a social application as an example, FIG12 shows a schematic diagram of the first scenario to which the image display method of an embodiment of the present application is applicable.

[0225] As shown in a in Figure 12, a multitasking interface is displayed on the screen of the electronic device. The multitasking interface includes a social application and a video application. When a user clicks on the social application, the social application is switched in and the video application is switched out in response to the operation, resulting in an interface as shown in b in Figure 12, in which the social application is displayed on the screen.

[0226] Exemplarily, based on the social application being the entry application and the video application being the exit application, a Vsync signal is sent to the video application at a first frequency, or sending the Vsync signal to the video application is stopped; in response to the operation, a Vsync signal is sent to the social application at a second frequency; wherein the first frequency is less than the second frequency; the image rendered in the video application is synthesized with the image rendered in the social application to obtain a target image; and the target image is displayed.

[0227] In an embodiment of the present application, for the incoming application, a Vsync signal is sent to it at the original frequency, and for the outgoing application, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the incoming application can have sufficient rendering resources and improve the smoothness of the incoming application.

[0228] Exemplarily, when the foreground application returns to the desktop from a third-party application or other system application, the specific process includes: receiving a sixth operation of the user returning from the first application to the desktop application; in response to the sixth operation, stopping sending the Vsync signal to the first application, sending the Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; in response to the sixth operation, sending the Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; synthesizing the image rendered in the first application with the image rendered in the desktop application to obtain a target image; and displaying the target image.

[0229] For example, in this scenario, the rendering area of ​​each foreground application meets the rendering control condition. Figure 13 shows a schematic diagram of a second scenario applicable to the image display method of the embodiment of the present application.

[0230] As shown in Figures a and b of Figure 13, the user swipes up, and in response to this operation, the display window of the social application gradually decreases until it disappears, resulting in the system desktop shown in Figure 13c. As a result, the display interface of the electronic device returns from the social interface to the system desktop. It is understandable that the social application can be a third-party application or a system application.

[0231] Exemplarily, based on the social application being the outgoing application and the Launcher being the incoming application, a Vsync signal is sent to the social application at a first frequency, or sending the Vsync signal to the video application is stopped; in response to the operation, a Vsync signal is sent to the Launcher at a second frequency; wherein the first frequency is less than the second frequency; the image rendered in the social application is synthesized with the image rendered by the Launcher to obtain a target image; and the target image is displayed.

[0232] In the process of returning to the system desktop from the social interface, social applications and Launcher may be rendered at the same time, which may cause excessive system load. The electronic device can control the rendering of social applications and Launcher based on their rendering priority.

[0233] In this scenario, since the rendering of the Launcher is more important to the user experience, the Launcher is at the top of the window z-order. When the foreground application includes the Launcher and other applications, the z-order of the windows corresponding to the Launcher and other applications does not change when the foreground application is switched; when the foreground application is two applications other than the Launcher, when the foreground application is switched, the z-order of the application swapped in moves forward and the rendering priority increases, and the z-order of the application swapped out moves backward and the rendering priority decreases.

[0234] In this scenario, the rendering priority of the application being swapped in is increased, while the rendering priority of the application being swapped out is decreased, thereby ensuring the smoothness of the application being displayed in the foreground. Furthermore, in this scenario, the window touched by the user may not correspond to the application being swapped in. Therefore, the rendering priority of the application is not adjusted based on whether it corresponds to the focused window.

[0235] For example, the initial rendering priority of an electronic device is set from high to low, from 0 to 2. The initial rendering priorities of the social interface and the launcher are 2 and 1, respectively. When returning from the social interface to the system desktop, the launcher's rendering priority is reduced by 1 from its initial rendering priority, becoming 0. The rendering priority of the social application is increased by 1 from its initial rendering priority, becoming 3. Furthermore, the z-order of the windows corresponding to the launcher and the social application remains unchanged. The correspondence between foreground applications and rendering priorities can be shown in Table 1.

[0236] Table 1 Correspondence between foreground applications and rendering priorities

[0237] As can be seen from Table 1, by adjusting the rendering priority of social applications and the rendering priority of the Launcher, with the rendering priority of the social applications being 3 and the rendering priority of the Launcher being 0, the rendering of the social applications can be limited, ensuring the rendering of the Launcher and improving the smoothness of the Launcher. Limiting the rendering of social applications can include stopping the sending of Vsync signals to the social applications and / or reducing the frequency of sending Vsync signals.

[0238] In an embodiment of the present application, for the incoming application, a Vsync signal is sent to it at the original frequency, and for the outgoing application, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the incoming application can have sufficient rendering resources and improve the smoothness of the incoming application.

[0239] The multi-window display scenario can be a split-screen display scenario. For example, the upper half window of the screen and the lower half window of the screen respectively display different applications. For example, the upper half window of the screen displays a PPT file, and the lower half window of the screen displays a Word document.

[0240] Exemplarily, a first interface is displayed, where the first interface includes a first application; a second operation is received from the user for displaying the first application and the second application in split screen; in response to the second operation, the first application is displayed in the first window, and the second application is displayed in the second window; a third operation is received for the second application in the second window; in response to the third operation, a vertical synchronization signal Vsync signal is sent to the first application at a first frequency, or the sending of the Vsync signal to the first application is stopped; in response to the second operation, a Vsync signal is sent to the second application at a second frequency; wherein the first frequency is less than the second frequency; the rendered image in the first application is synthesized with the rendered image in the second application to obtain a target image; and the target image is displayed.

[0241] For example, in this scenario, the rendering area of ​​each foreground application meets the rendering control condition. Figure 14 shows a schematic diagram of a third scenario to which the image display method according to an embodiment of the present application is applicable.

[0242] As shown in Figure 14, the display interface of the electronic device includes a ppt application and a word application. The upper half window of the screen displays the ppt file, and the lower half window of the screen displays the word document. When the user edits the ppt file based on the word document, the user touches the upper half window of the screen, thereby determining that the upper half window is the focus window.

[0243] For example, the initial rendering priority of an electronic device is set from 0 to 2 from high to low. The initial rendering priority of the Word application and the initial rendering priority of the PPT application are both 2. Since the window corresponding to the PPT application is the focused window, the rendering priority of the PPT application is reduced by 1 based on its initial rendering priority, and the rendering priority of the PPT application becomes 1. Since the window corresponding to the Word application is not the focused window, the rendering priority of the Word application is increased by 1 based on its initial rendering priority, and the rendering priority of the Word application becomes 3. The corresponding relationship between foreground applications and rendering priorities can be shown in Table 2.

[0244] Table 2 Correspondence between foreground applications and rendering priorities

[0245] As can be seen from Table 2, by adjusting the rendering priority of the Word application and the rendering priority of the PPT application, the rendering priority of the Word application is 3 and the rendering priority of the PPT application is 1. This can limit the rendering of the Word application, ensure the rendering of the PPT application, and improve the smoothness of the PPT application. Among them, limiting the rendering of the Word application can include stopping the sending of Vsync signals to the Word application and / or reducing the sending frequency of the Vsync signals.

[0246] In an embodiment of the present application, for the application in the focus window, a Vsync signal is sent to it at the original frequency, and for the application in the non-focus window, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the application in the focus window can have sufficient rendering resources, thereby improving the smoothness of the application in the focus window.

[0247] The multi-window display scenario can be a floating window display scenario, for example, picture-in-picture display of a video application, displaying windows such as web pages and files on the desktop, and so on.

[0248] Exemplarily, a first application is displayed in full screen; a fourth operation for triggering a second application is received; in response to the fourth operation, the second application is displayed in full screen, and the first application is displayed in a floating window on the second application; a fifth operation for the second application is received; in response to the fifth operation, a vertical synchronization signal Vsync signal is sent to the first application at a first frequency, or the Vsync signal is stopped from being sent to the first application; in response to the fifth operation, a Vsync signal is sent to the second application at a second frequency; wherein the first frequency is less than the second frequency; the rendered image in the first application is synthesized with the rendered image in the second application to obtain a target image; and the target image is displayed.

[0249] Taking the fourth scenario of picture-in-picture display of a video application as an example, in which the rendering area of ​​each foreground application meets the rendering control conditions, FIG15 shows a schematic diagram of the fourth scenario applicable to the image display method of an embodiment of the present application.

[0250] As shown in a in Figure 15, when the user is watching a video in full screen through the video software, a message notification pops up from the social application (for example, WeChat). The user clicks on the message notification, and the display interface of the electronic device switches from full-screen display of the video application's playback interface to full-screen display, the social application's dialog interface, and the video application's playback interface in the form of a floating window, resulting in the display interface of the electronic device as shown in b in Figure 15, including two foreground applications, the social application and the video application.

[0251] When two foreground applications, a social application and a video application, are displayed at the same time, the social application and the video application may be rendered at the same time, which may cause the system load to be too high. The electronic device can control the rendering of the social application and the video application according to the rendering priority of the social application and the video application.

[0252] It should be noted that in this scenario, since the user switches from the full-screen display interface of the video application to the dialog box of the social application to view messages or reply to messages, the user's focus is on the social application, and the window corresponding to the social application is determined as the focus window, and the window corresponding to the video application is determined as the non-focus window.

[0253] Exemplarily, based on operations such as viewing messages or replying to messages, it can be determined that the social application corresponds to the focus window and the video application corresponds to the focus window, and a Vsync signal is sent to the video application at a first frequency, or the sending of the Vsync signal to the video application is stopped; in response to the operation, a Vsync signal is sent to the social application at a second frequency; wherein the first frequency is less than the second frequency; the image rendered in the video application and the image rendered in the social application are synthesized to obtain a target image; and the target image is displayed.

[0254] For example, the initial rendering priority of an electronic device is set from high to low, from 0 to 2. The initial rendering priority of a social interface and a video application is 2 and 1, respectively. When two foreground applications are displayed simultaneously, the window corresponding to the social application is the focused window. The rendering priority of the social application is reduced by 1, and the rendering priority of the social application becomes 1. The window corresponding to the video application is not the focused window. The rendering priority of the video application is increased by 1, and the rendering priority of the video application becomes 2. The corresponding relationship between foreground applications and rendering priorities can be shown in Table 3.

[0255] Table 3 Correspondence between foreground applications and rendering priorities

[0256] As can be seen from Table 1, by adjusting the rendering priority of social applications and the rendering priority of video applications, the rendering priority of social applications is greater than the rendering priority of video applications. In this way, the rendering of video applications can be restricted, and social applications can be rendered according to the original Vsync cycle, so that the display interface of the social applications that users are interested in can be refreshed smoothly, thereby improving the user experience.

[0257] In an embodiment of the present application, for the application in the focus window, a Vsync signal is sent to it at the original frequency, and for the application in the non-focus window, the sending frequency of its Vsync signal is reduced, or the sending of the Vsync signal to it is stopped, so that the application in the focus window can have sufficient rendering resources, thereby improving the smoothness of the application in the focus window.

[0258] For example, FIG16 is a flowchart of an image display method provided by an embodiment of the present application. The method shown in FIG16 can be executed by the electronic device 100 shown in FIG1 . As shown in FIG16 , the method includes:

[0259] S1601: Determine a first rendering priority of the first application according to a first initial rendering priority of the first application, whether the first application corresponds to a focus window, and / or an application switching type of the first application.

[0260] The first rendering initial priority is related to the application type of the first application and / or the rendering load of the first application.

[0261] In an embodiment of the present application, different applications may have different initial rendering priorities. Exemplarily, the initial rendering priority may be divided into high-medium-low. For example, the initial rendering priority may be set to 0-2 in order from high to low, with 0 corresponding to high initial rendering priority, 1 corresponding to medium initial rendering priority, and 2 corresponding to low initial rendering priority. Of course, the initial rendering priority may also be set to 0-2 in order from low to high, with 2 corresponding to high initial rendering priority, 1 corresponding to medium initial rendering priority, and 0 corresponding to low initial rendering priority. In an embodiment of the present application, there is no specific limitation on the division of the initial rendering priority.

[0262] In one possible implementation, the initial rendering priority can be set based on the application type. Specifically, the initial rendering priority of system applications can be set higher than the initial rendering priority of third-party applications. For example, system applications may include desktop launchers, system user interfaces, etc., while third-party applications may include WeChat, Toutiao, etc.

[0263] In another possible implementation, the initial rendering priority can be set based on the rendering load of the application. Specifically, the initial rendering priority of an application with a large rendering load can be set higher than the initial rendering priority of an application with a small rendering load. For example, if the rendering load of a gaming application is greater than that of a social networking application, the initial rendering priority of the gaming application can be set higher than that of the social networking application.

[0264] In another possible implementation, the initial rendering priority can be set based on the application type and the rendering load of the application. Specifically, when multiple applications have the same application type, the initial rendering priority can be set based on the rendering load of the application, and the initial rendering priority of the application with a large rendering load is set higher than the initial rendering priority of the application with a small rendering load. For example, when the two foreground applications are a third-party game application and a third-party social application, when the rendering load of the third-party game application is greater than the rendering load of the third-party social application, the initial rendering priority of the third-party game application is set higher than the initial rendering priority of the third-party social application.

[0265] In an embodiment of the present application, the application switching type includes switching out of the application or switching in of the application. Switching in of the application can be understood as switching in of the application to the foreground of the electronic device, and switching out of the application can be understood as switching out of the foreground of the electronic device, as well as switching in of the application to the background of the electronic device.

[0266] Exemplarily, based on the first initial rendering priority of the first application, when the window corresponding to the first application is a focus window and / or the application switching type of the first application is to switch into the application, the first initial rendering priority of the first application is increased; when the window corresponding to the first application is a non-focus window and / or the application switching type of the first application is to switch out of the application, the first initial rendering priority of the first application is lowered, thereby obtaining the first rendering priority of the first application.

[0267] S1602: Determine a second rendering priority of the second application according to a second initial rendering priority of the second application, whether the second application corresponds to a focus window, and / or an application switching type of the second application.

[0268] The second rendering initial priority is related to the application type of the second application and / or the rendering load of the second application interface to be displayed. The description of the second rendering initial priority can be referred to the description in step S1501 and will not be repeated here.

[0269] It should be noted that the first application and the second application are applications running in the foreground of the electronic device.

[0270] S1603: When the first rendering priority is lower than the second rendering priority, send a Vsync signal to the first application at a first frequency, or stop sending the Vsync signal to the first application; and send a Vsync signal to the second application at a second frequency.

[0271] The first frequency is lower than the second frequency, and the first frequency can be understood as a frequency obtained by reducing the second frequency.

[0272] In an embodiment of the present application, the rendering priority can be used to control the rendering of multiple foreground applications to varying degrees. For example, when rendering multiple foreground applications, the higher the rendering priority of the foreground application, the more computing resources are given, which can improve the smoothness of the foreground application with a high rendering priority.

[0273] For example, the rendering priority can be divided into high-medium-low. For example, the rendering priority can be set to 0-2 in order from high to low, where high corresponds to 0, medium corresponds to 1, and low corresponds to 2. Of course, the rendering priority can also be set to 0-2 in order from low to high, where high corresponds to 2, medium corresponds to 1, and low corresponds to 0.

[0274] It should be noted that in the embodiment of the present application, there is no specific limitation on the division of rendering priorities.

[0275] Exemplarily, when the first rendering priority of the first application is lower than the second rendering priority of the second application, a Vsync signal is sent to the second application at a second frequency, and the sending of the Vsync signal to the first application is stopped, or, based on the second frequency, the sending frequency of the Vsync signal of the first application is reduced to the first frame rate.

[0276] S1604: Synthesize the image rendered in the first application and the image rendered in the second application to obtain a target image.

[0277] For example, a Vsync signal is sent to a first application at a first frequency, or the sending of the Vsync signal to the first application is stopped, the first application is rendered to obtain an image rendered by the first application, and a Vsync signal is sent to a second application at a second frequency to render the second application to obtain an image rendered by the second application. The image rendered by the first application is synthesized with the image rendered by the second application to obtain a target image.

[0278] S1605: Display the target image.

[0279] Exemplarily, an interface synthesized by the first application and the second application is displayed on the screen of the electronic device, where the synthesized interface is a target image synthesized by the image rendered in the first application and the image rendered in the second application.

[0280] In the image display method provided in an embodiment of the present application, the first application and the second application are applications running in the foreground of the electronic device, and the first rendering priority of the first application is determined according to the first initial rendering priority of the first application, whether the first application corresponds to the focus window and / or the application switching type of the first application; the second rendering priority of the second application is determined according to the second initial rendering priority of the second application, whether the second application corresponds to the focus window and / or the application switching type of the second application; when the first rendering priority is lower than the second rendering priority, a Vsync signal is sent to the first application at a first frequency, or the Vsync signal is stopped from being sent to the first application; and a Vsync signal is sent to the second application at a second frequency; wherein the first frequency is lower than the second frequency; the image rendered in the first application is synthesized with the image rendered in the second application to obtain a target image; and the target image is displayed. In this method, by comparing the first rendering priority of the first application and the second rendering priority of the second application, for the foreground application with a lower rendering priority, the sending frequency of the Vsync signal is reduced or the sending of the Vsync signal is stopped, and for the foreground application with a higher rendering priority, the original sending frequency of the Vsync signal is maintained, thereby improving the fluency of the foreground application with a higher rendering priority and enhancing the user experience.

[0281] Exemplarily, based on the above embodiment, the image display method further includes:

[0282] In a possible implementation, the electronic device is in a scenario where a foreground application is switched, the first application is a switched-out application, and the second application is a switched-in application.

[0283] Exemplarily, determining a first rendering priority for a first application includes determining the first rendering priority for the first application based on a first initial rendering priority for the first application and an application switching type for the first application. And determining a second rendering priority for a second application includes determining a second rendering priority for the second application based on a second initial rendering priority for the second application and an application switching type for the second application.

[0284] When the electronic device is in a scenario where the foreground application is switched, the first rendering priority of the first application is determined based on the first rendering initial priority of the first application and the application switching type of the first application; the second rendering priority of the second application is determined based on the second rendering initial priority of the second application and the application switching type of the second application, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0285] In one possible implementation, determining a first rendering priority for a first application based on a first initial rendering priority of the first application and an application switching type of the first application includes: based on the first application being a switched-out application, lowering the first initial rendering priority of the first application to obtain the first rendering priority of the first application. And determining a second rendering priority for a second application based on a second initial rendering priority of the second application and the application switching type of the second application includes: based on the second application being a switched-in application, raising the second initial rendering priority of the second application to obtain the second rendering priority of the second application.

[0286] For example, in order to improve the smoothness of applications switching from the background to the foreground, based on the first application being the switched-out application, the first rendering initial priority of the first application is lowered to obtain the first rendering priority of the first application; based on the second application being the switched-in application, the second rendering initial priority of the second application is increased to obtain the second rendering priority of the second application, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0287] In one possible implementation, based on the initial rendering priority of each application, for an application to be switched out, 1 is added to its initial rendering priority, and for an application to be switched in, 1 is subtracted from its initial rendering priority.

[0288] For example, the initial rendering priority of the electronic device is set from 0 to 2 from high to low, the initial rendering priority of the social interface and the initial rendering priority of the Launcher are 2 and 1 respectively. In the process of returning from the social interface to the system desktop, the rendering priority of the Launcher is reduced by 1 based on its initial rendering priority, and the rendering priority of the Launcher becomes 0. The rendering priority of the social application is increased by 1 based on its initial rendering priority, and the rendering priority of the social application becomes 3.

[0289] In a possible implementation, the electronic device is in a split-screen display scenario, and the first application and the second application are displayed in two different windows respectively.

[0290] Exemplarily, determining a first rendering priority of a first application includes: determining the first rendering priority of the first application based on the first rendering initial priority of the first application and whether the first application corresponds to a focus window; and determining a second rendering priority of a second application includes: determining the second rendering priority of the second application based on the second rendering initial priority of the second application and whether the second application corresponds to a focus window.

[0291] When the electronic device is in a split-screen display scenario, the first application and the second application are displayed in two different windows, and the two different windows can correspond to the focus window and the non-focus window respectively. Specifically, when the first application corresponds to the focus application, the second application corresponds to the non-focus application; when the first application corresponds to the non-focus application, the second application corresponds to the focus application.

[0292] Exemplarily, based on the first rendering initial priority of the first application, the first rendering priority of the first application is determined according to whether the first application corresponds to the focus window; based on the second rendering initial priority of the second application, the second rendering priority of the second application is determined according to whether the second application corresponds to the focus window, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0293] In one possible implementation, the first rendering priority of the first application is determined based on the first rendering initial priority of the first application and whether the first application corresponds to a focus window, including: based on the first application corresponding to a non-focus window, lowering the first rendering initial priority of the first application to obtain the first rendering priority of the first application; based on the second rendering initial priority of the second application and whether the second application corresponds to a focus window, determining the second rendering priority of the second application, including: based on the second application corresponding to the focus window, increasing the second rendering initial priority of the second application to obtain the second rendering priority of the second application.

[0294] In an embodiment of the present application, the focus window is a window that the user pays more attention to. In order to improve the fluency of the focus window, based on the non-focus window corresponding to the first application, the first rendering initial priority of the first application is lowered to obtain the first rendering priority of the first application; based on the focus window corresponding to the second application, the second rendering initial priority of the second application is increased to obtain the second rendering priority of the second application.

[0295] Among them, the focus window can be determined according to the user's operation. For example, when two foreground applications are displayed in split screen, the user performs a touch operation on the upper half of the screen, then the foreground application displayed in the upper half of the screen is the focus window, and the foreground application displayed in the lower half of the screen is the non-focus window.

[0296] In one possible implementation, based on the initial rendering priority of each application, for the application corresponding to the focus window, 1 is added to the initial rendering priority, and for the application corresponding to the non-focus window, 1 is subtracted from the initial rendering priority.

[0297] For example, the initial rendering priority of the terminal device is set from high to low, from 0 to 2. The display interface of the terminal device includes a ppt application and a word application. The initial rendering priority of the word application and the initial rendering priority of the ppt application are both 2. The upper half of the screen displays a ppt file, and the lower half of the screen displays a word document. The user edits the ppt file based on the word document. The user touches the upper half of the screen, thereby determining that the upper half of the window is the focus window. Since the window corresponding to the ppt application is the focus window, the rendering priority of the ppt application is reduced by 1 based on its initial rendering priority, and the rendering priority of the ppt application becomes 1; since the window corresponding to the word application is not the focus window, the rendering priority of the word application is increased by 1 based on its initial rendering priority, and the rendering priority of the word application becomes 3.

[0298] In an embodiment of the present application, for applications displayed in the focus window, the corresponding initial rendering priority is increased, and for applications displayed in the non-focus window, the corresponding initial rendering priority is lowered, which can improve the fluency of the applications displayed in the focus window.

[0299] In a possible implementation, the electronic device is in a floating window display scenario.

[0300] Exemplarily, determining a first rendering priority of a first application includes: determining the first rendering priority of the first application based on the first rendering initial priority of the first application and whether the first application corresponds to a focus window; and determining a second rendering priority of a second application includes: determining the second rendering priority of the second application based on the second rendering initial priority of the second application and whether the second application corresponds to a focus window.

[0301] In an embodiment of the present application, when the electronic device is in a floating window display scenario, for two applications running in the foreground, when the application type of the first application is the incoming application, the application type of the second application is the outgoing application; when the application type of the first application is the outgoing application, the application type of the second application is the incoming application. When the first application corresponds to the focused application, the second application corresponds to the non-focused application; when the first application corresponds to the non-focused application, the second application corresponds to the focused application.

[0302] Exemplarily, based on the first rendering initial priority of the first application, the first rendering priority of the first application is determined according to whether the first application corresponds to the focus window; based on the second rendering initial priority of the second application, the second rendering priority of the second application is determined according to whether the second application corresponds to the focus window, and then the rendering control method for the first application and the second application can be determined by comparing the first rendering priority and the second rendering priority.

[0303] In one possible implementation, the first rendering priority of the first application is determined based on the first rendering initial priority of the first application and whether the first application corresponds to a focus window, including: based on the first application corresponding to a non-focus window, lowering the first rendering initial priority of the first application to obtain the first rendering priority of the first application; based on the second rendering initial priority of the second application and whether the second application corresponds to a focus window, determining the second rendering priority of the second application, including: based on the second application corresponding to the focus window, increasing the second rendering initial priority of the second application to obtain the second rendering priority of the second application.

[0304] In an embodiment of the present application, the focus window is a window that the user pays more attention to. In order to improve the fluency of the focus window, based on the non-focus window corresponding to the first application, the first rendering initial priority of the first application is lowered to obtain the first rendering priority of the first application; based on the focus window corresponding to the second application, the second rendering initial priority of the second application is increased to obtain the second rendering priority of the second application.

[0305] Among them, the focus window can be determined according to the user's operation. For example, when two applications are running in the foreground, application 2 is displayed in full screen on the screen of the electronic device, and application 1 is displayed on application 2 in the form of a floating window. The user performs a touch operation on application 2, then application 2 corresponds to the focus window and application 1 corresponds to the non-focus window.

[0306] For example, the initial rendering priority set by the terminal electronic device is 0 to 2 from high to low, and the initial rendering priority of the social interface and the initial rendering priority of the video application are 2 and 1 respectively. When the two foreground applications, the social application and the video application, are displayed at the same time, the social application is displayed in full screen on the screen of the electronic device, and the video application is displayed on the social application in the form of a floating window. Since the user replies to messages and other operations in the dialog box of the social application, the window corresponding to the social application is the focus window, and the rendering priority of the social application is reduced by 1 based on its initial rendering priority, and the rendering priority of the social application becomes 1; the window corresponding to the video application is a non-focus window, and the rendering priority of the video application is increased by 1 based on its initial rendering priority, and the rendering priority of the video application becomes 2.

[0307] In one possible implementation, before determining the first rendering priority of the first application and determining the second rendering priority of the second application, the method further includes: obtaining a rendering area of ​​the first application and a rendering area of ​​the second application. Determining the first rendering priority of the first application and the second rendering priority of the second application includes: when both the rendering area of ​​the first application and the rendering area of ​​the second application are greater than a first preset value, determining the first rendering priority of the first application and determining the second rendering priority of the second application.

[0308] In the embodiment of the present application, the size of the rendering area can be used to represent the size of the rendering load.

[0309] For example, when a first application and a second application are running in the foreground simultaneously, if both the rendering area of ​​the first application and the rendering area of ​​the second application are greater than a first preset value, the system's rendering load is large, which may affect the smoothness of the first application and / or the second application. Therefore, when both the rendering area of ​​the first application and the rendering area of ​​the second application are greater than the first preset value, a first rendering priority of the first application and a second rendering priority of the second application are determined, and rendering control of the first application and the second application is performed based on the first rendering priority and the second rendering priority, thereby reducing the system's rendering load and improving the smoothness of the applications.

[0310] In one possible implementation, before determining the first rendering priority of the first application and determining the second rendering priority of the second application, it also includes: obtaining window information of the application running in the foreground and / or switching information of the application running in the foreground; and determining that the application running in the foreground includes the first application and the second application based on the window information of the application running in the foreground and / or switching information of the application running in the foreground.

[0311] In the embodiments of the present application, window information can be understood as the corresponding window interface of each application during operation, and each window interface has its own corresponding information and data. This information and data may include attribute information such as window size, window position, and window layout. For example, WeChat has a corresponding WeChat window interface during operation, and WeChat window information may include information such as the position of each button and the size of the dialog box.

[0312] In the embodiments of the present application, the switching information of the foreground application can be understood as information generated when the foreground application of the electronic device switches, so as to facilitate tracking and management of the switching of the foreground application. For example, when the foreground application switches from a social application to a game application, the switching information of the foreground application will be generated.

[0313] For example, it can be determined that the foreground running applications include a first application and a second application based on window information of the foreground running application and / or foreground running application switching information. When it is determined that the foreground running applications include the first application and the second application, the first rendering priority of the first application and the second rendering priority of the second application are compared, and rendering control of the first application and the second application is then performed to improve application fluency. When it is determined that the foreground running applications include the first application but not the second application, it is not necessary to obtain the first rendering priority of the first application, thereby saving system resources.

[0314] The embodiments described in the above exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of methods consistent with some aspects of the present application as detailed in the appended claims.

[0315] Having described the electronic device performance adjustment method according to an embodiment of the present application above, the electronic device provided in the embodiment of the present application for executing the aforementioned image display method is now described. Those skilled in the art will appreciate that the methods and apparatuses may be combined and referenced with one another, and that the electronic device provided in the embodiment of the present application may execute the steps in the aforementioned image display method.

[0316] Figure 17 shows a schematic diagram of the structure of an image display device provided in an embodiment of the present application. As shown in Figure 17, the image display device can be an electronic device in an embodiment of the present application. The image display device includes: a touch screen 1701, wherein the touch screen includes a touch panel and a display screen; one or more processors 1702; a memory 1703; multiple application programs; and one or more computer programs, wherein the one or more computer programs are stored in the memory 1703, and the one or more computer programs include instructions. When the instructions are executed by the image display device, the image display method in the above embodiment is implemented.

[0317] Figure 18 is a schematic diagram of the hardware structure of an image display device provided in an embodiment of the present application. Referring to Figure 18 , the device includes: a memory 1801, a processor 1802, and a display screen 1804. The device may also include an interface circuit 1803, wherein the memory 1801, processor 1802, interface circuit 1803, and display screen 1804 are capable of communicating. Exemplarily, the memory 1801, processor 1802, interface circuit 1803, and display screen 1804 may communicate via a communication bus. The memory 1801 is used to store computer-executable instructions, which are controlled for execution by the processor 1802 and displayed by the display screen 1804, thereby implementing the image display method provided in the following embodiments of the present application.

[0318] In possible implementations, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0319] Optionally, the interface circuit 1803 may further include a transmitter and / or a receiver. Optionally, the processor 1802 may include one or more CPUs, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in this application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0320] The device of this embodiment can be used to execute the steps executed in the above method embodiment. Its implementation principles and technical effects are similar and will not be described in detail here.

[0321] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0322] In one possible implementation, computer-readable media may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium designed to carry or store the desired program code in the form of instructions or data structures and accessible by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwave are used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and optical disk as used herein include optical disk, laser disk, optical disk, digital versatile disk (DVD), floppy disk and Blu-ray disk, where disks generally reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0323] The present application embodiment is described with reference to the flow chart and / or block diagram according to the method, device (system) and computer program product of the embodiment of the present application.It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processing unit of general-purpose computer, special-purpose computer, embedded processing machine or other programmable data processing equipment to produce a machine, so that the instruction executed by the processing unit of computer or other programmable data processing equipment produces the device for realizing the function specified in one flow chart flow chart or multiple flow charts and / or one block or multiple blocks of block diagram.

[0324] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image display method, characterized in that: Applied to electronic equipment, the method includes: determining a first rendering priority of the first application based on a first initial rendering priority of the first application, whether the first application corresponds to a focus window, and / or an application switching type of the first application; wherein the first initial rendering priority is related to the application type of the first application and / or the rendering load of the first application, and the application switching type includes switching out of the application or switching in of the application; determining a second rendering priority of the second application based on a second initial rendering priority of the second application, whether the second application corresponds to a focus window, and / or an application switching type of the second application, wherein the second initial rendering priority is related to the application type of the second application and / or a rendering load of a to-be-displayed interface of the second application; The first application and the second application are applications running in the foreground of the electronic device; When the first rendering priority is lower than the second rendering priority, sending a vertical synchronization signal Vsync signal to the first application at a first frequency, or stopping sending the Vsync signal to the first application; and sending a Vsync signal to the second application at a second frequency; wherein the first frequency is lower than the second frequency; Combining the image rendered in the first application with the image rendered in the second application to obtain a target image; The target image is displayed.

2. The method according to claim 1, wherein the electronic device is in a scenario where a foreground application is switched, the first application is a switched-out application, and the second application is a switched-in application; Determining the first rendering priority of the first application includes: determining a first rendering priority of the first application according to a first rendering initial priority of the first application and an application switching type of the first application; Determining the second rendering priority of the second application includes: A second rendering priority of the second application is determined according to the second rendering initial priority of the second application and the application switching type of the second application.

3. The method according to claim 2, characterized in that The determining, according to the first initial rendering priority of the first application and the application switching type of the first application, the first rendering priority of the first application includes: Based on the fact that the first application is a switched-out application, lowering a first rendering initial priority of the first application to obtain a first rendering priority of the first application; The determining, according to the second initial rendering priority of the second application and the application switching type of the second application, the second rendering priority of the second application includes: Based on the fact that the second application is an entry application, the second initial rendering priority of the second application is increased to obtain a second rendering priority of the second application.

4. The method according to claim 1, wherein the electronic device is in a split-screen display scenario, and the first application and the second application are displayed in two different windows respectively; Determining the first rendering priority of the first application includes: determining a first rendering priority of the first application according to a first rendering initial priority of the first application and whether the first application corresponds to a focus window; Determining the second rendering priority of the second application includes: Determine the second rendering initial priority of the second application and whether the second application corresponds to the focus window. The second rendering priority of the second application.

5. The method according to claim 4, wherein determining the first rendering priority of the first application according to the first rendering initial priority of the first application and whether the first application corresponds to a focus window comprises: Based on the non-focus window corresponding to the first application, lowering the first rendering initial priority of the first application to obtain the first rendering priority of the first application; The determining, according to the second initial rendering priority of the second application and whether the second application corresponds to the focus window, the second rendering priority of the second application includes: Based on the fact that the second application corresponds to the focus window, the second rendering initial priority of the second application is increased to obtain the second rendering priority of the second application.

6. The method according to claim 1, wherein the electronic device is in a floating window display scenario; Determining the first rendering priority of the first application includes: determining a first rendering priority of the first application according to a first rendering initial priority of the first application and whether the first application corresponds to a focus window; Determining the second rendering priority of the second application includes: A second rendering priority of the second application is determined according to the second rendering initial priority of the second application and whether the second application corresponds to a focus window.

7. The method according to claim 6, wherein determining the first rendering priority of the first application based on the first rendering initial priority of the first application and whether the first application corresponds to a focus window comprises: Based on the non-focus window corresponding to the first application, lowering a first rendering initial priority of the first application to obtain a first rendering priority of the first application; The determining, according to the second initial rendering priority of the second application and whether the second application corresponds to the focus window, the second rendering priority of the second application includes: Based on the fact that the second application corresponds to the focus window, the second initial rendering priority of the second application is increased to obtain the second rendering priority of the second application.

8. The method according to any one of claims 1 to 7, further comprising: before determining the first rendering priority of the first application and determining the second rendering priority of the second application: Obtaining a rendering area of ​​the first application and a rendering area of ​​the second application; Determining the first rendering priority of the first application and the second rendering priority of the second application includes: when the rendering area of ​​the first application and the rendering area of ​​the second application are both greater than a first preset value, determining the first rendering priority of the first application and determining the second rendering priority of the second application.

9. The method according to any one of claims 1 to 8, further comprising: before determining the first rendering priority of the first application and determining the second rendering priority of the second application: Obtaining window information of the application running in the foreground and / or switching information of the application running in the foreground; According to the window information of the application running in the foreground and / or the switching information of the application running in the foreground, it is determined that the application running in the foreground includes the first application and the second application.

10. An image display method, characterized in that: Applied to electronic equipment, the method includes: Displaying a multitasking interface, wherein the multitasking interface includes a first application and a second application; Receiving a first user operation; the first operation is used to switch the first application in and out of the second application; In response to the first operation, sending a vertical synchronization signal (Vsync) to the first application at a first frequency, or stopping sending the Vsync signal to the first application; In response to the first operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; Combining the image rendered in the first application with the image rendered in the second application to obtain a target image; The target image is displayed.

11. An image display method, characterized in that: Applied to electronic equipment, the method includes: Displaying a first interface, wherein the first interface includes a first application; receiving a second operation from a user for displaying the first application and the second application in split-screen mode; In response to the second operation, displaying the first application in a first window and displaying the second application in a second window; receiving a third operation on the second application in the second window; In response to the third operation, sending a vertical synchronization signal (Vsync) to the first application at a first frequency, or stopping sending the Vsync signal to the first application; In response to the second operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; Combining the image rendered in the first application with the image rendered in the second application to obtain a target image; The target image is displayed.

12. An image display method, characterized in that: Applied to electronic equipment, the method includes: Display the first application in full screen; receiving a fourth operation for triggering the second application; In response to the fourth operation, displaying the second application in full screen, and displaying the first application in a floating window on the second application; receiving a fifth operation on the second application; In response to the fifth operation, sending a vertical synchronization signal (Vsync) to the first application at a first frequency, or stopping sending the Vsync signal to the first application; In response to the fifth operation, sending a Vsync signal to the second application at a second frequency; wherein the first frequency is less than the second frequency; Combining the image rendered in the first application with the image rendered in the second application to obtain a target image; The target image is displayed.

13. An image display method, characterized in that: Applied to electronic equipment, the method includes: receiving a sixth operation of the user returning from the first application to the desktop application; In response to the sixth operation, sending a vertical synchronization signal (Vsync) to the first application at a first frequency, or stopping sending the Vsync signal to the first application; In response to the sixth operation, sending a Vsync signal to the desktop application at a second frequency; wherein the first frequency is less than the second frequency; Combining the image rendered in the first application with the image rendered in the desktop application to obtain a target image; The target image is displayed.

14. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is configured to call a program in the memory so that the electronic device executes the method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

16. A computer program product, characterized in that The method comprises a computer program which, when being executed, causes a computer to execute the method according to any one of claims 1 to 13.

Images