Display method, electronic equipment and computer readable storage medium

Abstract

The invention provides a display method, electronic equipment and a computer readable storage medium. In the method, the electronic device can generate the input event based on the operation of the user on the touch screen and then send the input event to the application program without waiting for the synchronization of the vertical synchronization signal. In this way, the electronic device may reduce the latency from the generation of the input event by the input management service to the reception of the input event by the application. According to the embodiment of the invention, the application program can process the input event more quickly to generate the rendering frame, so that the electronic equipment can display the rendering frame on the touch screen more quickly, and the chirality of the electronic equipment is improved.

Description

[0001] This application is a divisional application. The original application has the application number 202410307355.9 and the original application date is March 15, 2024. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to display methods, electronic devices, and computer-readable storage media. Background Technology

[0003] Electronic devices can receive user input on a touchscreen and, in response, change the content displayed on the touchscreen to provide feedback. Sometimes, however, the time it takes for the electronic device to change the displayed content is too long, resulting in a delayed response on the user interface long after the user has completed their action, negatively impacting the user experience. Summary of the Invention

[0004] This application provides a display method, an electronic device, and a computer-readable storage medium. The electronic device can directly send input events to an application without waiting for a vertical synchronization signal after generating them based on user actions. In this method, the application does not need to wait a long time for the input event to arrive and can process the input event as early as possible. This effectively avoids situations where the application reaches the rendering time, but the input event has not yet been processed, thereby improving the responsiveness of the electronic device under the application.

[0005] In a first aspect, this application provides a display method applied to an electronic device, the electronic device having a first application and a second application installed, wherein the first application is a game application and the second application is a non-game application, the method comprising: receiving a first operation performed on the first application and generating a first event based on the first operation; providing the first event to the first application in response to generating the first event; generating a first layer through the first application based on the first event; displaying a first user interface based on the first layer; receiving a second operation performed on the second application and generating a second event based on the second operation; providing the second event to the second application in response to a vertical synchronization (VSync-app) signal; generating a second layer through the second application based on the second event; and displaying a second user interface based on the second layer.

[0006] The first and second events are input events generated based on user actions. The electronic device can determine how to process the input events based on the type of application. When the application is a game, the electronic device can provide the input event directly to the game application after it is generated. When the application is a non-game application, the electronic device can wait for the VSync-app signal to arrive before providing the input event to the application.

[0007] In conjunction with the first aspect, in some embodiments, the first application may include a logic thread and a rendering thread, wherein the logic thread is used to generate first rendering data based on a first event, and the rendering thread is used to generate a first layer based on the first rendering data.

[0008] In conjunction with the first aspect, in some embodiments, the electronic device may include a touchscreen manager and an input management service. The touchscreen manager is used to determine that a first application is an application that does not rely on the VSync-app signal, and the input management service is used to provide a first event to the first application in response to the generation of a first event. The touchscreen manager is also used to determine that a second application is an application that relies on the VSync-app signal, and the input management service is also used to provide a second event to the second application in response to the vertical synchronization VSync-app signal.

[0009] Understandably, non-gaming applications often rely on the VSync-app signal to trigger drawing, rendering, and generating render frames. Therefore, electronic devices need to wait for the VSync-app signal to synchronize when providing input events to non-gaming applications. However, for gaming applications, the timing of generating render frames is controlled by their own engine. In this case, it's unnecessary for the electronic device to wait for the VSync-app signal before sending input events to the gaming application. The electronic device can directly provide the generated input events to the gaming application without waiting for the VSync-app signal. This allows the gaming application to process input events earlier, enabling it to complete the processing when the rendering time arrives, allowing it to generate render frames based on the latest input events, thus improving responsiveness.

[0010] In conjunction with the first aspect, in some embodiments, the first event is generated and provided to the first application between a first VSync-app signal and a second VSync-app signal, which are two adjacent vertical synchronization signals.

[0011] Understandably, since electronic devices do not wait for the arrival of VSync-app signals when providing input events to game applications, the timing of the first event being sent to the first application after it is generated between two adjacent VSync-app signals is also between these two adjacent VSync-app signals.

[0012] In conjunction with the first aspect, in some embodiments, before providing the first event to the first application, the method further includes: stopping the transmission of the VSync-app signal; before providing the second event to the second application, the method further includes: retransmitting the VSync-app signal.

[0013] In conjunction with the first aspect, in some embodiments, the first user interface does not include a layer for non-game applications.

[0014] The electronic device iterates through the layers contained in the first user interface. When the first user interface does not contain layers for non-game applications, the electronic device can directly stop sending VSync-app signals, thus reducing power consumption. However, when the electronic device determines that the second user interface contains layers for non-game applications, since non-game applications rely on the VSync-app signal to trigger drawing and rendering processes, the electronic device needs to restart sending the VSync-app signal to ensure that non-game applications can stably output rendered frames. Here, "the first user interface does not contain layers for non-game applications" means that the first user interface displayed on the electronic device's touchscreen only contains the user interface for game applications; conversely, "the second user interface contains both game application layers and non-game application layers" means that the second user interface displayed on the touchscreen includes both game application and non-game application user interfaces. These game application and non-game application user interfaces can be displayed as two separate user interfaces, or one user interface can be displayed globally while the other is displayed in a window, etc., simultaneously on the touchscreen.

[0015] In conjunction with the first aspect, in some embodiments, the second user interface further includes a third layer of the first application, wherein the frame rate of the first application is a first frame rate, the frame rate of the second application is a second frame rate, and the first frame rate is greater than the second frame rate.

[0016] by Figure 8 In the illustrated embodiment, the frame rate of the game application is twice that of the non-game application. The game application controls the output of rendered frames using its own engine, while the non-game application primarily relies on the VSync-app signal to output rendered frames. The second user interface can be image 1, the third layer can be layer 1, and the second layer can be layer 1'. When the electronic device determines that the second user interface includes layers for both the game application and the non-game application, it can control the frame rate of the non-game application by adjusting the frequency of the VSync-app signal, making the non-game application's frame rate lower than that of the game application. The electronic device may include a surface synthesizer, which can be used to determine the layers (including the second and third layers) included in the second user interface and adjust the transmission frequency of the VSync-app signal to the second frame rate.

[0017] When gaming applications output rendered frames at a high frame rate, the load on electronic devices is high, while non-gaming applications do not need to refresh the display at the same high frame rate. Electronic devices can control the frequency of VSync-app to allow non-gaming applications to output rendered frames at a lower frame rate. This ensures that users can use both gaming and non-gaming applications normally at the same time, and also reduces the power consumption of electronic devices.

[0018] In conjunction with the first aspect, in some embodiments, before generating the first event according to the first operation, the method further includes: determining that the sampling rate of the touch screen is a multiple of the refresh rate, and determining that the first application is in the first list.

[0019] Optionally, before generating the first event based on the first operation, the method further includes: determining that the sampling rate of the touch screen is a multiple of the refresh rate, or determining that the first application is in the first list.

[0020] In conjunction with the first aspect, in some embodiments, before generating the second event according to the second operation, the method further includes: determining that the sampling rate of the touch screen is not a multiple of the refresh rate, and / or determining that the second application is not in the first list.

[0021] The electronic device can store the identifiers of one or more applications in a first list. This first list may store identifiers of applications that do not depend on the VSync-app signal. "Does not depend on the VSync-app signal" means that the application does not need to rely on VSync-app for synchronization during drawing, rendering, and generating render frames. The electronic device can send input events to such applications without waiting for the VSync-app signal to ensure the application's functionality. Therefore, the electronic device can first determine whether the application's identifier is in the first list, and then determine whether the application needs to depend on the VSync-app signal. Optionally, the application identifier can be the application's package name.

[0022] When the touchscreen's sampling rate is a multiple of the refresh rate, the number of input events generated by the electronic device between every two adjacent image frames is consistent, meaning the timing of input event generation is more uniform. In this case, the application generates rendering frames based on the same number of input events each time, and even if the electronic device doesn't wait for the VSync-app signal when providing input events to the application, the application can still output stable rendering frames. However, if the touchscreen's sampling rate is not a multiple of the refresh rate, the number of input events generated by the electronic device between every two adjacent image frames may be inconsistent. For example, the electronic device might generate two input events in one VSync-app cycle and one input event in the next VSync-app cycle. In this case, the application generates rendering frames based on different numbers of input events each time. Without waiting for the VSync-app signal for synchronization, the number of input events used by the application to generate each rendering frame is difficult to control, making it easier to cause uneven screen display. Therefore, before determining whether sending input events to the application does not require waiting for the VSync-app signal, the electronic device can first determine whether the touchscreen's sampling rate is a multiple of the refresh rate, thus ensuring that the application can output stable rendering frames.

[0023] Electronic devices can provide input events to an application without relying on the VSync-app signal, provided that the touchscreen's sampling rate is a multiple of the refresh rate and / or the application's identifier is in the first list. This ensures that the application can still output stable rendered frames even without relying on the VSync-app signal, thus guaranteeing a stable user interface display and a positive viewing experience.

[0024] Secondly, this application provides a display method applied to an electronic device having a first application installed. The method includes: receiving a first operation performed on the first application and generating a first event based on the first operation; providing the first event to the first application in response to generating the first event; generating a first layer based on the first event through the first application; and displaying a first user interface based on the first layer.

[0025] In conjunction with the second aspect, in some embodiments, the first event is generated and provided to the first application between a first VSync-app signal and a second VSync-app signal, which are two adjacent vertical synchronization signals.

[0026] In conjunction with the second aspect, in some embodiments, the first user interface contains only a first layer, and the method further includes: stopping the transmission of VSync-app signals.

[0027] In conjunction with the second aspect, in some embodiments, before generating the first event according to the first operation, the method further includes: determining that the sampling rate of the touch screen is a multiple of the refresh rate, and determining that the first application is in a first list.

[0028] In conjunction with the second aspect, in some embodiments, the electronic device is further equipped with a second application, and the method further includes: receiving a second operation acting on the second application and generating a second event based on the second operation; providing the second event to the second application in response to a vertical synchronization VSync-app signal; generating a second layer through the second application based on the second event; and displaying a second user interface based on the second layer.

[0029] In conjunction with the second aspect, in some embodiments, the method further includes retransmitting the VSync-app signal before providing the second event to the second application.

[0030] In conjunction with the second aspect, in some embodiments, the second user interface further includes a third layer of the first application, wherein the frame rate of the first application is a first frame rate, the frame rate of the second application is a second frame rate, and the first frame rate is greater than the second frame rate.

[0031] In conjunction with the second aspect, in some embodiments, the electronic device includes a surface synthesizer for determining that the second user interface includes a second layer and a third layer; the surface synthesizer is also used to adjust the transmission frequency of the VSync-app signal to a second frame rate.

[0032] In conjunction with the second aspect, in some embodiments, before generating the second event according to the second operation, the method further includes: determining that the sampling rate of the touch screen is not a multiple of the refresh rate, and / or determining that the second application is not in the first list.

[0033] In conjunction with the second aspect, in some embodiments, the electronic device includes a touchscreen manager and an input management service, wherein the touchscreen manager is used to determine that the first application is an application that does not rely on the VSync-app signal, and the input management service is used to provide the first event to the first application in response to the generation of the first event.

[0034] In conjunction with the second aspect, in some embodiments, the input management service is also used to determine whether the second application is an application that depends on the VSync-app signal, and to provide a second event to the second application in response to the vertical synchronization VSync-app signal.

[0035] In conjunction with the second aspect, in some embodiments, the first application includes a logic thread and a rendering thread, the logic thread being used to generate first rendering data based on a first event, and the rendering thread being used to generate a first layer based on the first rendering data.

[0036] In conjunction with the second aspect, in some embodiments, the first application is a game application.

[0037] In conjunction with the second aspect, in some embodiments, the second application is a non-gaming application.

[0038] Thirdly, this application provides an electronic device including a touch screen, a memory, and a processor coupled to the memory; the touch screen is used to receive user operations and display an interface, the memory stores a computer program, and when the processor executes the computer program, the electronic device implements the method described in either the first or second aspect.

[0039] Fourthly, this application provides a computer-readable storage medium storing a computer program or computer instructions, which are executed by a processor to implement the method described in any one of the first or second aspects above.

[0040] Fifthly, embodiments of this application provide a computer program product, which, when executed by a processor, implements the method described in either the first or second aspect described above.

[0041] In a sixth aspect, embodiments of this application provide a chip including a processor and a memory, wherein the memory is used to store computer programs or computer instructions, and the processor is used to execute the computer programs or computer instructions stored in the memory, causing the chip to perform the method described in either the first or second aspect above.

[0042] The solutions provided in the third to sixth aspects above are used to implement or cooperate with the methods provided in the first or second aspects above, and therefore can achieve the same or corresponding beneficial effects as the methods in the first or second aspects, which will not be elaborated here. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application;

[0044] Figure 2 This is a software structure block diagram of the electronic device provided in the embodiments of this application;

[0045] Figures 3A-3B These are user interfaces of some electronic devices in game scenarios provided in the embodiments of this application;

[0046] Figure 4 This is a schematic diagram illustrating the process of displaying game screens on an electronic device according to an embodiment of this application;

[0047] Figure 5 This is a schematic diagram illustrating another process for displaying game screens on an electronic device according to an embodiment of this application;

[0048] Figure 6 This is a flowchart illustrating how an electronic device controls the VSync-app signal to be turned on or off, as provided in an embodiment of this application.

[0049] Figures 7A-7B These are user interfaces displayed on some electronic devices provided in the embodiments of this application;

[0050] Figure 8 This is a schematic diagram illustrating the process of an electronic device displaying a user interface at different frame rates in a multi-window environment, as provided in an embodiment of this application.

[0051] Figure 9 This is a flowchart of a display method provided in an embodiment of this application;

[0052] Figure 10 This is a schematic diagram of the structure of a display device provided in an embodiment of this application;

[0053] Figure 11 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0054] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0055] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0056] The hardware structure of the electronic device 100 provided in this application embodiment is introduced first here.

[0057] Figure 1 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application.

[0058] The following description uses electronic device 100 as an example to illustrate the embodiment. It should be understood that electronic device 100 may have more than Figure 1The more or fewer components shown can be combined into two or more components, or they can have different component configurations. Figure 1 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0059] Electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a camera 193, a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a touch sensor 180B, etc.

[0060] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0061] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0062] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0063] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0064] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, etc.

[0065] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0066] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0067] Camera 193 is used to capture still images or videos. Electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0068] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0069] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats.

[0070] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0071] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0072] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as facial recognition, fingerprint recognition, mobile payment, etc.). The data storage area may store data created during the use of electronic device 100 (such as facial information template data, fingerprint information templates, etc.). Furthermore, internal memory 121 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0073] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0074] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0075] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0076] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0077] The microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals.

[0078] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands.

[0079] Touch sensor 180B, also known as a "touch panel," can be located on display screen 194. The touch sensor 180B and display screen 194 together form a touchscreen, also known as a touch display. Touch sensor 180B is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180B may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0080] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to illustrate the software structure of the electronic device.

[0081] Figure 2 This is a software structure block diagram of the electronic device provided in the embodiments of this application.

[0082] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the native libraries layer, and the kernel layer.

[0083] The application layer can include a series of application packages. For example... Figure 2 As shown, the application layer can include applications such as games and instant messaging.

[0084] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer includes predefined functions. It may include game scene optimization services, input management services, and more.

[0085] Game scene optimization service is used to schedule system resources of electronic devices to optimize the gaming experience.

[0086] The input management service can resample and process touch events generated by the touch driver, and then distribute the processed input events to the application. The application can then process the input events based on its own logic to generate a new user interface. In some embodiments, the input management service can receive control information from the touchscreen manager, and then determine whether to wait for a vertical synchronization signal when sending an input event to the application. A description of the vertical synchronization signal can be found in subsequent embodiments and will not be elaborated here.

[0087] A native library can include multiple functional modules. For example, a graphics processing library, a surface flinger, and a touchscreen manager.

[0088] Graphics processing libraries are used to implement graphics drawing, image rendering, compositing, and layer processing.

[0089] A surface compositer is used to fuse two-dimensional (2D) and three-dimensional (3D) layers for multiple applications, thereby generating a user interface to be displayed. An application's user interface can be considered a layer. The user interface to be displayed on a touchscreen may contain the user interfaces of multiple applications. The surface compositer composites these layers according to certain rules, merging the user interfaces of different applications into a single user interface for subsequent display on the touchscreen. In some embodiments, the surface compositer can control the activation or deactivation of the vertical synchronization signal based on the type and number of layers contained in the user interface to be displayed. A description of the vertical synchronization signal can be found in subsequent embodiments and will not be elaborated upon here.

[0090] The touchscreen manager can be used to acquire or adjust touchscreen parameters and control the touchscreen display. In some embodiments, the touchscreen manager can determine whether the application needs to rely on a vertical synchronization signal to generate rendering frames based on the application's identifier. A description of the vertical synchronization signal can be found in subsequent embodiments and will not be elaborated upon here.

[0091] The kernel layer is the layer between hardware and software. At a minimum, the kernel layer includes GPU drivers, touch drivers, display drivers, etc. The kernel layer receives instructions from the software layer and controls the hardware layer to perform corresponding tasks.

[0092] The GPU driver can be used to drive the GPU to further render the images drawn by the application, generating the application's user interface (i.e., the application's layers).

[0093] Touch drivers can receive user actions on the touchscreen and process the touch actions into touch events (including touch coordinates, timestamps of the touch actions, etc.).

[0094] The display driver pushes the user interface to be displayed into the frame buffer (FB). The frame buffer is a storage space in the electronic device 100, which can be located in either video memory or main memory. The contents of the frame buffer correspond to the interface display on the touch screen; it can be simply understood as a cache corresponding to the content displayed on the touch screen. In other words, modifying the contents of the frame buffer modifies the image displayed on the touch screen, thus allowing the touch screen to display the user interface. The touch screen consists of a touch sensor 180B and a display screen 194.

[0095] Understandable Figure 2 The illustrated software architecture does not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include... Figure 2 This can be shown as having more or fewer modules, combining some modules, or splitting some modules, etc.

[0096] Electronic devices typically change part or all of their user interface content based on user actions to achieve human-computer interaction. The shorter the delay between the electronic device receiving the user's action and the corresponding change in displayed content, the better the device's responsiveness. When an electronic device has poor responsiveness, the device may only respond on the user interface a long time after the user completes their action, which significantly impacts the user experience in scenarios where responsiveness is crucial.

[0097] This section describes scenarios where electronic devices have poor responsiveness.

[0098] Figures 3A-3BThese are user interfaces for some electronic devices in game scenarios provided in the embodiments of this application.

[0099] like Figure 3A As shown, the electronic device 100 can display a game interface 301, which may include a wheel control 302 and a character image 303. The wheel control 302 is first used to control the movement of the character image 303. At time t1, the electronic device 100 can receive an operation from the user who presses the wheel control 302 and slides it to the right for time A and then releases it. At time t2, in response to the above operation, the electronic device 100 controls the character image 303 to move to the right at a preset speed for time A until... Figure 3B The position shown. When the difference between time t1 and time t2 is large, the user will perceive that the character texture 303 is not able to move in time according to the user's operation, that is to say, the responsiveness of the electronic device is poor.

[0100] To address this issue, embodiments of this application provide a display method, an electronic device, and a computer-readable storage medium. In this method, after receiving a user's operation on a touchscreen, the electronic device can directly send the input event generated based on the user's operation to the application program without waiting for a vertical synchronization signal. This allows the application program to acquire the input event more quickly and generate a rendering frame based on it. The electronic device can then display the rendering frame on the touchscreen more quickly to provide feedback to the user's operation, thus improving the responsiveness of the electronic device.

[0101] Among these scenarios, gaming demands a high degree of responsiveness from electronic devices. Users typically require timely feedback on the touchscreen from their actions while playing games. For ease of description and better understanding, subsequent embodiments of this application will use gaming scenarios as an example to illustrate the display method provided. However, this display method is not limited to gaming scenarios and can be applied to other scenarios as well. This application does not impose any limitations on the application scenarios of the above-described display method in electronic devices.

[0102] When displaying a user interface, electronic devices typically require vertical synchronization (VSync) signals for coordination. VSync signals can include VSync-app, VSync-sf, and HW-VSync signals. The VSync-app signal triggers the input management service to send input events to the application and notifies the application to begin drawing and generating a new frame of image. The VSync-sf signal triggers the surface compositor to composite the user interface to be displayed based on layers from one or more applications. The HW-VSync signal triggers the display driver to send the user interface to be displayed. The VSync-app and VSync-sf signals can be sent by the electronic device at preset time intervals.

[0103] The following section describes in detail the process of displaying a user interface on an electronic device.

[0104] When the touch sensor receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The touch driver in the kernel layer sends the raw input event to the input management service. The input management service performs resampling and other processing on the raw input event, and then sends the processed input event to the game application when the VSync-app signal arrives.

[0105] After receiving an input event, the game application can update its logic based on the input event. This can include updating physical information (such as speed, position, and collision handling), changing object states, and so on. Figure 3A(The coordinates of the moved character texture 303 are shown). The game application can then draw and render the user interface to be displayed based on the results of the above logical update (hereinafter referred to as rendering data), generating rendering frames. The threads responsible for logical updates and rendering frame generation in the game application can be independent of each other. The thread responsible for logical updates is called the logic thread, and the thread responsible for generating rendering frames is called the rendering thread. Furthermore, the game application does not rely on the VSync-app signal to trigger the drawing and rendering of layers; instead, the game engine controls the frequency of rendering frame generation by the rendering thread (while most non-game applications need to wait for VSync-app before starting the drawing and rendering of a layer). Although the game application does not respond to the VSync-app signal to generate rendering frames, the time interval between any two adjacent rendering frames generated by the game application is consistent. Specifically, the time interval between two adjacent rendering frames generated by the game application can be equal to the time interval between two adjacent user interface frames refreshed by the touchscreen; that is, the frequency of rendering frame generation by the game application can be consistent with the touchscreen refresh rate, ensuring that each rendering frame generated by the game application can ultimately be displayed. Optionally, the electronic device may include a synchronization signal to trigger the game application to generate rendering frames. The timing of sending this synchronization signal may differ from the timing of sending the VSync-app signal. After receiving an input event, the game application's logic thread first handles the logic update, and then sends the updated result to the rendering thread for drawing, rendering, etc. A rendering frame generated by the application is also a layer of the application. After generating the layer, the application can send the layer to the surface compositor.

[0106] After receiving the VSync-sf signal, the Surface Composer can composite layers from different applications into a single image, resulting in the user interface to be displayed on the touchscreen, also known as the user interface to be displayed (or the image to be displayed). After generating the user image to be displayed, SurfaceFlinger can send it to the display driver.

[0107] The display driver can send the image to be displayed after receiving HW-VSync, and then the touch screen can display the image.

[0108] by Figures 3A-3B In the illustrated embodiment, after the electronic device receives a user's action of pressing and sliding the wheel control 302 to the right, the input management service generates input events for pressing and sliding to the right based on the operation. Then, upon receiving the VSync-app signal, the input management event sends the generated input event to the game application. The game application can perform logic updates based on the pressing and sliding input events. For example, the game application determines that after the operation, the character texture 303 should be changed from... Figure 3A Move to the position shown Figure 3B At the indicated location, the game application will then complete the rendering of one frame of the user interface, which is referred to here as the rendering frame. In this rendering frame, the character texture 303 is located at... Figure 3B At the indicated location. The game application can then send the generated rendered frame to SurfaceFlinger. Upon receiving the VSync-sf signal, SurfaceFlinger composites the layers to generate the frame to be displayed. The display driver, upon receiving the HW-VSync signal, sends this frame to the display, and finally, the touchscreen displays the frame. Figure 3B The user interface shown.

[0109] The time interval between two adjacent VSync-app signals emitted by the electronic device can be the same as or different from the time interval between two adjacent VSync-sf signals emitted by the electronic device and the time interval between two adjacent HW-VSync signals emitted by the electronic device. Furthermore, the timing of the electronic device emitting the VSync-app, VSync-sf, and HW-VSync signals each time can be the same or different. In some embodiments, the timing of the electronic device emitting the VSync-app, VSync-sf, and HW-VSync signals may be offset, and this application embodiment does not impose limitations on this. For ease of description and better understanding, the following embodiments of this application describe the display method provided by this application embodiment with the electronic device emitting the VSync-app, VSync-sf, and HW-VSync signals at the same time each time. The display method when the electronic device emits the VSync-app, VSync-sf, and HW-VSync signals in other ways can refer to the method where the electronic device emits the VSync-app, VSync-sf, and HW-VSync signals at the same time each time, and this application embodiment will not elaborate on this.

[0110] This section describes a scenario where electronic devices display game graphics.

[0111] Figure 4 This is a schematic diagram illustrating the process of displaying game screens on an electronic device according to an embodiment of this application.

[0112] like Figure 4 As shown, the electronic device sends the first VSync-app signal ( Figure 4 During the time interval between the first VSync-app signal (represented by VSync-app1) and the second VSync-app signal, the input management service generates input event 1 (…). Figure 4 (This refers to Incident 1 in China).

[0113] After the electronic device sends the second VSync-app signal, the input management service can respond to the VSync-app signal by sending input event 1 to the game application. The game application's logic thread can generate rendering data 1 based on event 1, and then the rendering thread obtains the rendering data 1 from the logic thread and generates layer 1 based on the rendering data 1. Between the second and third VSync-app signals, the input management service also generates input event 2.

[0114] In response to the third VSync-app signal, the input management service can send input event 2 to the game application. The game application's logic thread can process input event 2 and generate rendering data 2. The rendering thread then generates layer 2 based on rendering data 2. The surface compositor can respond to the VSync-sf signal (…). Figure 4 Image 1 (not shown) is synthesized based on layer 1. Between the third and fourth VSync-app signals, the input management service also generates input event 3.

[0115] In response to the fourth VSync-app signal, the input management service can send input event 3 to the game application. The method by which the game application handles input event 3 is similar to the methods used for handling input events 1 and 2 described above, and will not be repeated here. The surface synthesizer can respond to the VSync-sf signal ( Figure 4 Image 2 (not shown) is composited based on layer 2. The display driver can be based on the HW-VSync signal ( Figure 4 (Not shown in the image) The synthesized image 1 is sent to the display, and then the touch screen displays image 1.

[0116] It should be noted that game applications will generate rendering frames based on their own logic even if they do not receive input events. Figure 4 The image only shows the process of the electronic device displaying the user interface after a user action causes the game application's user interface to be redrawn. Additionally, Figure 4 The frequency at which the input management service generates input events is only used as an example; the input management service can generate more than [a certain number of events] between every two adjacent VSync-apps. Figure 4 The number of input events shown may be more or less, and the number of input events generated in two adjacent VSync-app cycles may be the same or different, which is not limited in this application embodiment.

[0117] In game applications, the logic thread generates rendering data and the rendering thread generates rendering frames independently. The rendering thread does not wait for the logic thread to finish processing input events before rendering; instead, it renders based on its preset cycle. If the logic thread cannot process the input events before the rendering thread starts generating the current cycle's rendering frames, the rendering thread can only synchronize the input events in the next frame layer. This leads to poor responsiveness on electronic devices.

[0118] by Figure 4 Taking the second and third VSync-app signals as an example, if the game application's logic thread is still generating rendering data 1, while the rendering thread has reached the point where it needs to start generating a new rendering frame, then the rendering thread cannot generate layer 1 based on the latest rendering data (i.e., rendering data 1). Consequently, layer 1 cannot reflect the screen changes caused by input event 1. In this case, the rendering thread can only process rendering data 1 when generating subsequent rendering frames (layer 2). Therefore, the user interface displayed on the electronic device cannot provide timely feedback based on user actions, meaning the electronic device has poor responsiveness.

[0119] For most non-gaming applications, drawing and rendering to generate render frames rely on the VSync-app signal. Therefore, the input management service needs to wait for the VSync-app signal to synchronize when sending input events to the application. However, for gaming applications, which do not rely on the VSync-app signal to trigger render frame generation, it is unnecessary for the input management service to wait for the VSync-app signal before sending input events. To improve the responsiveness of electronic devices, the latency between the input management service generating input events and the game application receiving them can be reduced. The earlier the game application receives input events, the earlier its logic threads can process them, allowing the rendering thread to receive rendering data and generate render frames earlier.

[0120] In some embodiments, the input management service in an electronic device can send input events to the game application immediately after they are generated, without waiting for the VSync-app signal.

[0121] Figure 5 This is a schematic diagram illustrating another process for displaying game screens on an electronic device, as provided in an embodiment of this application.

[0122] like Figure 5As shown, after generating an input event, the input management service no longer waits for the VSync-app signal but sends it directly to the game application. Between VSync-app1 and VSync-2, the input management service generates input event 1. Input event 1 is immediately sent to the game application without waiting for VSync-app2. This allows the game application to receive the input event faster, enabling the logic thread to generate rendering data 1 based on input event 1. The rendering thread receives the rendering data faster and generates layer 1 based on it. After receiving layer 1, the surface compositor can composite the image to be displayed (containing layer 1 generated by the game application) after receiving the next VSync-sf signal. The display driver then sends this frame to the screen, and the touchscreen ultimately displays it.

[0123] Similarly, other input events generated by the input management service can be sent directly to the game application. The method for electronic devices to handle other input events can refer to the method for handling input event 1 above, and will not be repeated here.

[0124] It should be noted that, Figure 5 This only demonstrates one scenario of how a game application generates rendered frames. The game application may generate rendered frames based on input events earlier than [the actual timeframe] within each VSync-app cycle. Figure 5 The timing shown may also be later than Figure 5 The timing shown in the embodiments of this application is not limited in this respect.

[0125] As can be seen, because the input management event no longer needs to wait for the VSync-app signal when sending input events, the game application can receive input events earlier. This reduces the latency of input events traveling from the input management service to the game application, thus preventing the rendering thread from being unable to generate rendering frames based on new input events due to the logic thread's inability to process them in time. Under this method, the game application's logic thread can process input events earlier and then hand them over to the rendering thread to generate rendering frames. The image frames corresponding to each input event can be displayed on the touchscreen faster. This allows users to see the electronic device's feedback based on user actions on the touchscreen more quickly, thereby improving the user's gaming experience.

[0126] For gaming applications, they can generate rendering frames at a certain frequency based on their own engine, thus outputting rendering frames relatively stably even if the gaming application does not rely on the VSync-app signal. Other applications, however, require the VSync-app signal for synchronization when generating rendering frames. In some embodiments, to ensure that applications can generate more stable rendering frames, the electronic device can determine whether the VSync-app signal is enabled or disabled based on the application type.

[0127] Figure 6 This is a flowchart illustrating how an electronic device controls the VSync-app signal to be turned on or off, as provided in an embodiment of this application.

[0128] like Figure 6 As shown, the method for an electronic device to control the VSync-app signal to be turned on or off may include the following steps:

[0129] S601, Application A starts.

[0130] S602. Application A sends message A to the input management service, indicating that application A has started.

[0131] S603, The input management service sends the identifier of application A to the touch screen manager.

[0132] After application A starts, it can send message A to the input management service. The input management service can then send the identifier of application A to the touchscreen manager based on message A. The identifier of application A can be the package name of application A. Optionally, application A can be a game application.

[0133] Optionally, the electronic device may add all the identifiers of game applications stored in the electronic device to the enable list, or the electronic device may only store the identifiers of some of the game applications stored in the electronic device to the enable list. The identifiers of the applications stored in the enable list may be added in response to user operations, or they may be added by the electronic device based on preset rules; this embodiment of the application does not impose any limitations on this.

[0134] The enable list is not limited to game applications; it may also include identifiers for other types of applications. For example, the enable list may also include identifiers for other applications that do not rely on the VSync-app signal to trigger drawing and rendering of the user interface. This application embodiment does not impose any limitations on this.

[0135] S604, The touchscreen manager determines whether application A is in the enabled list based on the identifier of application A.

[0136] The touchscreen manager can check if an application's identifier exists in the enable list. If an application's identifier exists in the enable list, it indicates that the application does not require the VSync-app signal for synchronization.

[0137] When the application's identifier is not present in the enable list, the touchscreen manager can determine that the application needs to rely on the VSync-app signal. The application's reliance on the VSync-app signal means that the application needs to rely on the VSync-app signal to trigger the drawing and rendering of the user interface.

[0138] For ease of description and better understanding, in the following embodiments, applications identified as being in the enable list are referred to as in-list applications, and applications identified as not being in the enable list are referred to as non-list applications. In-list applications do not require the VSync-app signal to generate rendering frames according to a preset period, while non-list applications need to wait for the VSync-app signal to arrive before starting to draw, render, and generate rendering frames. When the electronic device determines that an application does not require the VSync-app signal, it may choose not to send the VSync-app signal to the application, or it may still send the VSync-app signal to the application; this application embodiment does not impose any restrictions on this.

[0139] S605, Touchscreen Manager queries the Surface Compositeer for the touchscreen refresh rate.

[0140] S606, the touch screen manager queries the touch screen sampling rate from the touch driver.

[0141] S607, Touchscreen Manager determines the touchscreen refresh rate as a multiple of the sampling rate.

[0142] When the touchscreen's sampling rate is not a multiple of the refresh rate, the number of input events generated by the input management service may differ between two adjacent VSync-app signals. For example, the input management service might generate one input event between the Kth and (K+1th)th VSync-app signals, and two input events between the (K+1th)th and (K+2th)th VSync-app signals, where K is any positive integer. If the application does not rely on the VSync-app signals, the number of input events it depends on each time it generates a rendering frame may be uncontrollable, potentially causing display jitter and affecting the user's viewing experience. The refresh rate represents the number of user interfaces displayed per second on an electronic device's touchscreen.

[0143] In some embodiments, after determining that the application's identifier is in the enable list, the touchscreen manager further determines, based on the touchscreen's sampling rate and refresh rate, whether the input management service needs to rely on the VSync-app signal to send input events to application A in the current touchscreen state. Only when the application's identifier exists in the application scheme list and the touchscreen's sampling rate is a multiple of the refresh rate, does the touchscreen manager determine that the input management service can send input events to application A without relying on the VSync-app signal, thereby improving the responsiveness of the electronic device running application A. This avoids the electronic device focusing solely on improving responsiveness while neglecting the stability of the display, leading to a degraded user experience.

[0144] S608, the touchscreen manager sends control information A to the input management service, indicating that application A does not need to rely on the VSync-app signal.

[0145] S609. When the input management service determines to send an input event to application A based on control information A, it does not need to wait for the VSync-app signal.

[0146] S610, The input management service sends control information B to the surface synthesizer. Control information B is used to indicate that the VSync-app signal should be turned off.

[0147] After receiving control information A from the touchscreen manager, the input management service can determine that it no longer needs to wait for the VSync-app signal when sending input events to the application.

[0148] Since periodically sending VSync-app signals by electronic devices consumes certain resources, in some embodiments, the input management service can send control information B to the surface synthesizer. After receiving control information B, the surface synthesizer can determine whether to turn off the VSync-app signal based on the layers contained in the user interface to be displayed. Optionally, control information B can also be sent by other modules in the electronic device, such as the touch screen manager; this application embodiment does not limit this.

[0149] S611, The surface synthesizer traverses the layers of the user interface to be displayed.

[0150] S612, The surface synthesizer determines the layers of the user interface to be displayed that are only included in the enable list of the applications.

[0151] S613, Surface synthesizer disables VSync-app signal.

[0152] After receiving control signal B, the surface synthesizer can traverse the layers contained in the user interface to be displayed. When the surface synthesizer determines that only the layers of applications included in the enable list are present, it can turn off the VSync-app signal (or keep the VSync-app signal off) to save resources. The reason why the layers of the user interface to be displayed include layers of multiple applications is that the user interfaces of multiple applications are displayed in a composite manner on the touch screen. For example, the above composite display can include, but is not limited to, split-screen display, windowed display, etc.

[0153] In some embodiments, the user interface to be displayed may include layers for applications not included in the enable list, as well as layers for applications not included in the enable list. The surface synthesizer can keep the VSync-app signal enabled. Specifically, the input management service does not need to wait for the VSync-app signal when sending input events to applications within the enable list (e.g., application A), but still needs to wait for the VSync-app signal when sending input events to applications not included in the enable list.

[0154] When multiple application user interfaces are displayed on a touchscreen, electronic devices consume significant power, and different applications require different frame rates. Take application A and application B as examples, where application A is an in-list application (e.g., a game) and application B is a non-list application (e.g., an instant messaging application). When the user interface to be displayed on the touchscreen contains layers of both application A and application B, to ensure a good user experience, application A might be displayed at a higher frame rate. Correspondingly, the electronic device will also increase the touchscreen refresh rate to ensure that every frame of layer generated by application A is displayed. In this case, if application B also generates layers at a high frame rate, it is unnecessary for the user and will unnecessarily increase the power consumption of the electronic device. In some embodiments, the surface synthesizer can adjust the frequency of the VSync-app signal. For example, the surface synthesizer can reduce the frequency of the VSync-app signal to be lower than the frame rate of application A. This way, application A, which does not rely on the VSync-app signal, can still display the user interface at a higher frame rate, while application B, which relies on VSync-app, can display the user interface at a lower frame rate, thereby reducing the power consumption of the electronic device while ensuring a good user experience. Frame rate refers to the number of user interfaces displayed per second by the application, that is, the number of rendered frames generated per second. An application's frame rate may differ from the refresh rate of an electronic device, and different applications may have the same or different frame rates.

[0155] In some embodiments, steps S611-S613 can be executed cyclically. That is, the surface compositor iterates through the layers contained in the user interface to be displayed when compositing each frame of the user interface. Furthermore, the surface compositor can control the VSync-app signal to be turned on or off when the layers contained in the user interface to be displayed change.

[0156] In some embodiments, in step S604, the touchscreen manager may also determine that the application's identifier is not in the enable list, and / or in step S607, the touchscreen manager determines that the touchscreen refresh rate is not a multiple of the sampling rate. In these cases, the touchscreen manager can send control information C to the input management service, indicating that the aforementioned application requires the VSync-app signal. The touchscreen manager can determine that sending an input event to the application requires the VSync-app signal. Furthermore, the touchscreen manager sends control information D to the surface synthesizer, instructing the surface synthesizer to enable the VSync-app signal. The surface synthesizer can then traverse the layers of the user interface to be displayed. If it contains only layers of applications not in the enable list, it can change the VSync-app signal from off to on (or keep the VSync-app signal on). If the layers of the user interface to be displayed also contain layers of applications in the enable list, the surface synthesizer can adjust the frequency of the VSync-app signal in addition to enabling it. In this way, the input management service does not need to wait for the VSync-app signal when it needs to send input events to applications in the list, while sending input events to non-list applications is still triggered by VSync-app.

[0157] In some embodiments, steps S605-S607 are optional, that is, the touchscreen manager may determine whether the input management service should wait for the VSync-app signal when sending an input event to application A based solely on whether the identifier of application A is in the enable list.

[0158] In some embodiments, step S604 is optional, that is, the touch screen manager can determine whether the input management service should wait for the VSync-app signal when sending an input event to application A based solely on whether the touch screen's sampling rate is a multiple of the refresh rate.

[0159] When the user interface displayed on the touchscreen includes both user interfaces of listed and non-listed applications, the electronic device can adjust the frequency of the VSync-app signal to save power.

[0160] The following describes some scenarios of electronic device display user interfaces provided in the embodiments of this application.

[0161] Figures 7A-7B These are user interfaces displayed on some electronic devices provided in the embodiments of this application.

[0162] like Figure 7A As shown, the electronic device displays the user interface 701 of a game application, where the game application is an application in a list. Since the user interface displayed by the electronic device only contains the user interface 701 of the game application, the electronic device can turn off (or keep off) the VSync-app signal to improve the responsiveness of the game application, thereby improving the user's gaming experience.

[0163] like Figure 7B As shown, the electronic device 100 displays a user interface 701 for a game application and a user interface 702 for an instant messaging application. The instant messaging application is a non-list application, meaning its frame rate is consistent with the frequency of the VSync-app signal. The entire user interface displayed on the touchscreen (including user interface 701 and user interface 702) is composited by a surface synthesizer receiving the corresponding layers of user interface 701 from the game application and the corresponding layers of user interface 702 from the instant messaging application. The electronic device can enable (or keep enabled) VSync-app and adjust the frequency of the VSync-app signal to control the power of the instant messaging application.

[0164] For example, to ensure a good gaming experience, game applications refresh the screen at a frame rate A, where frame rate A is relatively high (e.g., 90Hz, 120Hz, etc.). Electronic devices can adjust VSync-sf and HW-VSync to match frame rate A, ensuring that every layer generated by the game application can be displayed. The frame rate of an instant messaging application is called frame rate B. Since users don't have high refresh rate requirements for instant messaging applications, and instant messaging applications rely on the VSync-app signal to output rendering frames, the frequency of the VSync-app signal determines the frame rate B of the instant messaging application. Electronic devices can set the VSync-app signal to be lower than frame rate A, thus making the frame rate B of the instant messaging application lower than frame rate A. This allows electronic devices to ensure that game applications display at a high frame rate to guarantee a good gaming experience, while also ensuring that the frame rate of instant messaging applications meets user needs while reducing the power consumption of the electronic device, thereby improving the user experience in scenarios with multiple applications.

[0165] Figure 8 This is a schematic diagram illustrating the process of an electronic device displaying a user interface at different frame rates in a multi-window environment, as provided in an embodiment of this application.

[0166] For ease of description, we will use the assumption that the frame rate of an application in the list is twice that of an application outside the list. For example, if the frame rate of an application in the list is 120Hz and the frame rate of an application outside the list is 60Hz, then the time interval between two adjacent rendered frames generated by an application in the list is twice the time interval between two adjacent rendered frames generated by an application outside the list. Furthermore, applications in the list do not depend on the VSync-app signal when generating rendered frames, while applications outside the list only begin generating rendered frames upon receiving the VSync-app signal.

[0167] Between the first and second VSync-sf signals, the application within the list generates layer 1, and the application outside the list generates layer 1'. After receiving layer 1 and layer 1', the surface compositor can synthesize a frame of user interface to be displayed when the second VSync-sf signal arrives: image 1. This frame of user interface contains both the rendered frame generated by the application within the list (layer 1) and the rendered frame generated by the application outside the list (layer 1').

[0168] Between the second and third VSync-sf signals, the in-list applications generate layer 2. At this time, since the non-list applications are still waiting for VSync-app2, the non-list applications do not generate new rendering frames.

[0169] Between the third and fourth VSync-sf signals, applications within the list generate layer 3, while non-list applications generate layer 2' in response to the second VSync-sf signal. The surface compositor synthesizes the second frame of the user interface, image 2, in response to the third VSync-sf signal. This user interface consists of layer 2 and layer 1'. The display driver can respond to the HW-VSync signal ( Figure 8 (Not shown in the image) After that, the user interface containing layer 1 + layer 1' synthesized by the surface synthesizer is sent to the display, and then the touch screen can display the user interface.

[0170] After the fourth VSync-sf signal, applications in the list generate layer 4, while non-list applications are still waiting for VSync-app3. The surface compositor can then composite the user interface to be displayed: image 3, which contains layer 3 and layer 2'. The display driver can respond to the HW-VSync signal to send the user interface (including layer 2 and layer 1') composited by the surface compositor, thereby displaying the user interface on the touchscreen.

[0171] The frame rate of applications in the list is not limited to being higher than that of applications not in the list; it can also be that the frame rate of applications in the list is lower than that of applications not in the list. This application does not impose any restrictions on this.

[0172] In some embodiments, Figure 8The listed applications can be game applications, while the non-listed applications can be other applications (such as instant messaging applications). Electronic devices can maintain a high frame rate when displaying the user interface of game applications, while other applications often do not require a high frame rate. Therefore, electronic devices can display the user interface of non-game applications at a lower frame rate than game applications. This ensures a good gaming experience for the user while avoiding excessive power consumption caused by non-game applications unnecessarily refreshing their user interface at the game application's frame rate in multi-window scenarios.

[0173] The display method provided in the embodiments of this application is described below.

[0174] Figure 9 This is a flowchart of a display method provided in an embodiment of this application. For example... Figure 9 As shown, this method may include, but is not limited to, the following steps:

[0175] S901, Receive a first operation applied to a first application, and generate a first event based on the first operation.

[0176] The first application can be a game application. The electronic device can receive a first operation applied to the user interface of the first application, which may include, but is not limited to, pressing, releasing, and swiping. Furthermore, the input management service in the electronic device can generate a first event based on the first operation.

[0177] S902, In response to generating the first event, provide the first event to the first application.

[0178] The input management service in an electronic device can directly send the first event to the first application in response to its generation. Specifically, the input management service does not wait for the VSync-app signal after generating the first event; instead, it provides it directly to the first application. The first application can be a game application on the electronic device.

[0179] In some embodiments, the touchscreen manager is used to determine whether the first application is an application that depends on the VSync-app signal. Here, an application that depends on the VSync-app signal means that the application needs to rely on the VSync-app synchronization mechanism to output rendered frames. Since game applications control the output of rendered frames by their own engine, game applications are applications that do not depend on the VSync-app signal.

[0180] by Figure 5 In the illustrated embodiment, the first event could be, for example, input event 1. After generating event 1, the input management service directly sends it to the game application. Figure 5The first VSync-app signal (also known as VSync-app1 signal) can be called the first VSync-app signal, and the second VSync-app signal (also known as VSync-app2 signal) can be called the second VSync-app signal. The first event is generated between the first VSync-app signal and the second VSync-app signal and provided to the first application. The first VSync-app signal and the second VSync-app signal are two adjacent vertical synchronization signals emitted by the electronic device.

[0181] S903, Generate the first layer based on the first event through the first application.

[0182] S904. Display the first user interface based on the first layer.

[0183] The first application in an electronic device can generate a first layer based on a first event, so as to... Figure 5 Taking the illustrated embodiment as an example, the first layer can be layer 1. The first application may include a logic thread and a rendering thread. After the first application receives the first event, the logic thread first generates first rendering data based on the first event, where the first rendering data can be rendering data 1. Then, the rendering thread can generate a first layer based on the first rendering data, where the first layer can be, for example, layer 1. Subsequently, the surface compositer in the electronic device can generate a first user interface based on the first layer, the display driver sends the first user interface to the display, and finally, the touchscreen displays the first user interface. The first user interface can be, for example, image 1.

[0184] Understandably, after generating the first event, the electronic device can directly provide it to the first application without waiting for the VSync-app signal. This reduces the latency between input event generation and application reception, allowing the application to process input events earlier and preventing situations where rendering time has arrived but the corresponding rendering data for the latest input event has not yet been generated. This method effectively improves the responsiveness of the electronic device. Furthermore, when the first application is a game, its rendering frame generation does not depend on the VSync-app signal, and the input management service does not need to wait for the VSync-app signal to arrive before sending input events to the first application. This ensures that the first application still generates rendering frames according to its cycle, improving the responsiveness of the electronic device when running the first application.

[0185] In some embodiments, the first user interface contains only a first layer, and the method further includes stopping the transmission of the VSync-app signal. The surface synthesizer in the electronic device can determine the layers contained in the first application interface before synthesizing it. When the surface synthesizer determines that it contains only the first layer of the first application, it indicates that there is no layer of the application that depends on the VSync-app signal in the first application interface. Thus, the electronic device can stop transmitting (or keep the VSync-app signal off) to reduce the power consumption of the electronic device.

[0186] In some embodiments, before generating the first event according to the first operation, the method further includes: determining that the sampling rate of the touch screen is a multiple of the refresh rate, and determining that the first application is in a first list. Only when the sampling rate of the touch screen is a multiple of the refresh rate does the electronic device generate the same number of input events between refreshing two frames of the user interface. This ensures that the application generates rendering frames based on the same number of input events each time, avoiding the problem of discontinuous display. The first list can be an enable list, where the first list is not limited to a list of data structures, but is used to represent a list storing the identifiers of one or more applications. It can be a list, a linked list, etc. The embodiments of this application do not limit the data structure type of the first list. The first application being in the first list means that the identifier of the first application is in the first list. The applications corresponding to the identifiers included in the first list can be set by the user or preset by the electronic device. The embodiments of this application do not limit this. Optionally, the identifier of the application can be the package name of the application.

[0187] In some embodiments, the electronic device further includes a second application, and the method further includes: receiving a second operation acting on the second application and generating a second event based on the second operation; providing the second event to the second application in response to a vertical synchronization (VSync-app) signal; generating a second layer through the second application based on the second event; and displaying a second user interface based on the second layer.

[0188] The second application can be an application not listed in the first list. The second application relies on the VSync-app signal to trigger the drawing and rendering process; therefore, the input management service still needs to wait for the VSync-app signal when sending input events to the second application. Subsequently, the second application can generate a second layer based on the second event, the surface compositor can composite a second user interface based on the second layer, the display driver can send the second user interface to the display, and the touchscreen can then display the second user interface. The touchscreen manager in the electronic device is used to determine whether the second application relies on the VSync-app signal.

[0189] In some embodiments, before providing the second event to the second application, the method further includes resending the VSync-app signal. Since the first application does not rely on the VSync-app signal, while the second application does, it is necessary to re-enable the VSync-app signal when the electronic device is running the second application to ensure that the second application can stably output image frames.

[0190] In some embodiments, the second user interface further includes a third layer of the first application, wherein the frame rate of the first application is a first frame rate, the frame rate of the second application is a second frame rate, and the first frame rate is greater than the second frame rate.

[0191] by Figure 7B Taking the scenario shown as an example, the second user interface can be a user interface displayed on a touchscreen (including user interface 701 and user interface 702). The third layer can be the layer corresponding to user interface 701, and the second layer can be the layer corresponding to user interface 702. The electronic device can display the user interface of the first application at a first frame rate and the user interface of the second application at a second frame rate. To ensure a good gaming experience, the first frame rate is usually higher, and the frame rate of the game application is not affected by the VSync-app signal. The electronic device can control the frame rate of the second application by adjusting the frequency of the VSync-app signal, making the second frame rate lower than the first frame rate. This avoids the second application unnecessarily outputting rendering frames at the first frame rate, leading to excessive power consumption of the electronic device. The adjustment of the VSync-app signal frequency can be performed by a surface synthesizer in the electronic device. When the surface synthesizer determines that the second user interface contains not only the third layer of the game application but also a second layer of a non-game application, it can adjust the transmission frequency of the VSync-app signal to the second frame rate.

[0192] In some embodiments, before generating the second event according to the second operation, the method further includes: determining that the sampling rate of the touchscreen is not a multiple of the refresh rate, and / or determining that the second application is not in the first list. When the sampling rate of the touchscreen is not a multiple of the refresh rate, when the input management service sends input events to the application without relying on the VSync-app signal, it may cause the application to generate rendering frames based on a different number of input events each time, resulting in a less smooth final user interface. Therefore, the electronic device can enable (or keep enabled) the VSync-app signal when it is determined that the touchscreen sampling rate is not a multiple of the refresh rate, and / or that the second application is not in the first list.

[0193] The apparatus for performing the above method provided in the embodiments of this application will now be described. Figure 10 As shown, Figure 10This is a schematic diagram of a display device provided in an embodiment of this application. The display device may be an electronic device as described in this application embodiment, or a chip or chip system within an electronic device.

[0194] like Figure 10 As shown, the display device 1000 may include an operation acquisition module 1001, a processing module 1002, a display module 1003, and a storage module 1004.

[0195] The storage module 1004 can store a first application, and the processing module 1002 is used to periodically send VSync-app signals. The operation acquisition module 1001 is used to receive a first operation applied to the first application. The processing module 1002 is used to generate a first event based on the first operation, and in response to generating the first event, provide the first event to the first application. The processing module 1002 is also used to generate a first layer based on the first event through the first application; the display module 1003 is used to display a first user interface based on the first layer.

[0196] In some embodiments, when the first user interface contains only the first layer, the processing module 1002 is also used to stop sending the VSync-app signal.

[0197] In some embodiments, before generating the first event according to the first operation, the processing module 1002 is further configured to determine that the sampling rate of the touch screen is a multiple of the refresh rate, and to determine the first application in the first list.

[0198] In some embodiments, the storage module 1004 further stores a second application. The operation acquisition module 1001 is further configured to receive a second operation acting on the second application. The processing module 1002 is further configured to generate a second event based on the second operation, and to provide the second event to the second application in response to a vertical synchronization (VSync-app) signal; the processing module 1002 is further configured to generate a second layer based on the second event through the second application. The display module 1003 is further configured to display a second user interface based on the second layer.

[0199] In some embodiments, before providing the second event to the second application, the processing module 1002 is also configured to resend the VSync-app signal.

[0200] In some embodiments, before generating the second event according to the second operation, the processing module 1002 is further configured to determine that the sampling rate of the touch screen is not a multiple of the refresh rate, and / or determine that the second application is not in the first list.

[0201] In some embodiments, the first application is a game application, and the second application is a non-game application.

[0202] In some embodiments, the processing module 1002 and the storage module 1004 are connected by a line. The storage module 1004 may include one or more memories, which may be devices in one or more devices or circuits used to store programs or data. The storage module 1004 may exist independently and be connected to the processing module 1002 via a communication bus. Alternatively, the storage module 1004 may be integrated with the processing module 1002.

[0203] The storage module 1004 can store computer-executable instructions for the methods in the electronic device, so that the processing module 1002 can execute the methods in the above embodiments. The storage module 1004 can be a register, cache, or random access memory (RAM), etc., and can be integrated with the processing module 1002. The storage module 1004 can also be a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, and can be independent of the processing module 1002.

[0204] Figure 11 This is a schematic diagram of a chip structure provided in an embodiment of this application. Figure 11 As shown, chip 1100 includes one or more (including two) processors 1101, communication lines 1102 and communication interfaces 1103. Optionally, chip 1100 also includes a memory 1104.

[0205] In some implementations, memory 1104 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof.

[0206] The methods described in the embodiments of this application can be applied to processor 1101, or implemented by processor 1101. Processor 1101 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit in the hardware of processor 1101 or by instructions in software form. The processor 1101 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. Processor 1101 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0207] The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read-only memory (EEPROM). This storage medium is located in memory 1104, and processor 1101 reads information from memory 1104 and, in conjunction with its hardware, completes the steps of the above method.

[0208] The processor 1101, memory 1104 and communication interface 1103 can communicate with each other via communication line 1102.

[0209] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.

[0210] This application also provides a computer program product comprising one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. For example, available media may include magnetic media (e.g., floppy disk, hard disk, or magnetic tape), optical media (e.g., digital versatile disc (DVD)), or semiconductor media (e.g., solid-state disk (SSD)).

[0211] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program or instructions. When the computer program or instructions are executed by a processor, they implement the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0212] As one possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM, or other optical disc storage; computer-readable media may include disk storage or other disk storage devices. Furthermore, any connecting cable may also be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. As used herein, disks and optical discs include optical discs (CD), laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0213] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0214] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0215] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0216] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A display method, the method being applied to an electronic device, the electronic device having a first application and a second application installed, the first application being a game application and the second application being a non-game application, characterized in that, include: Between the first vertical synchronization VSync signal and the second VSync signal, a first operation performed by the user on the user interface of the first application is received, and a first event is generated after the first operation, wherein the second VSync signal is located after the first VSync signal, and the first VSync signal is adjacent to the second VSync signal; Between the first VSync signal and the second VSync signal, the first event is provided to the first application, and the first application generates first rendering data based on the first event; Between the second VSync signal and the third VSync signal, the first application generates a first layer based on the first rendering data, wherein the third VSync signal is located after the second VSync signal and is adjacent to the second VSync signal; Between the first VSync signal and the second VSync signal, a second operation performed by the user on the user interface of the second application is received, and a second event is generated after the second operation; In response to the second VSync signal, the second event is provided to the second application; Between the second VSync signal and the third VSync signal, a second layer is generated by the second application based on the second event; Between the third VSync signal and the fourth VSync signal, the first layer and the second layer are composited by a surface synthesizer to obtain a first image. The third VSync signal and the fourth VSync signal are two adjacent vertical synchronization signals. The third VSync signal is located after the second VSync signal, and the fourth VSync signal is located after the third VSync signal and is adjacent to the third VSync signal. Following the fourth VSync signal, the first image is displayed via the display driver.

2. The method as described in claim 1, characterized in that, A first event is generated after the first operation, including: After the first operation, the first event is generated based on the first operation.

3. The method as described in claim 1 or 2, characterized in that, The first operation is the user's operation on the first control of the first application, and the first control is used to control the movement of the character texture of the first application.

4. The method as described in claim 3, characterized in that, The first operation is the user pressing the first control and sliding it.

5. The method according to any one of claims 1-4, characterized in that, The step of generating first rendering data based on the first event through the first application includes: The first rendering data is generated by the logical thread of the first application based on the first event.

6. The method according to any one of claims 1-5, characterized in that, The step of generating the first layer based on the first rendering data through the first application includes: The first layer is generated by the rendering thread of the first application based on the first rendering data.

7. The method according to any one of claims 1-6, characterized in that, The electronic device includes a touch screen, the sampling rate of the touch screen is a first sampling rate, the refresh rate of the image displayed on the touch screen is a first refresh rate, and the first sampling rate is N times the first refresh rate, where N is an integer greater than 0.

8. The method according to any one of claims 1-7, characterized in that, The electronic device includes an input management service, and providing the first event to the first application includes: The input management service provides the first event to the first application.

9. The method according to any one of claims 1-8, characterized in that, The first VSync signal, the second VSync signal, the third VSync signal, and the fourth VSync signal are all VSync-app signals.

10. The method according to any one of claims 1-9, characterized in that, The user interfaces of the first application and the second application are displayed in a split-screen manner.

11. The method according to any one of claims 1-9, characterized in that, The user interface of the first application is displayed globally, while the user interface of the second application is displayed in a window.

12. An electronic device, characterized in that, The electronic device includes: a touch screen, a memory, and a processor coupled to the memory; the memory stores a computer program, and when the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1-11.

13. A computer-readable storage medium comprising computer instructions, characterized in that, When the computer instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-11.

Images