Interface display method, electronic device, storage medium, and chip

Abstract

The present application relates to the technical field of image processing. Provided are an interface display method, an electronic device, a storage medium, and a chip. The method is applicable to a dynamic effect period during which the display of an interface of a first application is switched to the display of a system desktop or a multi-task interface, the first application is an application during the display of which a layer needs to be drawn by means of SurfaceView, and it is configurable that during the dynamic effect period, the rendering that a GPU performs on the layer which is drawn on the basis of SurfaceView is paused, so as to reduce the load of the GPU. The method can reduce the phenomenon of lagging or screen freeze caused by an excessive load of the GPU during the dynamic effect period.

Description

Interface display method, electronic device, storage medium and chip Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an interface display method, electronic equipment, storage medium and chip. Background Art

[0002] The operating system provides multiple view classes, such as the View class and the SurfaceView class. The View class draws its content on the main UI thread, while the SurfaceView class redraws its content in a separate thread. Therefore, the SurfaceView class avoids issues like blocking the main UI thread. In some high-refresh scenarios, SurfaceView can provide a good user experience. However, when displaying complex content, electronic devices often experience lag or screen freezing.

[0003] Summary of the Invention

[0004] The present application provides an interface display method, electronic device, storage medium and chip, which can reduce the phenomenon of lag or screen freezing when the electronic device displays some more complex content.

[0005] To achieve the above objectives, the first aspect of this application adopts the following technical solutions:

[0006] receiving a first operation from a user;

[0007] In response to the first operation, a first application is started, wherein the layers of the first application include a layer drawn by the view and a layer drawn by the SurfaceView;

[0008] After the first application is started, the application interface of the first application is displayed;

[0009] In the process of displaying the application interface of the first application, the method further includes:

[0010] Render the first layer of the first application, where the first layer is a layer drawn using a view;

[0011] Render the second layer of the first application, where the second layer is a layer drawn using SurfaceView;

[0012] Composite the first layer and the second layer to obtain the third layer;

[0013] Displaying a first interface of a first application, where the first interface is obtained based on the third layer;

[0014] receiving a second operation from the user;

[0015] In response to a second operation, switching from displaying the first application to displaying the second application;

[0016] In the process of switching from displaying the first application to displaying the second application, the method further includes:

[0017] Render the fourth layer of the first application, where the fourth layer is a layer drawn using a view;

[0018] Render the fifth layer of the second application, where the fifth layer is drawn using a view.

[0019] Composite the fourth layer and the fifth layer to obtain the sixth layer;

[0020] Displays the second interface during the switching process. The second interface is obtained based on the sixth layer.

[0021] In the present application, the application interface of the second application can be the system desktop or the multitasking interface. The first application is an application that includes a layer drawn by SurfaceView, such as a map application, a video application, or a game application. In the process of displaying the application interface of the first application, it is necessary to use view to draw the layer, and it is also necessary to use SurfaceView to draw the layer. Therefore, in the process of displaying the application interface of the first application, the GPU not only needs to render the layer drawn by view, but also needs to render the layer drawn by SurfaceView. Similarly, the synthesized layer also includes the layer drawn by view of the first application and the layer drawn by SurfaceView. The first interface belongs to the application interface of the first application. In the process of switching from displaying the application interface of the first application to the application interface of the second application, the layer drawn by SurfaceView can no longer be rendered by the GPU. Similarly, the synthesized layer does not include the layer drawn by SurfaceView of the first application. The second interface is not the interface of the second application after switching, but the interface displayed during the switching process. This can reduce the load on the GPU, thereby reducing the phenomenon of freeze or screen freezing caused by excessive GPU load.

[0022] As another implementation of the first aspect, before rendering the second layer of the first application, the method further includes:

[0023] Apply for the first buffer from the first buffer queue;

[0024] Use SurfaceView to draw the layer in the first buffer to obtain the second layer of the first application.

[0025] In this application, before GPU rendering uses SurfaceView to draw the layer, the rendering thread is also required to apply for a buffer from the first buffer queue corresponding to SurfaceView and draw the layer in the buffer.

[0026] As another implementation of the first aspect, the method is applied to an electronic device, the electronic device including: a launcher and a first application. After receiving a second operation from the user, the method further includes:

[0027] The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application;

[0028] The launcher sends the animation start information and the first duration to the first application. After the first application receives the animation start information and the first duration, the rendering thread of the first application waits for the task of requesting a buffer from the first buffer queue.

[0029] After the first application receives the first duration of the information indicating the start of the animation effect, the rendering thread resumes the execution state of the task of applying for the buffer from the first buffer queue.

[0030] In this application, as before, before the GPU renders the layer drawn using SurfaceView, the rendering thread needs to apply for a buffer from the first buffer queue corresponding to SurfaceView and draw the layer in the buffer; therefore, after the animation starts, the rendering thread can be set to wait for a certain period of time for the task of applying for a buffer from the first buffer queue. The first period is the estimated duration of the process of switching from displaying the first application to displaying the second application. During the period when the task of applying for a buffer from the first buffer queue by the rendering thread is in the waiting state, the rendering thread cannot successfully apply for a buffer, so the GL thread cannot execute the subsequent task of drawing the layer using SurfaceView. Accordingly, the GPU will not execute the task of rendering the layer drawn using SurfaceView, which can reduce the load on the GPU. After the first period of time, the animation is estimated to end, and the settings during the animation period can be restored to the original state, that is, the waiting state of the task of applying for a buffer from the first buffer queue by the rendering thread is ended, so that the task of applying for a buffer from the first buffer queue by the rendering thread is restored to the execution state.

[0031] As another implementation of the first aspect, the electronic device further includes: SurfaceFlinger; after the first application is started, the method further includes:

[0032] The first application registers the listener;

[0033] The first application sends the listener of the first application and the package name of the first application to SurfaceFlinger; SurfaceFlinger stores the package name of the first application and the listener of the first application;

[0034] The launcher sends the animation start information and the first duration to the first application, including:

[0035] The launcher sends the animation start information, the first duration, and the application package name of the first application to SurfaceFlinger;

[0036] After receiving the information about the start of the motion effect, the first duration, and the application package name of the first application, SurfaceFlinger determines the listener of the first application according to the application package name of the first application;

[0037] SurfaceFlinger sends the motion effect start information and the first duration to the first application through the listener of the first application.

[0038] In this application, SurfaceFlinger can be used as a bridge for communication between the system desktop (launcher) and the first application, wherein the first application and SurfaceFlinger can transmit messages through the listener of the first application, so that the launcher that detects the start or end of the animation effect transmits the message of the start or end of the animation effect to the first application, so that the first application actively sets the task of the rendering thread to apply for the buffer to the wait state.

[0039] As another implementation of the first aspect, the method is applied to an electronic device, the electronic device including: a launcher and a first application. After receiving a second operation from the user, the method further includes:

[0040] The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application;

[0041] The launcher sends a message about the start of the animation effect to the first application. After the first application receives the message about the start of the animation effect, the rendering thread of the first application is in a waiting state for the task of applying for a buffer from the first buffer queue.

[0042] As another implementation of the first aspect, after setting the task of the rendering thread applying for a buffer from the first buffer queue to a waiting state, the method further includes:

[0043] When the task of the rendering thread of the first application requesting a buffer from the first buffer queue is in a waiting state, the method further includes:

[0044] After detecting that the animation has ended, the launcher sends an animation end message to the first application;

[0045] After the first application receives the information that the animation effect ends, the rendering thread resumes the execution state of the task of applying for the buffer from the first buffer queue.

[0046] In this application, since the first duration is the estimated switching duration, the end of the first duration is not completely consistent with the actual end of the animation effect. This application adopts another implementation method. When the animation effect ends, the information about the end of the animation effect is used to trigger the rendering thread to restore the task of applying for the buffer from the first buffer queue to the execution state.

[0047] As another implementation of the first aspect, the method is applied to an electronic device, the electronic device including: a launcher and a SurfaceFlinger, wherein a data structure of a buffer in a first cache queue includes a block fence, and after receiving a second operation from a user, the method further includes:

[0048] The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application;

[0049] The launcher sends a message to SurfaceFlinger about the start of the animation.

[0050] After receiving the information about the start of the animation, SurfaceFlinger sets the block fence in the data structure of the buffer in the first cache queue to the first flag. When the block fence of the buffer in the first buffer queue is the first flag, the task of the rendering thread of the first application to apply for the buffer from the first buffer queue is in a waiting state.

[0051] In this application, other methods can also be used to control the GPU to stop rendering the layer drawn with SurfaceView during the animation. For example, the data structure of the buffer in the first buffer queue corresponding to SurfaceView can be expanded to add a block fence. The block fence can be set under a specific flag (the flag corresponding to the animation period), and the task of the rendering thread to apply for the buffer from the first buffer queue is in a waiting state. Specifically, the block fence in the buffer in the first buffer queue can be obtained by the rendering thread to determine the specific status of the task applying for the buffer.

[0052] As another implementation of the first aspect, when the task of the rendering thread of the first application requesting a buffer from the first buffer queue is in a waiting state, the method further includes:

[0053] After detecting that the animation has ended, the launcher sends an animation end message to SurfaceFlinger.

[0054] After receiving the information that the animation ends, SurfaceFlinger sets the block fence in the data structure of the buffer in the first cache queue to the second flag. When the block fence of the buffer in the first buffer queue is the second flag, the rendering thread resumes the execution state of the task of applying for the buffer from the first buffer queue.

[0055] In this application, corresponding to the setting of the block fence as a specific flag during the animation, after the animation ends, the flag corresponding to the block fence needs to be restored from the specific flag to the original flag.

[0056] As another implementation of the first aspect, the method applies an electronic device, the electronic device including a launcher, a SurfaceFlinger, and a first application, and the method further includes:

[0057] The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application;

[0058] The launcher sends a message to SurfaceFlinger about the start of the animation.

[0059] After receiving the information that the animation starts, SurfaceFlinger sets the number of buffers in the first buffer queue to 1 and the consumption flag of the first buffer queue to the third flag. When the second buffer in the first buffer queue is not idle, the rendering thread of the first application applies for the second buffer from the first buffer queue in a waiting state. The second buffer is the buffer in the first buffer queue when the number of buffers in the first buffer queue is 1.

[0060] This application also provides another way to control the GPU to no longer render the layer drawn by SurfaceView during the animation. For example, during the animation, the number of buffers in the first buffer queue can be set to 1 through SurfaceFlinger, and SurfaceFlinger itself can be set to no longer consume the buffers in the first buffer queue. Therefore, when the number of buffers in the first buffer queue is 1 and is not idle, the rendering thread will not be able to apply for an idle buffer, and naturally will not execute the subsequent steps of drawing the layer using SurfaceView, and the GPU does not need to render the layer drawn by SurfaceView, thereby reducing the GPU load.

[0061] As another implementation of the first aspect, the electronic device further includes: a GPU, after which SurfaceFlinger sets the number of buffers in the first buffer queue to 1 and sets the consumption flag of the first buffer queue to a third flag, the method further includes:

[0062] If the second buffer in the first buffer queue is idle, the rendering thread of the first application applies for the second buffer from the first buffer queue;

[0063] The rendering thread of the first application uses SurfaceView to draw the layer in the second buffer to obtain the seventh layer of the first application;

[0064] The rendering thread of the first application returns the second buffer storing the seventh layer to the first buffer queue and sets the second buffer to the queued state;

[0065] The rendering thread of the first application sends rendering instructions to the GPU;

[0066] After receiving the rendering instruction, the GPU renders the seventh layer in the first buffer queue;

[0067] GPU sends synthesis instructions to SurfaceFlinger after rendering;

[0068] SurfaceFlinger does not respond to the synthesis instruction when the consumption flag of the first buffer queue is the third flag.

[0069] In this application, when the number of buffers in the first buffer queue is 1 and it is in an idle state, the rendering thread will apply for the idle buffer and execute the subsequent steps of drawing the layer using SurfaceView, and the GPU will also render the layer drawn using SurfaceView once; however, since SurfaceFlinger no longer consumes the layer drawn using SurfaceView, the buffer will remain in the queue state, and the subsequent rendering thread will not be able to apply for the idle buffer, and naturally will not execute the subsequent steps of drawing the layer using SurfaceView, and the GPU does not need to render the layer drawn using SurfaceView, thereby reducing the GPU load.

[0070] As another implementation of the first aspect, the method further includes:

[0071] After detecting that the animation has ended, the launcher sends an animation end message to SurfaceFlinger.

[0072] After receiving the information that the animation effect ends, SurfaceFlinger sets the number of buffers in the first buffer queue to the initial number and sets the consumption flag of the first buffer queue to the fourth flag. SurfaceFlinger responds to the synthesis instruction when the consumption flag of the first buffer queue is the fourth flag. The initial number is the number before the number of buffers in the first buffer queue is switched to 1.

[0073] In this application, after the animation ends, the number of buffers in the first buffer queue needs to be restored to the initial number, and SurfaceFlinger needs to be set to continue responding to the synthesis instruction and consume the buffers in the first buffer queue, so that the buffers in the first buffer queue form a rotation.

[0074] In a second aspect, an electronic device is provided, comprising a processor, wherein the processor is configured to call a computer program stored in a memory to implement any one of the methods of the first aspect of the present application.

[0075] In a third aspect, a chip is provided, comprising a processor, wherein the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement any method of the first aspect of the present application.

[0076] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer instructions are executed on an electronic device, the electronic device implements any one of the methods of the first aspect of the present application.

[0077] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a device, enables the device to execute any method of the first aspect of the present application.

[0078] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] FIG1 is a schematic diagram of a hardware structure of an electronic device provided in an embodiment of the present application;

[0080] FIG2 is a schematic diagram of an animation effect of a map application switching to a system desktop provided by an embodiment of the present application;

[0081] FIG3 is a schematic diagram of an animation effect of a game application switching to a multi-tasking interface according to an embodiment of the present application;

[0082] FIG4 is a schematic diagram of an interface drawing, rendering and synthesis process provided by an embodiment of the present application;

[0083] FIG5 is a schematic diagram of sharing and rotation of a buffer queue provided in an embodiment of the present application;

[0084] FIG6 is a schematic diagram of a GPU overload according to an embodiment of the present application;

[0085] FIG7 is a schematic diagram of a method for reducing GPU load provided by an embodiment of the present application;

[0086] FIG8 is an interactive diagram of the method for reducing GPU load shown in FIG7 according to an embodiment of the present application;

[0087] FIG9 is a schematic diagram of information of meta data of a graphics buffer in a buffer queue provided by an embodiment of the present application;

[0088] FIG10 is an interactive diagram of a method for reducing GPU load based on the buffer queue shown in FIG9 according to an embodiment of the present application;

[0089] FIG11 is an interactive diagram of another method for reducing GPU load provided by an embodiment of the present application;

[0090] FIG12 is an interactive diagram of another method for reducing GPU load provided by an embodiment of the present application;

[0091] FIG13 is a schematic diagram showing the difference between before and after the load reduction mode provided in an embodiment of the present application;

[0092] FIG14 is another schematic diagram showing the difference between the load reduction mode before and after the load reduction mode is adopted according to an embodiment of the present application;

[0093] FIG15 is a comparison diagram of the NO-fence latch being enabled and the NO-fence latch being disabled according to an embodiment of the present application. DETAILED DESCRIPTION

[0094] In the following description, specific details such as specific system structures and technologies are provided for illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details.

[0095] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0096] It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more than two. "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0097] In addition, in the description of this application specification and the appended claims, the terms "first", "second", "third", "fourth", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0098] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0099] The interface display method provided in the embodiments of the present application may be an electronic device such as a tablet computer, a mobile phone, a wearable device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or the like. The embodiments of the present application do not limit the specific type of electronic device.

[0100] Figure 1 shows a schematic diagram of the structure of an electronic device. Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. Sensor module 180 may include a pressure sensor 180A, a touch sensor 180K, an ambient light sensor 180L, and the like.

[0101] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0102] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a coprocessor (SCP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. For example, the processor 110 is used to execute the interface display method in the embodiment of the present application.

[0103] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0104] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0105] The USB interface 130 is an interface that complies with USB standards, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transfer data between the electronic device 100 and peripheral devices.

[0106] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0107] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0108] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.).

[0109] In addition, the internal memory 121 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0110] The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger via the USB interface 130.

[0111] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0112] In some other embodiments, the power management module 141 may also be provided in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

[0113] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0114] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0115] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1.

[0116] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0117] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0118] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0119] The audio module 170 is used to convert digital audio signals into analog audio signals for output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0120] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0121] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0122] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only monitor voice information but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, and directional recording functions. The headphone jack 170D is used to connect wired headphones. The headphone jack 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0123] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device comprising at least two parallel plates having a conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0124] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0125] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0126] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0127] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts and can also be used for touch vibration feedback.

[0128] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0129] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0130] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0131] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or removed from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0132] The embodiments of the present application do not particularly limit the specific structure of the execution subject of an interface display method; as long as the code recording an interface display method according to the embodiments of the present application can be executed to communicate according to an interface display method provided by the embodiments of the present application. For example, the execution subject of an interface display method provided by the embodiments of the present application can be a functional module in an electronic device that can call and execute a program, or a communication device used in an electronic device, such as a chip.

[0133] In the Android system, a variety of view classes are provided to implement the layout and display of the user interface, such as view and SurfaceView. SurfaceView is a subclass of the view class, which can obtain image data directly from memory or hardware interfaces such as DMA. It is a very important drawing view. View-based window refresh requires redrawing the application window of the application. The view is drawn in the user interface (UI) thread. For some more complex scene views, it may block the main thread. Therefore, view is suitable for conventional interface layout and interaction. SurfaceView's window refresh does not require redrawing the application window of the application. It uses an independent drawing and rendering thread and is suitable for scenes that require complex drawing, frequent updates or real-time rendering, such as game scenes, video scenes, map scenes, etc.

[0134] For the game scenes, video scenes, and map scenes mentioned above, SurfaceView is not used exclusively for drawing. Instead, SurfaceView is used for drawing while view is used. The application displays content to the user as the user interface. Activity is a component of the application. Each activity has an associated application window for displaying the user interface of the application. The application window can contain multiple views to construct the specific content of the user interface, such as buttons and text boxes in the user interface. SurfaceView will create a new window behind the application window, which is equivalent to digging a hole in the application window. SurfaceView is displayed in the hole, and view is displayed on the application window. Therefore, view can be displayed on top of SurfaceView.

[0135] Referring to (a) in Figure 2, taking a map application as an example, the electronic device displays the user interface of the map application. Some content in the user interface displayed on the application window (for example, the search box, surrounding controls, travel controls, etc.) is drawn using view, while the map image and pointer located behind the application window are drawn using SurfaceView.

[0136] Referring to (a) in Figure 3, taking a game application as an example, the electronic device displays the user interface of the game application. Some content in the user interface displayed on the application window (for example, character avatars, character names, troop controls, landmark controls, etc.) is drawn using view, while the game map image located behind the application window is drawn using SurfaceView.

[0137] In the applications shown in Figure 2 or Figure 3, images drawn with SurfaceView consume more GPU resources when rendered through the GPU. Due to the limited capabilities of the chip, in some scenarios, the screen may become stuck or frozen.

[0138] Continuing with the scenario in Figure 2 as an example, when the electronic device displays the interface shown in Figure 2 (a), the user slides their finger upward from the bottom of the display screen (this operation can be used as an example of a second operation that triggers the start of an animation effect) to trigger the electronic device to display the system desktop. After detecting the gesture operation that triggers the display of the system desktop, the electronic device begins executing the animation effect of returning to the system desktop. This animation effect can refer to the process shown in Figures 2 (a) to 2 (d).

[0139] As shown in (b) of FIG2 , after the animation begins, the map interface gradually shrinks; as the map interface shrinks, the area of ​​the system desktop below the map interface that is not covered by the map interface gradually appears.

[0140] As shown in (c) of FIG. 2 , as the map interface becomes smaller and smaller, the map interface is reduced to a square card, and the area displayed on the system desktop below the map interface becomes larger and larger.

[0141] As shown in (d) in Figure 2, after the map interface disappears, the system desktop is fully displayed and the animation ends.

[0142] During this animation process, the electronic device displays content including the map interface, the system desktop, and the system status bar. The electronic device is continuously executing the following processes: the map application uses view to draw the image and then renders the image, the system desktop uses view to draw the image and then renders the image, and the system status bar uses view to draw the image and then renders the image; at the same time, the electronic device is also continuously executing the following processes: the map application uses SurfaceView to draw the image and then renders the image. In electronic devices, whether it is an image drawn by view or an image drawn by Surface, GPU rendering is required, and even GPU synthesis is required. Therefore, excessive GPU resources are consumed during this animation process, and freezes or freezing of the screen are more common.

[0143] Continuing with the scenario in Figure 3 as an example, when the electronic device displays the interface shown in Figure 3 (a), the user slides their finger upward from the bottom of the display screen and pauses (this operation can be used as an example of a second operation that triggers the start of an animation effect) to trigger the electronic device to display the multitasking interface. After detecting the gesture operation that triggers the display of the multitasking interface, the electronic device begins to execute the animation effect of returning to the multitasking interface. This animation effect can refer to the process shown in Figures 3 (a) to 3 (c).

[0144] As shown in (b) of Figure 3, after the animation begins, the game interface gradually shrinks; as the game interface shrinks, the area of ​​the multitasking interface below the game interface that is not covered by the game interface is gradually displayed.

[0145] As shown in (c) in Figure 3, as the game interface becomes smaller and smaller, other task cards in the multitasking interface will be displayed. After the game interface and other task cards in the multitasking interface are displayed as a fixed size, the animation ends.

[0146] During this animation process, the electronic device displays content including the game interface and the multi-tasking interface. The electronic device continuously executes the following processes: the game application uses view to draw the image and then renders the image, the multi-tasking interface uses view to draw the image and then renders the image; at the same time, the electronic device also continuously executes the following processes: the game application uses SurfaceView to draw the image and then renders the image. In electronic devices, whether it is an image drawn by view or an image drawn by SurfaceView, the GPU is required for rendering. Therefore, excessive GPU resources are consumed during this animation process, and freezes or freezing of the screen are more common.

[0147] The embodiment shown in Figure 3 takes the process of switching the interface of the horizontal screen display application to the multi-tasking interface as an example. In actual applications, the process of switching the interface of the vertical screen display application to the multi-tasking interface is also applicable to the interface display method provided in the embodiment of the present application.

[0148] In the above embodiment, the map application and the game application before the animation effect starts can be used as examples of the first application before the animation effect starts; the system desktop or multi-tasking interface can be used as an example of the application interface of the second application displayed after the animation effect ends.

[0149] The user can trigger the electronic device to start the first application through a first operation to display the interface of the first application. The first operation can be a click operation on the application icon of the first application on the system desktop.

[0150] In order to make the refreshing process of the content displayed on the electronic device clearer, the processing flow of the content displayed on the electronic device in the scenario shown in FIG. 2 is described below with reference to FIG. 4 .

[0151] The drawing and rendering processes of the images drawn by the map application, the system desktop and the system status bar using views are similar. The embodiment of the present application takes the map application and the system desktop using views for drawing and subsequent rendering as an example.

[0152] The system desktop has a UI thread. The process of drawing images using views on the system desktop occurs within this thread. The system desktop can initiate a callback drawing task to the system desktop's UI thread via the scheduleTraversals function of viewRoot. Based on this drawing task, the system desktop's UI thread performs measurement, layout, and drawing operations to obtain image data 1. The system desktop's UI thread then calls the skia OpenGL interface via the HWUI to pass image data 1 to the GPU for rendering.

[0153] The map application has a UI thread. The process of drawing an image using a view is performed within the UI thread. The map application initiates a callback drawing task to the UI thread via the scheduleTraversals function of viewRoot. Based on the drawing task, the UI thread performs measurement, layout, and drawing operations to obtain image data 2. The UI thread then calls the skia OpenGL interface via the HWUI to pass image data 2 to the GPU for rendering.

[0154] If the map application needs to use SurfaceView for drawing and rendering, the drawing and rendering of the SurfaceView is performed in the GL thread (as an example of a rendering thread). The map application uses SurfaceView to draw image data 3, and passes the image data 3 to the GPU for rendering through the GL thread.

[0155] The GPU executes each rendering task it receives. Additionally, image data drawn by each thread can be stored in a buffer queue called a GraphicBuffer. This GraphicBuffer is used to transfer image data between the upper-layer application and SurfaceFlinger. Image data drawn by different applications, or by different applications using different methods, is stored in different buffer queues.

[0156] Taking the scenario shown in Figure 2 as an example, image data 1 drawn by the map application using view is stored in buffer queue 1, image data 2 drawn by the map application using SurfaceView is stored in buffer queue 2, image data drawn by the system desktop using view is stored in buffer queue 3, and image data 4 drawn by the system status bar using view is stored in buffer queue 4.

[0157] SurfaceFlinger is responsible for synthesizing the image data in each buffer queue. Each image data has its own display information, such as position, transparency, size, stacking order (Z-order), etc. SurfaceFlinger is responsible for synthesizing each image based on the above display information of each image data.

[0158] At the SurfaceFlinger level, the image data synthesized by SurfaceFlinger is recorded as a layer. When SurfaceFlinger performs synthesis, it first sends a list of layers in each buffer queue to the Hardware Composer (HWC) and asks the HWC how to handle these layers. The HWC marks the layer as an overlay or GELS composition and then feeds it back to SurfaceFlinger.

[0159] SurfaceFlinger does not handle the synthesis of layers marked as overlay. Refer to the processing flow of image data drawn by view on the system desktop in Figure 4.

[0160] SurfaceFlinger synthesizes the layer marked as GELS composition through the render engine (SkiaGLRenderEngine), referring to the synthesis of image data drawn by the view and the image data drawn by the SurfaceView in the map application in Figure 4.

[0161] SurfaceFlinger sends the layer marked as overlay and the synthesized layer marked as GELS composition to HWC processing by calling setClientTarget.

[0162] The HWC obtains the synthesized image based on the display parameters of each layer received through the underlying hardware (for example, the hardware synthesizer), and sends the synthesized image to the screen driver, which presents the synthesized image to the physical screen for refresh display.

[0163] The above-mentioned process of refreshing the content displayed on the electronic device is only an example, and there may be some differences in the specific implementation process.

[0164] From this example, we can understand that when an electronic device displays the interface of a map application, the GPU will render the first layer of the map application drawn by view, and will also render the second layer of the map application drawn by SurfaceView. The third layer synthesized by SurfaceFlinger includes the first and second layers, and finally the interface displayed on the display is obtained based on the third layer.

[0165] As shown in Figure 4, the GraphicBuffer in the buffer queue (BufferQueue) can store image data. The buffer queue can implement memory sharing and buffer rotation.

[0166] On the application side, each application window corresponds to a surface, and on the SurfaceFlinger side, this surface corresponds to a layer. When an application needs to draw on a surface, it first needs to obtain the address of the surface in memory. This memory is allocated by SurfaceFlinger. Because SurfaceFlinger and the application run in different processes, this buffer queue can be used to achieve memory sharing and buffer rotation between the application client (surface) and the SurfaceFlinger server (layer).

[0167] Referring to Figure 5, a buffer queue is used as an example to describe the rotation of buffers. A GraphicBuffer in a buffer queue can exist in multiple states: free, dequeued, queued, and acquired. The application side (map applications, system desktop, status bar, etc. are all applications) as a producer can apply for a free GraphicBuffer from the cache queue through the dequeue operation (dequeuebuffer) and update the state of the GraphicBuffer from the free state to the dequeued state.

[0168] After applying for GraphicBuffer, the application side uses view to measure, layout and draw, and fills the drawn image data into GraphicBuffer. This process can be recorded as production.

[0169] After the production is completed, the application side returns the GraphicBuffer storing the image data to the buffer queue through the queue buffer operation, and updates the state of the GraphicBuffer storing the image data from the dequeued state to the queued state.

[0170] The application side notifies the GPU to render the image in the GraphicBuffer sent back to the buffer queue. After the GPU completes the rendering, it can notify the consumer SurfaceFlinger to consume the layer in the GraphicBuffer in the queue state.

[0171] As a consumer, SurfaceFlinger can obtain a GraphicBuffer in the queue state from the buffer queue through a consumption operation (acquiredbuffer), and update the state of the GraphicBuffer from the queue state to the consumption state;

[0172] SurfaceFlinger starts consuming the layer in the GraphicBuffer. For example, in the embodiment shown in FIG. 4 , SurfaceFlinger performs a synthesis operation, releases the GraphicBuffer through a release operation (releasebuffer) after consumption, and updates the state of the GraphicBuffer from the consumption state to the idle state.

[0173] During the above buffer rotation process, the fence mechanism can be used to ensure data synchronization.

[0174] As a producer, the application side cannot directly apply for an idle GraphicBuffer, but needs to wait for a signal (NO-fence signal). After SurfaceFlinger consumes a batch of synchronized layers, it can send the NO-fence signal to each application (map application, system desktop and status bar). Each application can apply for an idle GraphicBuffer only after receiving the NO-fence signal. After the application side applies for an idle GraphicBuffer, it fills the GraphicBuffer with image data as a producer and sends the GraphicBuffer filled with image data back to the buffer queue. At the same time, it notifies the GPU to render. After the GPU rendering is completed, each application will send the NO-fence signal back to SurfaceFlinger.

[0175] SurfaceFlinger also cannot directly consume the GraphicBuffer filled with image data in the buffer queue. It also needs to wait for each NO-fence signal. After SurfaceFlinger receives each NO-fence signal, it can obtain the GraphicBuffer in the queue state from the buffer queue for consumption. In this way, data synchronization can be ensured.

[0176] After releasing the GraphicBuffer, SurfaceFlinger updates the state of the GraphicBuffer from the consumption state to the idle state, and at the same time sends a NO-fence signal, which is transmitted to each application side before the application side can continue to execute the next round of rotation.

[0177] Normally, the application side will send the GraphicBuffer filled with image data back to the buffer queue while the GPU is rendering. However, if the GPU has not completed rendering the previous image task, the application side will enter the blocking queue and wait for the GPU to complete rendering. Then, it will send the GraphicBuffer filled with image data back to the buffer queue and notify the GPU to render.

[0178] For complex scenes (for example, the scenes shown in Figure 2 or Figure 3), when the GPU performs high-intensity rendering of image data drawn by SurfaceView, it usually takes up more GPU resources. For example, it takes a long time for the GPU to process the rendering of image data drawn by SurfaceView, which will cause other rendering tasks (rendering of images drawn by view in map applications, rendering of images drawn by view in system desktops, and rendering of images drawn by view in status bars) to be in a waiting state. The GPU cannot render at the predetermined frame rate, and subsequent synthesis and display will not be able to be performed at the frame rate. Therefore, the scenes shown in Figure 2 or Figure 3 will experience lag or frozen screen.

[0179] As shown in Figure 6, after the GL thread of the map application is finished drawing, the GPU renders the image drawn by SurfaceView in the map application. Because this rendering task takes a long time, after the system desktop finishes drawing the image using view, the GPU cannot immediately render the image drawn by view on the system desktop. Instead, it needs to wait until the previous rendering task of the GPU (rendering corresponding to SurfaceView) is completed before it can render the image drawn by view on the system desktop. If this waiting process is long, for example, it has exceeded the Vsync signal period or even longer, then the scene shown in Figure 2 or Figure 3 will be stuck or frozen.

[0180] At present, in the scenario shown in FIG2 or FIG3, the drawing and rendering of the view used by various applications (map application, system desktop and status bar, etc.) is closely related to whether the user-perceived dynamic effect is smooth, while the drawing and rendering used by SurfaceView is not closely related to whether the user-perceived dynamic effect is smooth. This is because the above-mentioned dynamic effect process is the process of switching from a map application to a system desktop. The user does not care about the changes in the map image itself within the interface of the map application, but is more concerned about the changes in the content displayed by the electronic device, such as the size and position of the interface of the map application, the content displayed on the system desktop, etc.). Therefore, in the scenario shown in FIG2 or FIG3, the embodiment of the present application can reduce the GPU load by pausing the rendering of the image corresponding to SurfaceView, thereby reducing the probability of GPU blocking, thereby reducing the phenomenon of freeze or screen freezing during the above-mentioned dynamic effect process.

[0181] In a specific implementation, pausing the rendering of the image corresponding to SurfaceView to reduce the GPU load may include:

[0182] (1) Set the number of graphic buffers in the buffer queue corresponding to SurfaceView to 1, and set SurfaceFlinger to no longer consume the graphic buffers in the buffer queue corresponding to SurfaceView. In this way, after a graphic buffer in the buffer queue corresponding to SurfaceView is requested and returned to the queue, since SurfaceFlinger no longer consumes it, there will be no idle graphic buffers in the buffer queue corresponding to SurfaceView. Subsequent applications applying for graphic buffers will fail. Accordingly, applications cannot apply for idle graphic buffers, and naturally there will be no subsequent drawing. The GPU does not need to perform subsequent rendering on SurfaceView, and the GPU load will be greatly reduced. The buffer queue corresponding to SurfaceView can be recorded as the first buffer queue.

[0183] (2) Through extended design, specific information (block fence) is added to the graphics buffer of the buffer queue corresponding to SurfaceView. When the rendering of the image based on SurfaceView is no longer executed during the animation period, the specific information is set as a mark. When the rendering of the image based on SurfaceView is executed during the non-animation period, the specific information is set as another mark. It is also necessary to set: the application will only apply for an idle graphics buffer when the corresponding mark (marker during the non-animation period) is obtained, otherwise it will enter a waiting state until the corresponding mark (marker during the non-animation period) is obtained. Since the application side is always in a waiting state during the animation period, it is unable to apply for an idle graphics buffer in the buffer queue corresponding to SurfaceView. Naturally, subsequent drawing and GPU rendering of SurfaceView will no longer be executed, and the GPU load will be greatly reduced.

[0184] (3) The system desktop can monitor the start and end of the animation effect, so the system desktop sends the information of the start or end of the animation effect to SurfaceFlinger. After receiving the information of the start of the animation effect, SurfaceFlinger passes the information of the start of the animation effect to the application using SurfaceView, so that after the animation effect starts, the application's operation of applying for the graphics buffer in the buffer queue corresponding to SurfaceView is invalid, or the application task can continue, but the application task enters the waiting state. After the information of the end of the animation effect is passed to the application side, the application side continues to restore the operation of applying for the graphics buffer in the buffer queue corresponding to SurfaceView as a valid operation, or ends the waiting state. Since the application side cannot successfully apply (invalid operation or waiting state) to the idle graphics buffer in the buffer queue corresponding to SurfaceView during the animation, the subsequent rendering of SurfaceView will no longer be executed, and the GPU load will be greatly reduced.

[0185] Of course, the three methods listed above for reducing GPU load all use the example of transmitting the start or end of an animation. In practice, it's also possible to transmit the start and duration of an animation. This way, some settings can be restored to their values ​​during the non-animation period without transmitting the end of the animation.

[0186] Through the above three methods, it can be understood that when switching from a map application to a system desktop, when the electronic device displays the interface during the switching process, the GPU will render the fourth layer drawn by the view of the map application, and will also render the fifth layer drawn by the view of the system desktop. The sixth layer synthesized by SurfaceFlinger includes the fourth and fifth layers, and finally the interface displayed on the display is obtained based on the sixth layer.

[0187] The following describes each of the above methods for reducing GPU load.

[0188] Referring to FIG7 , a method for reducing GPU load is provided in an embodiment of the present application. This method for reducing GPU load is based on the scenario shown in FIG2 . This method corresponds to the above-mentioned method (1) for reducing load.

[0189] When an electronic device displays the interface of a map application, refreshing the interface of the map application includes at least two processes:

[0190] Process 1: Before executing the measurement, layout, and drawing operations of the view used by the map application, the UI thread of the map application first applies for an idle graphics buffer from buffer queue 1. The UI thread produces the applied graphics buffer to obtain a graphics buffer filled with image data. The UI thread sends the graphics buffer filled with image data to buffer queue 1 and notifies the GPU to render the graphics buffer filled with image data. After the GPU rendering is completed, it notifies SurfaceFlinger to consume the graphics buffer; after SurfaceFlinger consumes the graphics buffer in buffer queue 1 for layer synthesis, it releases the consumed graphics buffer.

[0191] Of course, in actual applications, there may be other applications (eg, system desktop, status bar, etc.) that execute the above process.

[0192] Process 2: The GL thread of the map application first requests an idle graphics buffer from buffer queue 2, and the GL thread produces it. After the production is completed, it obtains a graphics buffer filled with image data; the GL thread sends the graphics buffer filled with image data to buffer queue 2, and at the same time notifies the GPU to render the graphics buffer filled with image data. After the GPU rendering is completed, it notifies SurfaceFlinger to consume the graphics buffer; after SurfaceFlinger consumes the graphics buffer in buffer queue 2 for layer synthesis, it releases the consumed graphics buffer.

[0193] When the electronic device displays the interface of the map application, the user performs a gesture operation on the display screen of the electronic device to trigger the electronic device to return to the system desktop.

[0194] The system desktop (launcher) in the electronic device can detect the gesture operation, determine the start of the animation based on the gesture operation, and send the information of the start of the animation to SurfaceFlinger.

[0195] After SurfaceFlinger receives the information that the animation starts, it sets the number of buffers in buffer queue 2 in the above process 2 to 1 and sets the consumption flag to 0, which indicates that SurfaceFlinger does not have the ability to consume the graphic buffer in buffer queue 2.

[0196] The above settings are not related to process one, so process one will continue to execute in a loop; the above settings are related to process two and will change process two.

[0197] Process 2 after the above settings take effect:

[0198] The GL thread first applies for an idle graphics buffer from buffer queue 2. Since the number of graphics buffers in buffer queue 2 is 1, if the status of the graphics buffer is idle, the application is successful.

[0199] After the GL thread successfully applies for the graphics buffer, it starts production and sends the produced graphics buffer filled with image data to buffer queue 2. At the same time, it notifies the GPU to render the graphics buffer filled with image data. After the GPU rendering is completed, it notifies SurfaceFlinger to consume the graphics buffer. Since the consumption flag has been set to 0, SurfaceFlinger will no longer consume the graphics buffer in buffer queue 2 even if it receives a notification message (for example, a NO-fence signal), and the graphics buffer in buffer queue 2 will remain in the queue state.

[0200] When the map application uses the SurfaceView to draw through the GL thread, the GL thread requests an idle graphics buffer from buffer queue 2, but the request fails.

[0201] If the GL thread fails to successfully request an idle graphics buffer, subsequent drawing using SurfaceView will no longer be executed, and accordingly, the GPU will not need to perform image rendering using SurfaceView, so the GPU load will naturally decrease.

[0202] Of course, the system desktop (launcher) can also detect the end of the animation. After detecting the end of the animation, the system desktop (launcher) sends the information of the end of the animation to SurfaceFlinger. SurfaceFlinger also needs to restore the number of graphic buffers in buffer queue 2 to its initial state (for example, if the original number is n, then set it to n), and restore the consumption flag to its initial state (for example, if it was originally 1, then set it to 1). When the consumption flag is 1, SurfaceFlinger has the ability to consume the graphic buffer in buffer queue 2.

[0203] In order to make the above process of reducing the load more clear, the above process of reducing the GPU load is described through the timing diagram shown in FIG8 .

[0204] A11: After the system desktop (launcher) detects a gesture operation, it determines the start of the animation based on the gesture operation and sends the animation start information to SurfaceFlinger.

[0205] A12, after SurfaceFlinger receives the information that the animation starts, it changes the number of graphic buffers in buffer queue 2 to 1.

[0206] Among them, buffer queue 2 is the buffer queue where the image data drawn by the map application using SurfaceView is located.

[0207] A13, after receiving the information that the animation starts, SurfaceFlinger sets the consumption flag to 0 (as an example of the third flag). The 0 indicates that SurfaceFlinger does not have the ability to consume the graphics buffer in buffer queue 2. Even if the conditions for SurfaceFlinger to consume the graphics buffer in buffer queue 2 are met, it will no longer consume the graphics buffer in buffer queue 2. That is, when SurfaceFlinger synthesizes the layers in each buffer queue, it no longer synthesizes the layers in buffer queue 2, but synthesizes the layers in other buffer queues (the buffer queue where the image drawn by the view in the map application is located, the buffer queue where the image drawn by the view in the system desktop is located, and the buffer queue where the image drawn by the view in the status bar is located).

[0208] Among them, step A12 and step A13 are not in a strict order.

[0209] After the animation starts, the map application, system desktop and status bar all perform corresponding drawing tasks based on the Vsync signal. The embodiment of the present application only takes the map application performing the view-based drawing task based on the Vsync signal as an example to describe the difference between the process of drawing and rendering the image using the view after the animation starts and the process of drawing and rendering the image using the SurfaceView.

[0210] B11, after receiving the Vsync1 signal, the UI thread of the map application requests the idle graphicbuffer1-1 from buffer queue 1.

[0211] The UI thread of the map application can perform subsequent drawing tasks based on the Vsync signal. There is no strict order between the time when the UI thread performs each drawing task based on the Vsync signal and the time when the animation starts. Accordingly, the steps after step A11 and the steps after step B11 are not strictly sequential.

[0212] B12, the UI thread of the map application performs production work such as measurement, layout, and drawing based on the applied graphicbuffer1-1, and obtains graphicbuffer1-1 filled with image data.

[0213] B13, the UI thread of the map application sends graphicbuffer1-1 filled with image data back to buffer queue 1.

[0214] B14, after sending graphicbuffer1-1 back to buffer queue 1, the UI thread of the map application notifies the GPU to perform rendering.

[0215] B15, GPU receives the rendering notification information and starts rendering the image in graphicbuffer1-1.

[0216] B16, after GPU rendering is completed, notify SurfaceFlinger to consume graphicbuffer1-1.

[0217] SurfaceFlinger receives the information of notification consumption and waits for the Vsync signal.

[0218] While the map application (system desktop, status bar, etc.) is using the view's image drawing and rendering process, the system desktop is also using the view's image drawing and rendering process and the status bar is also using the view's image drawing and rendering process (not shown in the figure), and the map application is also using the SurfaceView's image drawing and rendering process, for example, steps C11 to C16 in the figure.

[0219] It should be noted that for the sake of clarity, although the diagram first describes the map application using the view's image drawing and rendering, and then describes the map application using the SurfaceView's image drawing and rendering, in actual applications, there is no strict order between the view's drawing and rendering and the SurfaceView's drawing and rendering.

[0220] C11, after the GL thread of the map application receives the drawing command issued by the application, it requests graphicbuffer2-1 from buffer queue 2.

[0221] C12, the GL thread of the map application produces based on the applied graphicbuffer2, and obtains graphicbuffer2-1 filled with image data.

[0222] C13, the GL thread of the map application sends graphicbuffer2-1 filled with image data back to buffer queue 2.

[0223] C14, after the GL thread of the map application sends graphicbuffer2-1 back to buffer queue 2, it notifies the GPU to perform rendering.

[0224] C15, GPU receives the rendering notification information and starts rendering the image in graphicbuffer2-1.

[0225] C16, after GPU rendering is completed, notify SurfaceFlinger to consume graphicbuffer2-1.

[0226] After receiving the notification consumption information, SurfaceFlinger sets the consumption flag to 0, so it does not have the ability to consume graphicbuffer2-1 in buffer queue 2.

[0227] B17, after SurfaceFlinger receives the Vsync2 signal, it starts consuming graphicbuffers from the buffer queue 1 of the map application and the buffer queues of other applications (system desktop, status bar, etc.) to obtain the synthesized image.

[0228] B18, SurfaceFlinger releases graphicbuffer1-1 and the graphicbuffers in the buffer queues of other applications (system desktop, status bar, etc.) after synthesizing the image.

[0229] In addition, it should be noted that when SurfaceFlinger receives Vsync2, the map application also receives the Vsync2 signal, so the UI thread of the map application executes steps D11 to D16.

[0230] This process can refer to steps B11 to B16, except that the graphic buffer requested from buffer queue 1 is graphic buffer 1-2. In this process, since there are multiple graphic buffers in buffer queue 1, even if graphic buffer 1-1 is not released, graphic buffer 1-2 can be obtained to produce the next frame of image.

[0231] However, when the GL thread receives the application's drawing task and requests an idle graphics buffer from buffer queue 2, the map application's GL thread will be unable to successfully request the graphics buffer because SurfaceFlinger has not consumed graphicbuffer2-1 and there is only one graphics buffer in buffer queue 2 (it is queued, not idle). Therefore, the GL thread of the map application will not be able to successfully request the graphics buffer during the animation. Naturally, the GPU will not perform image rendering using SurfaceView, which can reduce the GPU load.

[0232] D17, when SurfaceFlinger receives Vsync3, it consumes graphicbuffer1-2 in buffer queue 1 of the map application and graphicbuffers in buffer queues of other applications (for example, the system desktop and status bar) to obtain the synthesized image.

[0233] D18, SurfaceFlinger releases graphicbuffer1-2 in the map application's buffer queue 1, as well as graphicbuffers in the buffer queues of other applications.

[0234] During the animation, the GPU no longer renders images drawn with SurfaceView, significantly reducing the GPU load and preventing other applications or threads from rendering images. SurfaceFlinger can synthesize and display other images according to the Vsync signal cycle, thus reducing lag or screen freezes. From the user's perspective, the map interface gradually becomes smaller, the amount of content obscured on the system desktop gradually decreases, and the system status bar is updated in real time based on actual conditions. Users don't care about changes in the map image during the animation that returns the map application to the background to display the system desktop, so the user experience is not affected.

[0235] After the animation ends, SurfaceFlinger can modify the number of graphic buffers in buffer queue 2 and the consumption flag of SurfaceFlinger to return to the initial state.

[0236] E11, after the system desktop (Launcher) detects the end of the animation, it sends the information of the end of the animation to SurfaceFlinger.

[0237] E12, after SurfaceFlinger receives the information that the animation ends, it changes the number of graphic buffers in buffer queue 2 to n (initial state).

[0238] E13, after SurfaceFlinger receives the information that the animation effect ends, it sets the consumption flag to 1 (initial state, as an example of the fourth flag).

[0239] Similarly, the order in which step E12 and step E13 are executed is not limited.

[0240] After step E12 takes effect, since the time when the application actually exits the background and stops is later than the end time of the animation effect, the GL thread of the map application may still receive the drawing tasks issued by the map application. Of course, it may not receive the drawing tasks issued by the map application. If the GL thread of the map application receives the drawing tasks issued by the application, it can apply for an idle graphic buffer from buffer queue 2 to continue subsequent operations. If SurfaceFlinger receives the Vsynci (i is greater than 3) signal, it also starts to consume the graphic buffer in the buffer queue 1 of the map application, the graphic buffers in the buffer queues of other applications, and the graphic buffer2-1 in the buffer queue 2 of the map application, so that the graphic buffer in buffer queue 2 is rotated.

[0241] The embodiment of the present application also provides another method for reducing GPU load. This method for reducing GPU load is illustrated in the scenario shown in FIG2 . This method corresponds to the above-mentioned method (2) for reducing load.

[0242] 9 , which shows the specific structure of the graphic buffer in the buffer queue 2 according to an embodiment of the present application. Other information can be stored in the graphic buffer memory structure.

[0243] For example, if there are four graphic buffers in buffer queue 2, other information about each graphic buffer is stored in its own meta data.

[0244] Each meta Date stores the image width (width), image height (height), image pixel format (format), layer count (number of layers), and image usage rules (usage).

[0245] In the embodiments of the present application, a block fence can be added to the meta data of each graphics buffer through an extended design. During the non-animation period, the block fence is set to -1. During the animation period, the block fence is set to 1 (as an example of the first flag). Of course, the block fence parameters are only used for example. In actual applications, other settings can be made according to actual conditions.

[0246] Therefore, when the application side requests an idle graphics buffer from buffer queue 2, it needs to additionally obtain the block fence of the graphics buffer.

[0247] If the block fence obtained is 1, then wait until the animation ends and the block fence in buffer queue 2 is set to -1 (as an example of the second flag). Only then can a block fence with a value of -1 be obtained. In a specific implementation, this block fence can be transmitted together with the NO-fence signal. After waiting for the NO-fence signal, the value of the block fence is checked and the buffer is determined based on the value of the block fence.

[0248] After obtaining the block fence value of -1, the application side continues to request an idle graphics buffer from buffer queue 2.

[0249] Refer to Figure 10, which is a timing diagram of the method for reducing GPU load based on block fence implementation provided in an embodiment of the present application. This embodiment no longer describes the operations performed by the UI thread of each application. The operations performed by the UI thread are not affected by whether the animation starts or ends. Specifically, refer to the operations performed by the UI thread of the map application shown in Figure 8. Among them, in non-animation devices, the block fence of the graphics buffer in buffer queue 2 is -1.

[0250] F11, the GL thread of the map application obtains the block fence of -1 from buffer queue 2.

[0251] F12, after the GL thread of the map application obtains the block fence of -1, it requests the idle graphicbuffer2-1 from buffer queue 2.

[0252] F13, the GL thread of the map application applies for graphicbuffer2-1 and starts production, obtaining graphicbuffer2-1 filled with image data.

[0253] F14, the GL thread of the map application sends graphicbuffer2-1 filled with image data back to buffer queue 2.

[0254] F15, after the GL thread of the map application sends graphicbuffer2-1 back to buffer queue 2, it notifies the GPU to render.

[0255] F16, after receiving the rendering notification information, the GPU starts rendering the image data in graphicbuffer2-1.

[0256] F17, GPU notifies SurfaceFlinger for consumption after rendering is completed.

[0257] F18, SurfaceFlinger consumes graphicbuffer2-1 in buffer queue 2 based on the Vsync signal, as well as graphicbuffers in the buffer queues of other applications to obtain the synthesized image.

[0258] F19, SurfaceFlinger releases graphicbuffer2-1, as well as the graphicbuffers of other applications.

[0259] The above process is the process during the non-animation period. The following describes the process during the animation period.

[0260] G11, after detecting a gesture operation, the system desktop (launcher) determines the start of the animation based on the gesture operation and sends the animation start information to SurfaceFlinger.

[0261] G12, after SurfaceFlinger receives the information that the animation starts, it changes the blockfence of the graphics buffer in buffer queue 2 to 1.

[0262] H11, the GL thread of the map application obtains the blockfence of the idle graphics buffer in buffer queue 2 as 1 based on the map application task.

[0263] Since the block fence value obtained by the GL thread of the map application is 1, the application task enters the waiting state, so the subsequent drawing of the image using SurfaceView will be stopped, and the GPU will no longer perform the rendering of the image using SurfaceView. Therefore, the GPU load can be reduced.

[0264] When the system desktop detects that the animation has ended, it needs to set the blockfence of the graphicsbuffer in buffer queue 2 to -1.

[0265] J11, after the system desktop detects the end of the animation, it sends the information that the animation ends to SurfaceFlinger.

[0266] J12, after SurfaceFlinger receives the information that the animation ends, it sets the blockfence of the graphicsbuffer in buffer queue 2 to -1.

[0267] After setting the block fence value of the graphics buffer in buffer queue 2 to -1, if the GL thread receives the drawing task sent by the map application, it will perform subsequent actions.

[0268] K11, the GL thread obtains the block fence of the graphics buffer from buffer queue 2 based on the map application task, which is -1.

[0269] After the GL thread obtains the block fence of -1, it resumes the application task that was previously in the waiting state.

[0270] K12, GL thread gets the idle graphicsbuffer2-2 from buffer queue 2.

[0271] K13, GL thread performs production and obtains graphicbuffer2-2 filled with image data.

[0272] K14, the GL thread sends graphicbuffer2-2 filled with image data back to buffer queue 2.

[0273] K15, after the GL thread sends graphicbuffer2-2 back to buffer queue 2, it notifies the GPU to perform rendering.

[0274] K16, after receiving the rendering notification information, the GPU starts rendering the image data in graphicbuffer2-2.

[0275] K17, GPU notifies SurfaceFlinger to consume graphicsbuffer2-2 after rendering is completed.

[0276] K18, SurfaceFlinger consumes graphicbuffer2-2 in buffer queue 2, as well as the buffers in buffer queue 1 of the map application and the buffer queues of other applications to obtain the synthesized image.

[0277] K19, after SurfaceFlinger consumption, releases graphicbuffer2-2, as well as the graphicbuffers of the map application's buffer queue 1 and other applications' buffer queues.

[0278] As mentioned above, the GL thread will only apply for the graphics buffer from buffer queue 2 when it gets the block fence value of -1. Otherwise, it will remain in the waiting state and will no longer apply for the graphics buffer from buffer queue 2. In this way, the application side will not apply for the graphics buffer in buffer queue 2 based on SurfaceView, and naturally it will not be generated. The GPU will not need to render the SurfaceView later. Therefore, the GPU load can be reduced, and the occurrence of lag or screen freezing can be reduced.

[0279] The embodiment of the present application also provides another method for reducing GPU load. This method for reducing GPU load is illustrated in the scenario shown in FIG2 . This method corresponds to the above-mentioned method (3) for reducing load.

[0280] Referring to Figure 11 , there is a timing diagram illustrating reducing GPU load by proactively stopping the application from requesting graphics buffers in buffer queue 2, as provided in an embodiment of the present application. This embodiment does not describe the operations performed by the UI thread of each application, as the operations performed by the UI thread are not affected by whether the animation has started or ended.

[0281] Before the animation begins, the electronic device performs the following:

[0282] L11, map application starts.

[0283] L12, the map application registers the callback function with the listener.

[0284] L13, the map application sends a listener to SurfaceFlinger.

[0285] L14, after SurfaceFlinger receives the listener, it saves the listener to the layer of the map application.

[0286] In a specific implementation, after the map application is started, it can apply to SurfaceFlinger for the layer buffer queue corresponding to the map application, and SurfaceFlinger stores the listener in the layer corresponding to the map application.

[0287] M11, the map application requests graphicbuffer2-1 from buffer queue 2 through the GL thread.

[0288] M12: The map application applies for graphicbuffer2-1 through the GL thread and starts production, obtaining graphicbuffer2-1 filled with image data.

[0289] M13, the map application sends graphicbuffer2-1 filled with image data back to buffer queue 2 through the GL thread.

[0290] M14, the map application notifies the GPU to render after sending graphicbuffer2-1 back to buffer queue 2 through the GL thread.

[0291] M15, after receiving the rendering notification information, the GPU starts rendering the image data in graphicbuffer2-1.

[0292] M16, GPU notifies SurfaceFlinger for consumption after rendering is completed.

[0293] M17, SurfaceFlinger consumes graphicbuffer2-1 in buffer queue 2 based on the Vsync signal, as well as graphicbuffers in the buffer queues of other applications to obtain the synthesized image.

[0294] The above process is the process during the non-animation period. The following describes the process during the animation period.

[0295] N11, after the system desktop (launcher) detects the gesture operation, it can determine the start of the animation based on the gesture operation and send the animation start information to SurfaceFlinger.

[0296] When the system desktop determines the start of the animation, it can determine the package name of the map application displayed by the electronic device during the gesture operation. Therefore, when sending the information of the start of the animation, the package name of the map application is also transmitted.

[0297] N12. After receiving the animation start message, SurfaceFlinger determines the listener based on the map application package name carried in the animation start message. N13. SurfaceFlinger transmits the animation start message to the map application through the determined listener.

[0298] N14, the map application sets the animation start flag. Under the animation start flag, the application operation of the GL thread is in the wait state.

[0299] N15, when the GL thread of the map application receives the drawing task sent by the map application, it applies for a graphics buffer from buffer queue 2. This application operation will be in a waiting state (wait).

[0300] Because the application operation of the GL thread of the map application is in the wait state, it will not apply for the graphics buffer in buffer queue 2, and will not perform subsequent drawing and rendering, etc., which reduces the GPU load during the animation and thus reduces the lag or screen freezing during the animation.

[0301] In actual applications, you can also set the application operation of the GL thread under the animation start flag to an invalid operation. In this way, when the GL thread of the map application needs to apply for the graphic buffer in buffer queue 2, the application is an invalid operation, and naturally no subsequent drawing will be performed. The GPU will not render the image using SurfaceView, which reduces the GPU load during the animation, thereby reducing the freeze or freezing of the screen during the animation.

[0302] In addition, if the application operation of the GL thread under the animation start flag is set to an invalid operation, the application may trigger a stuck situation. Therefore, the application side can set a timeout. For example, if the graphics buffer is not applied after a certain period of time, the application task can be triggered again.

[0303] When the system desktop detects the end of the animation, it needs to pass the animation end information to the map application through SurfaceFlinger, and the map application modifies the animation start flag to the animation end flag.

[0304] P11, after the system desktop (Launcher) detects the end of the animation, it sends an animation end message to SurfaceFlinger, along with the package name of the map application determined when the animation started.

[0305] P12, after SurfaceFlinger receives the information that the animation effect ends, it determines the listener according to the package name of the map application carried in the information that the animation effect ends.

[0306] P13, SurfaceFlinger transmits the information that the animation ends to the map application through the specified listener.

[0307] P14, after receiving the information that the animation effect ends, the map application sets the animation effect end flag.

[0308] The operation of applying for graphics buffer at the end of the animation is a valid operation.

[0309] P15, the GL thread of the map application requests graphicbuffer2-2 from buffer queue 2.

[0310] As another embodiment of the present application, as shown in Figure 12, in step N11, when the system desktop transmits the information of the start of the animation effect to SurfaceFlinger, it can carry not only the package name of the map application related to the animation effect, but also the animation duration t. This duration can be recorded as the first duration. When SurfaceFlinger transmits the information of the start of the animation effect to the map application, it also transmits the animation duration t.

[0311] After receiving the animation start information and the animation duration t, the map application sets the animation start flag and sets the waiting time for the request task to t. The map application's GL thread will wait for t before continuing to request a graphics buffer from buffer queue 2. Of course, in actual applications, this waiting time can be different from the animation duration, for example, longer than the animation duration.

[0312] After the GL thread waits for a certain period of time to apply for the graphics buffer task, the GL thread continues to apply for graphics buffer 2-2 from buffer queue 2 according to the drawing task of the map application.

[0313] 13 and 14 , which are comparison diagrams of operations performed by the GL thread of the map application before and after the animation when the load reduction method provided by the embodiment of the present application is adopted.

[0314] Before the animation begins, the GL thread can request a graphics buffer, draw to the graphics buffer, return the drawn graphics buffer to the queue, and notify the GPU to render. After receiving the rendering notification, the GPU renders the SurfaceView.

[0315] After the animation begins, due to various settings to reduce the GPU load (for example, the various methods mentioned above), the GL thread of the map application will continue to be in a waiting state or directly fail to apply for the graphics buffer. That is, the GL thread of the map application cannot successfully apply for the graphics buffer, and the GPU will not perform the rendering work of the image drawn by SurfaceView, which can reduce the GPU load and reduce lag or freezing.

[0316] As another embodiment of the present application, when SurfaceFlinger is performing synthesis, it needs to wait for each layer involved in the synthesis to be rendered before it can synthesize. In this way, when SurfaceFlinger cannot obtain all layers or all NO-fence signals, it will not be able to synthesize and subsequently display according to the predetermined frame rate, and a freeze or frozen screen may occur.

[0317] Some systems provide property configuration. SurfaceFlinger reads the property configuration at startup; when the property is configured to always, the system will latch all buffers without NO-fence signals, that is, SurfaceFlinger no longer waits for the NO-fence signal, but instead organizes the graphics buffer in the queue in advance and passes it to HWC. HWC will then determine whether it has received all NO-fence signals when actually synthesizing. This is equivalent to saving SurfaceFlinger waiting time. However, there is a problem: HWC also needs to wait for the NO-fence signal, so if you continue to maintain the drawing and rendering of SurfaceView, it may not be possible to synthesize and subsequently display it at the predetermined frame rate, and there may be lag or frozen screen.

[0318] This application can set the buffer queue of SurfaceView images that prohibit SurfaceFlinger from latching and not releasing NO-fence during the animation without pausing the drawing and rendering of SurfaceView, that is, not allowing the unfinished rendering of the graphicbuffer to be consumed by SurfaceFlinger. In this way, SurfaceFlinger does not need to wait for the unfinished rendering of the graphicbuffer of SurfaceView when synthesizing.

[0319] For other view-based buffer queues, the latched no-signal buffer mechanism is enabled, and when SurfaceFlinger or HWC is synthesizing, it waits for the NO-fence signal to consume the graphic buffer. Referring to Figure 15, when SurfaceFlinger needs to wait for the images in each buffer queue to be rendered before synthesizing these images, it is possible that when the Vsync signal arrives, the image drawn by the system desktop using the view and the image drawn by the map application using the view can be rendered. SurfaceFlinger has received the NO-fence signal of the above two rendering processes, but because the image drawn by the map application using SurfaceView has not been rendered and the fence has not been released, SurfaceFlinger is blocked during synthesis. SurfaceFlinger needs to wait until the image corresponding to the SurfaceView is rendered and the fence is obtained before it can perform the synthesis of these layers. In actual applications, this synthesis operation may be handed over to the GPU for synthesis.

[0320] During the animation, SurfaceFlinger does not allow unfinished graphics buffers to be consumed. Similarly, after SurfaceFlinger receives the Vsync signal, SurfaceFlinger determines that the rendering of the image drawn by the view has completed and received the NO-fence signal, and the rendering of the image drawn by the SurfaceView has not completed and has not received the NO-fence signal, and then synthesizes the images in the buffer queues other than the buffer queue corresponding to the SurfaceView. Taking Figure 15 as an example, the images in the buffer queue corresponding to the system desktop and the buffer queue corresponding to the map application UI thread are synthesized. After synthesis, the synthesized image can be sent for display. In this case, although the content displayed by the electronic device does not update the content drawn by SurfaceView, the content related to the view in the map application and the system desktop are refreshed and displayed normally, which reduces the lag or freezing of the electronic device due to waiting for synthesis. Since the user will not care about the content related to SurfaceView in the map application during the animation, the missing content will not cause a poor user experience.

[0321] The above examples all use a map application as an example, taking the animation effect during the electronic device displaying the map application switching to the display system desktop as an example. In actual applications, the map application can also be any other application that uses SurfaceView for rendering during normal display, such as video applications, game applications, camera applications, etc. Similarly, the animation period can also be other periods when the user is not very concerned about the content corresponding to SurfaceView, such as the hot start of the above application, multi-window startup, etc.

[0322] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0323] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is run on an electronic device, it can implement the steps in the above-mentioned various method embodiments.

[0324] The embodiments of the present application further provide a computer program product. When the computer program product is run on an electronic device or a wireless router, the electronic device can implement the steps in the above-mentioned various method embodiments.

[0325] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0326] The present application also provides a chip comprising a processor coupled to a memory, wherein the processor invokes a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip may be a single chip or a chip module composed of multiple chips.

[0327] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0328] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0329] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An interface display method, characterized in that: include: Receiving a first operation from a user; In response to the first operation, starting a first application, wherein the layers of the first application include layers drawn by the view and layers drawn by the SurfaceView; After the first application is started, the application interface of the first application is displayed; Wherein, in the process of displaying the application interface of the first application, the method further includes: Rendering a first layer of the first application, where the first layer is a layer drawn by using a view; Rendering a second layer of the first application, where the second layer is a layer drawn using SurfaceView; Combining the first layer and the second layer to obtain a third layer; Displaying a first interface of the first application, where the first interface is obtained based on the third layer; receiving a second operation of the user; In response to the second operation, switching from displaying the first application to displaying a second application; Wherein, during the process of switching from displaying the first application to displaying the second application, the method further includes: Rendering a fourth layer of the first application, where the fourth layer is a layer drawn using a view; Rendering a fifth layer of the second application, where the fifth layer is a layer drawn using a view; Combining the fourth layer and the fifth layer to obtain a sixth layer; A second interface during the display switching process is obtained based on the sixth layer.

2. The method according to claim 1, characterized in that Before rendering the second layer of the first application, the method further includes: Apply for the first buffer from the first buffer queue; A layer is drawn in the first buffer using SurfaceView to obtain a second layer of the first application.

3. The method according to claim 2, characterized in that The method is applied to an electronic device, the electronic device comprising: a launcher and the first application, and after receiving the second operation of the user, the method further comprises: The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application; The launcher sends the information of the start of the motion effect and the first duration to the first application. After the first application receives the information of the start of the motion effect and the first duration, the task of the rendering thread of the first application to apply for a buffer from the first buffer queue is in a waiting state; After the first application receives the first duration of the information indicating the start of the motion effect, the task of the rendering thread applying for a buffer from the first buffer queue is restored to an execution state.

4. The method according to claim 3, characterized in that The electronic device further includes: SurfaceFlinger; after the first application is started, the method further includes: The first application registers a listener; The first application sends the listener of the first application and the package name of the first application to the SurfaceFlinger; the SurfaceFlinger stores the package name of the first application and the listener of the first application; The launcher sends the information about the start of the motion effect and the first duration to the first application, including: The launcher sends the information about the start of the animation, the first duration, and the the application package name of the first application; After receiving the information about the start of the motion effect, the first duration, and the application package name of the first application, the SurfaceFlinger determines a listener for the first application according to the application package name of the first application; The SurfaceFlinger sends the information about the start of the motion effect and the first duration to the first application through the listener of the first application.

5. The method according to claim 2, characterized in that The method is applied to an electronic device, the electronic device comprising: a launcher and the first application, and after receiving the second operation of the user, the method further comprises: The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application; The launcher sends information about the start of the motion effect to the first application. After the first application receives the information about the start of the motion effect, the task of the rendering thread of the first application to apply for a buffer from the first buffer queue is in a waiting state.

6. The method according to claim 5, characterized in that When the task of the rendering thread of the first application to apply for a buffer from the first buffer queue is in a waiting state, the method further includes: After detecting that the motion effect has ended, the launcher sends a message that the motion effect has ended to the first application; After the first application receives the information that the motion effect ends, the rendering thread resumes the execution state of the task of applying for a buffer from the first buffer queue.

7. The method according to claim 2, characterized in that The method is applied to an electronic device, the electronic device comprising: a launcher and a SurfaceFlinger, the data structure of the buffer in the first cache queue comprises a block fence, and after receiving the second operation of the user, the method further comprises: The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application; The launcher sends the information of starting the animation effect to the SurfaceFlinger; After receiving the information about the start of the animation, the SurfaceFlinger sets the block fence in the data structure of the buffer in the first cache queue to a first flag. When the block fence of the buffer in the first buffer queue is the first flag, the task of the rendering thread of the first application to apply for a buffer from the first buffer queue is in a waiting state.

8. The method according to claim 7, characterized in that When the task of the rendering thread of the first application to apply for a buffer from the first buffer queue is in a waiting state, the method further includes: After detecting that the animation has ended, the launcher sends information about the end of the animation to the SurfaceFlinger; After receiving the information that the animation ends, the SurfaceFlinger sets the block fence in the data structure of the buffer in the first cache queue to a second flag. When the block fence of the buffer in the first buffer queue is the second flag, the task of the rendering thread to apply for the buffer from the first buffer queue is restored to an execution state.

9. The method according to claim 2, characterized in that The method is applied to an electronic device, the electronic device includes a launcher, a SurfaceFlinger and the first application, and the method further includes: The launcher determines, based on the second operation, to switch from the application interface of the first application to the application interface of the second application; The launcher sends the information of starting the animation effect to the SurfaceFlinger; After receiving the information that the animation effect starts, the SurfaceFlinger sets the number of buffers in the first buffer queue to 1, sets the consumption flag of the first buffer queue to the third flag, and when the second buffer in the first buffer queue is not idle, the task of the rendering thread of the first application to apply for the second buffer from the first buffer queue is in a waiting state, and the second buffer is the buffer in the first buffer queue when the number of buffers in the first buffer queue is 1.

10. The method according to claim 9, characterized in that The electronic device further includes: a GPU, and after the SurfaceFlinger sets the number of buffers in the first buffer queue to 1 and sets the consumption flag of the first buffer queue to a third flag, the method further includes: If the second buffer in the first buffer queue is in an idle state, the rendering thread of the first application applies for the second buffer from the first buffer queue; The rendering thread of the first application uses SurfaceView to draw a layer in the second buffer to obtain the seventh layer of the first application; The rendering thread of the first application returns the second buffer storing the seventh layer to the first buffer queue, and sets the second buffer to be in the queue state; The rendering thread of the first application sends a rendering instruction to the GPU; After receiving the rendering instruction, the GPU renders the seventh layer in the first buffer queue; The GPU sends a synthesis instruction to SurfaceFlinger after rendering is completed; The SurfaceFlinger does not respond to the synthesis instruction when the consumption flag bit of the first buffer queue is the third flag.

11. The method according to claim 9, characterized in that The method further comprises: After detecting that the animation has ended, the launcher sends information about the end of the animation to the SurfaceFlinger; After receiving the information that the animation effect ends, the SurfaceFlinger sets the number of buffers in the first buffer queue to the initial number, sets the consumption flag of the first buffer queue to the fourth flag, and the SurfaceFlinger responds to the synthesis instruction when the consumption flag of the first buffer queue is the fourth flag. The initial number is the number before the number of buffers in the first buffer queue is switched to 1.

12. The method according to any one of claims 1 to 11, characterized in that The first application is a map application, a video application or a game application.

13. The method according to claim 3, characterized in that The first duration is an estimated duration of a process of switching from displaying the first application to displaying the second application.

14. An electronic device, characterized in that: The electronic device comprises a processor, and the processor is used to call a computer program in a memory to execute the method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 13.

16. A chip, characterized in that: The chip comprises a processor, and the processor is used to call a computer program in a memory to execute the method according to any one of claims 1-13.

Images