Interface processing method and electronic device

Abstract

Provided in the present application are an interface processing method and an electronic device. The method comprises: an electronic device starting a first application, and displaying a first interface of the first application, wherein a first layer corresponding to the first interface is composed by SF; in response to a first operation, displaying a second interface of the first application, wherein in the process of displaying the second interface, an activity corresponding to the first interface is being run, and the first layer is not composed by SF; and in response to a second operation, displaying a third interface of the first application, wherein in the process of executing the second operation, the first layer is composed by SF, and after the second operation, the first layer is not composed by SF. Thus, the interface processing process is optimized, so that the power consumption of an electronic device is reduced.

Description

A method for processing interfaces and an electronic device Technical Field

[0001] This application relates to the field of terminals, and more particularly to an interface processing method and an electronic device. Background Technology

[0002] The process of running applications and outputting the application interface on electronic devices involves processing stages such as drawing, rendering, layer composition, and display. These processing stages require a large amount of power. Therefore, how to optimize the interface processing to reduce the power consumption of electronic devices is an urgent problem to be solved.

[0003] Summary of the Invention

[0004] This application provides a method for processing an interface and an electronic device. The method includes: the electronic device launching a first application and displaying a first interface of the first application, wherein a first layer corresponding to the first interface is composited using Flash composite (SF); responding to a first operation, displaying a second interface of the first application, wherein during the display of the second interface, an activity corresponding to the first interface is being run, and the first layer is not composited using Flash composite; responding to a second operation, displaying a third interface of the first application, wherein during the execution of the second operation, the first layer is composited using Flash composite, and after the second operation, the first layer is not composited using Flash composite. This optimizes the interface processing and saves power consumption of the electronic device.

[0005] In a first aspect, this application provides an interface processing method applied in an electronic device. The method includes: launching a first application, displaying a first interface of the first application, and having a first layer composited by a display compositing system (SF); the first interface corresponds to the first layer; in response to a first operation, displaying a second interface of the first application, wherein during the display of the second interface, an activity corresponding to the first interface is being run, and the first layer is not composited by the SF.

[0006] After implementing the method described in the first aspect, in scenarios where the interface of the first application is switched, the first layer corresponding to the invisible first interface stops participating in the compositing, reducing the resources occupied by the SF compositing stage, reducing the power consumption of the electronic device, and also improving the running speed of the first application.

[0007] In conjunction with the method described in the first aspect, in response to the first operation, a second interface of the first application is displayed, specifically including: in response to the first operation, a second image frame is synthesized based on a second layer, and a second interface is displayed based on the second image frame, wherein the second layer corresponds to the second interface.

[0008] In this way, after switching to the second display interface, SF can synthesize the second image frame based on the second layer corresponding to the second interface, thereby preventing the first layer from being synthesized and reducing the power consumption of electronic devices.

[0009] In conjunction with the method described in the first aspect, the method further includes: displaying a third interface of the first application in response to a second operation, wherein the first layer is composited by the SF during the execution of the second operation, and the first layer is not composited by the SF after the second operation.

[0010] In this way, when the second operation triggers the first application to switch from the second interface to the first interface, the first layer can be synthesized by SF in a timely manner during a period of time during the execution of the second operation, thereby ensuring that the first interface can be displayed in a timely manner after the second operation. In conjunction with the method described in the first aspect, the second operation includes a press operation and a release operation, and the third interface is displayed in response to the release operation. The method further includes: in response to the press operation, synthesizing a first image frame based on the first layer and the second layer; displaying the second interface based on the first image frame; and the second layer corresponding to the second interface.

[0011] Thus, when the second operation includes a press and release click operation, after the press operation, the first layer is promptly included in the compositing process. This ensures that if the second operation triggers a return to the first interface, the first interface can be displayed promptly based on the image frame composed of the first layer after the release operation. It is understood that to ensure the aforementioned effect, even if the second operation triggers a switch to the third interface rather than a return to the first interface, the second operation must be pre-judged as a return to the first interface to execute the above method. In conjunction with the method described in the first aspect, the second operation includes a press operation, a slide operation, and a release operation. The third interface is displayed in response to the release operation. The method further includes: in response to the press operation, compositing a first image frame based on the first layer and the second layer; displaying the second interface based on the first image frame; the second layer corresponds to the second interface; in response to the slide operation, compositing a third image frame based on the first layer, the second layer, and the third layer; displaying an interface switching animation based on the third image frame, the interface switching animation including a fourth interface; the third layer corresponds to the third interface.

[0012] Thus, when the second operation is a sliding operation including pressing, sliding, and releasing, similar to the click operation, after the press operation, the first layer is promptly included in the composition process. This ensures that if the second operation triggers a return to the first interface, the first layer can be included in the composition during the sliding operation to display the animation effect of switching to the first interface, and after the release operation, the first interface can be displayed based on the image frame composed of the first layer. It is understandable that, to ensure the aforementioned effects, even if the second operation triggers a switch to the third interface rather than a return to the first interface, the second operation must be pre-judged as a return to the first interface to execute the above method.

[0013] In conjunction with the method described in the first aspect, the method further includes: in response to the press operation, starting a first timer; if the duration of the first timer reaches a first duration, detecting whether the first layer is covered; if the first layer is covered, the first layer is not composited by the SF; if the first layer is not covered, the first layer is composited by the SF.

[0014] In conjunction with the method described in the first aspect, the method further includes: in response to the press operation, starting a first timer, and before the duration of the first timer reaches a first duration, synthesizing a first image frame based on the first layer and the second layer.

[0015] Thus, after the press operation corresponding to the second operation, although the first layer is synthesized in a timely manner based on the predicted result that the second operation will return to the first interface, the second operation may actually trigger a return to the third interface. Therefore, further coverage detection of the first layer is required after the press operation. Furthermore, to reduce the power consumption of layer detection, the first layer is detected only after the press operation has reached a certain duration. If the first layer is covered, it is not synthesized; only if it is not covered is it synthesized. This optimizes the layer synthesis process through accurate detection based on prediction.

[0016] In conjunction with the method described in the first aspect, the method further includes: if the timing duration of the first timer has not reached the first duration, in response to the lift operation, turning off the first timer.

[0017] In this way, if a release operation is received within a short period of time after the press operation, such as within the first duration, there is no need to wait until the first duration after the press operation is reached before performing accurate detection of the first layer, thus reducing the power consumption caused by frequent layer detection.

[0018] In conjunction with the method described in the first aspect, the method further includes: in response to the lift operation, starting a second timer; if the duration of the second timer reaches a second duration, detecting whether the first layer is covered; if the first layer is covered, the first layer is not composited by the SF; if the first layer is not covered, the first layer is composited by the SF.

[0019] Thus, after the press operation corresponding to the second operation, although the first layer is synthesized in a timely manner based on the predicted result that the second operation will return to the first interface, the second operation may actually trigger a return to the third interface. Therefore, further coverage detection of the first layer is required after the release operation. Furthermore, to reduce the power consumption of layer detection, layer detection is performed only after the interface is stably displayed. Therefore, the first layer is detected only after the second duration of the release operation. If the first layer is covered, it is not synthesized; only if it is not covered is it synthesized. This optimizes the layer synthesis process through precise detection based on prediction.

[0020] In conjunction with the method described in the first aspect, detecting whether the first layer is covered specifically includes: detecting whether the first layer is covered based on the layer order and vertex coordinates of the first layer, and the layer order and vertex coordinates of the third layer; determining that the first layer is covered if the following conditions are met: the layer order of the first layer is lower than the layer order of the third layer; and the vertex coordinates of the first layer are located in the area contained by the vertex coordinates of the third layer.

[0021] In this way, by detecting the layer hierarchy, it can be determined whether the first layer is covered by the layer below, and by detecting the coordinates of the layers, it can be determined whether the entire content of the first layer is contained in the area covered by the third layer, thus improving the accuracy of layer optimization.

[0022] In conjunction with the method described in the first aspect, when the first layer is covered and before the first layer is composited by the SF, the method further includes: determining that the third layer does not contain transparent portions.

[0023] In this way, even when the first layer is covered by the third layer, it is possible to further detect whether the third layer is transparent, thereby accurately detecting whether the first layer is invisible. Only when the first layer is invisible will the first layer not be composited in SF, which can further improve the accuracy of layer optimization.

[0024] In conjunction with the method described in the first aspect, the method further includes: setting the control flag of the first application to true based on the fact that the first layer is covered; and preventing the first layer from being composited by the SF based on the fact that the control flag of the first application is true.

[0025] In this way, if the first layer is detected to be covered, the control flag is set to true to instruct SF not to composite the first layer.

[0026] In conjunction with the method described in the first aspect, the method further includes: determining whether the first application belongs to a preset whitelist of applications; based on the fact that the first application belongs to the whitelist of applications, during the display of the second interface, the activity corresponding to the first interface is being run, and the first layer is not composited by the SF.

[0027] In this way, by pre-setting applications that require layer optimization in a whitelist of applications, the method provided in this application is only implemented for applications within the whitelist, without having to be implemented for other applications, thereby saving power consumption of electronic devices.

[0028] In conjunction with the method described in the first aspect, the first interface is the main interface of the first application, and the second interface is a non-main interface of the first application.

[0029] In this way, after the main interface of the first application is switched to a non-main interface, the activities corresponding to the invisible main interface continue to run, while the first layer corresponding to the invisible main interface is not composited. This ensures that the activities of the first interface are preserved, but does not waste power by compositing the first layer.

[0030] In conjunction with the method described in the first aspect, when the second operation is a click operation, in addition to triggering the first application to switch from the second interface to the third interface, it can also trigger the first application to return from the second interface to the first interface. Regardless of whether the second operation triggers a switch to the third interface or a return to the first interface, the method described above needs to be executed after receiving the press operation corresponding to the second operation, and the method described above also needs to be executed after receiving the release operation corresponding to the second operation.

[0031] In conjunction with the method described in the first aspect, when the second operation is a swipe operation, in addition to triggering the first application to switch from the second interface to the third interface, it can also trigger the first application to return from the second interface to the first interface, and it can also trigger the first application to cancel the switch during the switching process (for example, when the swipe operation corresponding to the second operation includes: first swiping in the first direction, then keeping the hand still and swiping in the second direction, the first direction and the second direction are opposite), that is, ultimately maintaining the display of the second interface. Regardless of whether the second operation triggers a switch to the third interface, a return to the first interface, or continued display of the second interface, the method described above needs to be executed after receiving the press operation corresponding to the second operation, and the method described above also needs to be executed after receiving the release operation corresponding to the second operation.

[0032] In a second aspect, this application provides an electronic device comprising: one or more memories, one or more processors, and a display screen; the memories are coupled to the one or more processors, the memories being used to store computer program code including computer instructions, and the one or more processors calling the computer instructions to cause the electronic device to perform the methods described in any of the first aspects.

[0033] Thirdly, this application provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method described in any of the first aspects.

[0034] Fourthly, this application provides a computer program product, including a computer program that, when run on an electronic device, causes the electronic device to perform the method described in any of the first aspects. Attached Figure Description

[0035] Figure 1 illustrates the interface processing mechanism involved in the application runtime provided in the embodiments of this application;

[0036] Figures 2A-2C are schematic diagrams of switching from the front-end interface to the back-end interface of an application provided in this application;

[0037] Figures 3A and 3B are schematic diagrams of returning to the previous interface by clicking, as provided in this application;

[0038] Figures 4A and 4B are schematic diagrams of returning to the previous interface via a swipe operation provided in this application;

[0039] Figures 5A and 5B are schematic diagrams of returning to the previous interface through a reverse sliding operation provided in this application;

[0040] Figure 6 is a schematic diagram of the interface processing method provided in this application;

[0041] Figure 7 shows the OS interaction flow for switching whitelisted applications to the foreground provided in this application;

[0042] Figure 8 shows the OS interaction flow for triggering whitelisted applications to enter control provided in this application;

[0043] Figure 9 shows the OS interaction flow for triggering the exit control of whitelisted applications provided in this application;

[0044] Figure 10 is a schematic diagram of the hardware architecture of the electronic device provided in the embodiment of this application;

[0045] Figure 11 is a schematic diagram of the software architecture of the electronic device provided in the embodiment of this application. Detailed Implementation

[0046] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0047] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0048] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0049] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with a user. It realizes the conversion between the internal form of information and the form that the user can receive. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0050] Figure 1 illustrates the interface processing mechanism involved in application runtime.

[0051] As shown in Figure 1, the tasks performed by the electronic device when running the first application include, but are not limited to, the following:

[0052] 1. Upon receiving the operation to launch the first application, the electronic device begins to launch the first application. Launching the first application can also be understood as opening, starting, or enabling the first application.

[0053] 2. During the startup of the first application, the task of opening the first interface is executed. The first interface is usually the main interface of the first application, or the interface displayed when the first application was in the foreground state last time it switched from the foreground to the background.

[0054] The tasks for opening the first interface include, but are not limited to, the following:

[0055] (1) Launching Activity A: Specifically, after the user inputs the operation to launch the first application, the process of the first application is started, and the first application can launch Activity A. Activity A corresponds to the first screen, and the Activity Manager Service (AMS) is responsible for managing the lifecycle of Activity A. The lifecycle of Activity A includes creating Activity A, running Activity A, resuming Activity A, and destroying Activity A. Usually, Activity A is the main Activity of the first application, and the first screen is the main screen of the first application.

[0056] (2) Drawing and rendering interface elements: During the startup of Activity A, corresponding service modules such as the Window Manager Service (WMS) are initialized. WMS can create a window for Activity A and request the Surface Flinger (SF) to create a Surface object for Activity A. This Surface object of Activity A corresponds to the window of Activity A and is the basic unit for drawing content. The system's drawing and rendering mechanism can complete the drawing and rendering of interface elements in the Surface.

[0057] (3) Composite Layers: SF is the core component responsible for handling layer composites. SF can composite the first layer provided by the first application, the layers provided by the System UI, etc., into image frames, and finally send them to the display screen through the display driver to display the corresponding first interface. Among them, the first layer corresponds to the first interface. At this time, the first layer is in a visible state. The visible state of a layer usually refers to the state where the layer's display level is at the top and is not covered by other layers. For a detailed introduction to the visibility and invisibility of layers, please refer to the description at S61 later.

[0058] (4) Display: SF will send the synthesized image frame to the display screen through the display driver for display, forming the first interface seen by the user.

[0059] 3. During the process of displaying the first interface, receive the operation to open the second interface and execute the task of opening the second interface.

[0060] The task of opening the second interface is similar to that of opening the first interface, which involves launching Activity B, drawing and rendering interface elements, compositing an image frame based on the second layer, and sending it to the display. Activity B corresponds to the second interface, and the second layer corresponds to the second interface. While Activity B is running, AMS maintains the running state of Activity A. Furthermore, while Activity A and Activity B are running, both the first and second layers participate in the layer composition process of SF. However, after the image frame composed of the first and second layers is sent to the display, the second interface is displayed instead of the first interface. In other words, during the SF composition process where both the first and second layers participate, the first layer is invisible (usually meaning its display layer is one level below the second layer, equivalent to the first layer being covered by the second layer), while the second layer is visible (usually meaning its display layer is at the top). Therefore, the display shows the second interface instead of the first interface.

[0061] 4. During the display of the second interface, receive the operation to return to the first interface, close the second interface and return to the first interface.

[0062] When performing the task of closing the second interface, the Activity corresponding to the second interface is usually destroyed (Destroy Activity B). This is because the second interface is not a key interface similar to the main interface provided by the first application, and the probability of users using the second interface for a long time and frequently is small, so there is no need for it to be constantly running.

[0063] 5. During the display of the first interface, receive an operation to close the first application, close the first application and close the first interface.

[0064] When performing the task of closing the first application, it is equivalent to switching the first application from the foreground to the background or ending its operation. In this case, the user usually will not use the key interface of the first application, so the Activity of the first interface will also be destroyed (Destroy Activity A).

[0065] As explained in the preceding description of Figure 1, during the operation of an application on an electronic device, when the application switches from the previous level interface to the next level interface, the previous level Activity corresponding to the previous level interface remains running, and the layer corresponding to the previous level interface is also composited in SF (Screen Rendering). In other words, although the previous level interface is not displayed on the screen, its layers still undergo drawing, rendering, and participate in the SF compositing process.

[0066] While the aforementioned interface processing mechanism aims to ensure a smooth user experience by quickly restoring the latest state of the previous interface when the user returns, it can potentially waste system resources, especially during interface transitions where the user cannot see the previous interface, yet the previous interface continues to consume system resources for updates (including drawing, rendering, and compositing layers).

[0067] To address the aforementioned problems, this application provides an interface processing method, comprising: an electronic device launching a first application and displaying a first interface of the first application, wherein a first layer is composited using Flash; the first interface corresponds to the first layer; in response to a first operation, displaying a second interface of the first application, wherein during the display of the second interface, an activity corresponding to the first interface is being run, and the first layer is not composited using Flash; in response to a second operation, displaying a third interface of the first application, wherein during the execution of the second operation, the first layer is composited using Flash, and after the second operation, the first layer is not composited using Flash.

[0068] The first operation is used to control the first application to open the second interface. The second operation can be a click or a swipe. When the second operation is a click, it can be used to control the first application to switch from the second interface to the third interface, or return to the first interface. When the second operation is a swipe, it can be used to control the first application to switch from the second interface to the third interface, or return to the first interface, and display an interface switching animation (including a fourth interface) during the swipe. During the swipe to control the interface switching, if the swipe operation in the second operation includes first swiping in the first direction, then holding the swipe and swiping in the second direction, and the first and second directions are opposite, then this is equivalent to canceling the switch. That is, the second operation can be used to first control the first application to switch from the second interface to the first or third interface, and then cancel the interface switch during the interface switching process, ending the switch from the second interface to the first or third interface, and continuing to display the second interface.

[0069] In this embodiment, the first application can be a whitelisted application pre-installed on the electronic device, or it can be any application installed on the electronic device, such as a system-level application or a third-party application. The whitelisted application can be determined based on factors such as high commercialization rate and frequently used applications by users. Typically, whitelisted applications can provide multiple interfaces. This embodiment does not impose specific limitations in this regard.

[0070] In this embodiment, the first interface and the second interface are in a hierarchical relationship. "Hierarchical" means that when a user opens a new interface or returns to an old interface, the new interface and the old interface are respectively defined as the preceding interface and the following interface. Here, "new" and "old," "preceding" and "following" are relative concepts and do not have other limiting meanings, such as not being limited to a parent-child interface relationship. Optionally, the first interface can be the preceding interface of the second interface, specifically, it can be the previous level interface or the previous N levels interface. The number of levels can be related to user operations. For example, if a user achieves the purpose of switching interfaces with only one operation, the first interface can be the previous level interface of the second interface; if an employee needs to perform N operations to achieve the purpose of switching interfaces, then the first interface can be called the previous N levels interface of the second interface.

[0071] In one specific embodiment provided in this application, the first interface can be the main interface of the first application, the first Activity is the Main Activity of the first application, the first interface corresponds to the first Activity, and the first interface also corresponds to the first layer. Correspondingly, the second interface is a non-main interface in the first application, typically a subsequent interface or an N-level interface after the first interface, etc. The second Activity is a non-Main Activity of the first application, the second interface corresponds to the second Activity, and the second interface also corresponds to the second layer. Optionally, in some special cases, after switching from a non-main interface to the main interface or from a non-main interface to another non-main interface, the Activity corresponding to the non-main interface may continue to run, and the layer corresponding to the non-main interface continues to participate in compositing, but the layer corresponding to the non-main interface is an invisible layer, and the non-main interface is not displayed. Therefore, in this special case, the first interface can also be a non-main interface, and correspondingly, the second interface can be the main interface or another non-main interface, etc.

[0072] In this embodiment of the application, the method for predicting whether a user wants to return to the first interface based on user operations can be referred to in the method flow below, and will not be repeated here.

[0073] In this embodiment of the application, the method for further detecting whether to return to the first interface after a certain period of time following the user's operation can be referred to the method flow below, and will not be repeated here.

[0074] The following beneficial effects can be achieved by using the method provided in this application:

[0075] (1) Save resources and reduce power consumption. Specifically, in scenarios where the application interface changes, controlling the layers of the invisible interface to stop participating in compositing reduces the resources occupied by layer compositing, lowers the power consumption of electronic devices, and also improves the running speed of the application.

[0076] (2) Timely and effective display of interface switching animation and the interface after switching. Specifically, after controlling the invisible interface layer to stop participating in the composition, the system will also predict in time that the user may return to continue displaying the aforementioned invisible interface based on the user's operation. Therefore, it will control the layer that continues to be composed of the aforementioned invisible interface in a timely manner, so as to ensure that the switching animation and the interface after switching can be displayed in a timely manner after the user actually returns to the aforementioned invisible interface.

[0077] Next, we will describe in detail the series of user interfaces displayed when the application is running, with reference to the accompanying diagrams.

[0078] Figures 2A-2C are schematic diagrams illustrating the switching from the application's front-end interface to the back-end interface provided in this application.

[0079] Figure 2A illustrates an example scenario where a first application is launched and its first interface is displayed.

[0080] As shown in Figure 2A, an operation on the icon 211 of the first application on the desktop is detected. In response to this operation, the electronic device launches the first application and displays its first interface. The first interface can be the main interface of the first application. Optionally, the interface displayed after launching the first application can also be the interface displayed when the first application was in the foreground state last time it switched from the foreground to the background. This application will use the main interface of the first application as an example for illustration.

[0081] Specifically, during the process of launching the first application and displaying the first interface on the electronic device, the Activity A (also known as the first Activity) corresponding to launching the first interface in Figure 1 described above will be executed. This includes drawing and rendering the elements of the first interface, compositing the first layer corresponding to the first interface, and then displaying the first interface on the screen based on the composited image frame. These tasks will not be elaborated on here.

[0082] Figure 2B illustrates an example scenario where an interface switching animation is displayed in response to the operation of opening a second interface.

[0083] As shown in Figure 2B, an operation (also known as a first operation) is detected on control 221 in the first interface. In response to this operation, the electronic device launches the second interface of the first application and displays an animation of the first interface switching to the second interface. The first interface is the preceding interface of the second interface. For details on the preceding-following relationship between the first and second interfaces, please refer to the previous description; it will not be repeated here.

[0084] During the animation transitioning from the first interface to the second interface on the electronic device, the device specifically performs tasks such as continuing to run Activity A (also known as the first Activity) corresponding to the first interface (as described in Figure 1 above), drawing and rendering the elements of the first interface, compositing the first layer corresponding to the first interface, and launching Activity B (also known as the second Activity) corresponding to the second interface, drawing and rendering the elements of the second interface, and compositing the second layer corresponding to the second interface. In other words, during the layer compositing stage, both the first layer corresponding to the first interface and the second layer corresponding to the second interface participate in the compositing process.

[0085] Figure 2C illustrates an example of a scenario where a second interface is displayed after the interface switching animation stops.

[0086] As shown in Figure 2C, as the interface switching animation plays, the display area of ​​the second interface gradually increases, while the display area of ​​the first interface gradually decreases until it is completely covered by the second interface.

[0087] Without employing the interface processing method provided in this application (i.e., without using the optimized layer compositing mechanism), after the electronic device stops displaying the interface switching animation and displays the second interface, the electronic device will continue to run Activity A corresponding to the first interface, drawing and rendering the elements of the first interface, compositing the first layer corresponding to the first interface, and also running Activity A corresponding to the second interface, drawing and rendering the elements of the second interface, and compositing the second layer corresponding to the second interface. In other words, during the layer compositing stage, both the first layer corresponding to the first interface and the second layer corresponding to the second interface participate in the compositing. This is because the first interface is the key interface provided by the first application. During the operation of the first application, the user may frequently and for extended periods use the first interface. Therefore, after switching from the first interface to the second interface, Activity A corresponding to the first interface remains in the Running state. While Activity A is in the Running state, it typically performs tasks such as drawing and rendering the elements of the first interface and compositing the first layer corresponding to the first interface. This ensures that if the user wants to return to the first interface later, there is no need to restart Activity A corresponding to the first interface, thus accelerating the return response speed.

[0088] When using the interface processing method provided in this application (i.e., employing an optimized layer compositing mechanism), after the electronic device stops displaying the interface switching animation and displays the second interface, the electronic device will continue to run Activity A corresponding to the first interface, but will stop compositing the first layer corresponding to the first interface, and will continue to run Activity B corresponding to the second interface, drawing and rendering the elements of the second interface, and compositing the second layer corresponding to the second interface. In other words, during the layer compositing stage, the first layer corresponding to the first interface does not participate in the compositing. Optionally, the electronic device may continue to draw and render the elements of the first interface, or may also stop drawing and rendering the elements of the first interface. For the specific implementation of stopping the compositing of the first interface's layers in the scenario of displaying the second interface, please refer to the method flow described later, which will not be elaborated here.

[0089] Figures 3A and 3B are schematic diagrams of returning to the previous interface by clicking, as provided in this application.

[0090] Figure 3A exemplarily illustrates a scenario where an interface switching animation is displayed in response to an operation to switch to the first interface.

[0091] As shown in Figure 3A, after the electronic device completes the switch from the first interface to the second interface as described in Figures 2A-2C above and displays the second interface, if the user wants to return to the previous first interface, they can do so by clicking the corresponding control on the second interface. For example, the electronic device can detect a click operation on control 241 in the second interface. In response to this operation, the electronic device exits the second interface and returns to the first interface, displaying an animation of the switch from the second interface to the first interface. The first interface in Figure 3A may be the same as the first interface in Figure 2A, or it may be a new interface obtained by refreshing the first interface in Figure 2A to a certain extent.

[0092] When using the interface processing method provided in this application (i.e., using an optimized layer compositing mechanism), after the electronic device detects a click operation (also known as the second operation) on the control 241 in the second interface, it will stop compositing the first layer corresponding to the first interface and instead resume compositing the first layer corresponding to the first interface. For the specific implementation of the electronic device continuing to compose the layers of the first interface in the scenario where the user exits the second interface and returns to the first interface via a click operation, please refer to the method flow described later; it will not be elaborated here.

[0093] Figure 3B illustrates, for example, the scene where the first interface is displayed after the interface switching animation stops.

[0094] As shown in Figure 3B, as the interface switching animation plays, the display area of ​​the first interface gradually increases, while the display area of ​​the second interface gradually decreases until the first interface is fully displayed and the second interface is no longer displayed.

[0095] Specifically, during the process of the electronic device ending the display interface switching animation and displaying the first interface, tasks such as drawing and rendering the elements of the first interface and compositing the corresponding first layer are performed. However, tasks such as drawing and rendering the elements of the second interface and compositing the corresponding second layer are no longer performed. This is because the second interface is not a critical interface provided by the first application. During the operation of the first application, the user will not frequently or for extended periods use the second interface. Therefore, after switching from the second interface to the first interface, the corresponding Activity B for the second interface is destroyed, thereby clearing the cache and ensuring the running speed of the first application. Simultaneously, while Activity B in the second interface is in the Destroy state, tasks such as drawing and rendering the elements of the second interface and compositing the corresponding second layer are no longer performed.

[0096] Figures 4A and 4B are schematic diagrams of returning to the previous interface via a swipe operation provided in this application.

[0097] Figure 4A exemplarily illustrates a scenario where an interface switching animation is displayed in response to a right swipe operation.

[0098] As shown in Figure 4A, after the electronic device completes the switch from the first interface to the second interface and displays the second interface, if the user wants to return to the previous first interface, they can do so by swiping on the second interface. For example, the electronic device can detect a swipe operation from left to right on the second interface. In response to this operation (also known as the second operation), the electronic device exits the second interface and returns to the first interface, displaying an animation of the switch from the second interface to the first interface. The first interface in Figure 4A may be identical to the first interface in Figure 2A, or it may be a new interface obtained by refreshing the first interface in Figure 2A to a certain extent.

[0099] When using the interface processing method provided in this application (i.e., employing an optimized layer composition mechanism), after the electronic device detects a swipe operation on the second interface, it will continue to composite the first layer corresponding to the first interface, and will also perform tasks such as drawing and rendering elements of the first interface, as well as drawing and rendering elements of the second interface and compositing layers of the second interface. For the specific implementation of the electronic device continuing to composite the first layer corresponding to the first interface in the scenario where the user exits the second interface and returns to the first interface via a right swipe, please refer to the method flow described later; it will not be elaborated here.

[0100] Figure 4B illustrates an example of a scenario where, in response to the end of the rightward swipe and subsequent hand-lifting operation, the interface switching animation stops and the first interface is displayed.

[0101] As shown in Figure 4B, as the user swipes to the right, the display screen plays an interface transition animation. This animation specifically includes: the display area of ​​the first interface gradually increases, while the display area of ​​the corresponding second interface gradually decreases. After the user finishes swiping to the right and lifts their hand, the display area of ​​the second interface gradually decreases until the first interface is completely displayed and the second interface is no longer shown.

[0102] Specifically, during the process of displaying only the first interface after the electronic device finishes its interface switching animation, tasks such as drawing and rendering the elements of the first interface and compositing the corresponding first layer are performed. However, tasks such as drawing and rendering the elements of the second interface and compositing the corresponding second layer are no longer performed. This is because the second interface is not a critical interface provided by the first application. During the operation of the first application, the user will not frequently or for extended periods use the second interface. Therefore, after switching from the second interface to the first interface, the Activity B corresponding to the second interface is destroyed to clear the cache and ensure the running speed of the first application. Simultaneously, while Activity B corresponding to the second interface is in the Destroy state, tasks such as drawing and rendering the elements of the second interface and compositing the corresponding second layer are no longer performed.

[0103] Figures 5A and 5B are schematic diagrams of the return to the previous interface via a reverse sliding operation provided in this application.

[0104] Figure 5A exemplarily illustrates a scenario of interface switching animations displayed in response to a right swipe operation and in response to a left swipe operation without lifting the hand after ending the right swipe.

[0105] As shown in Figure 5A, after the electronic device completes the switch from the first interface to the second interface and displays the second interface, if the user wants to return to the previous first interface, they can do so by swiping on the second interface. Furthermore, if the user wants to cancel the return to the first interface while swiping, they can continue swiping in the opposite direction without lifting their hand to interrupt the return and continue displaying the second interface. For example, the electronic device can detect a swipe to the right from the left side of the second interface. In response to this swipe, the electronic device displays an animation of switching from the second interface to the first interface. If, after detecting a swipe from the left side of the second interface to the right, it also detects a swipe to the left without lifting the hand, the electronic device will interrupt the switch to the first interface and display an animation of returning to the second interface. The first interface in Figure 5A may be the same as the first interface in Figure 2A, or it may be a new interface obtained by refreshing the first interface in Figure 2A.

[0106] Figure 5B illustrates, for example, a scene where the interface switching animation stops and the second interface is displayed in response to the end of the swipe-to-left and hand-lifting operation.

[0107] As shown in Figure 5B, in response to the end of the leftward swipe operation, the interface switching animation played on the display screen includes: the display area of ​​the first interface gradually decreases, and the display area of ​​the corresponding second interface gradually increases until the second interface is fully displayed and the first interface is no longer displayed.

[0108] When using the interface processing method provided in this application (i.e., employing an optimized layer compositing mechanism), after the electronic device returns to the second interface, it will continue running Activity A corresponding to the first interface, but will stop compositing the first layer corresponding to the first interface, and will continue running Activity B corresponding to the second interface, drawing and rendering elements of the second interface, and compositing the second layer corresponding to the second interface. In other words, during the layer compositing stage, the first layer corresponding to the first interface does not participate in the compositing. Optionally, the electronic device may continue drawing and rendering elements of the first interface, or may also stop drawing and rendering elements of the first interface. Regarding the specific implementation of stopping the compositing of the first layer corresponding to the first interface in the scenario of undoing the switch from the second interface to the first interface and returning to the second interface, please refer to the method flow described later, which will not be elaborated here.

[0109] It is understood that Figures 3A-3B and 4A-4B are merely illustrative examples showing that after displaying the second interface of the aforementioned first application, the user can input corresponding click or swipe operations (also referred to as second operations) to switch the display of the first application's first interface. In addition, in other application scenarios, after displaying the second interface of the aforementioned first application, the user can input corresponding click or swipe operations (also referred to as second operations) to switch the first application from the second interface to a third interface. This third interface corresponds to the third layer, and during the switching process, an interface switching animation is displayed. This interface switching animation includes a fourth interface, which includes part of the content of the second interface and part of the content of the third interface. Specifically, during the display of the interface switching animation, SF can synthesize a third image frame based on the first layer, the second layer, and the third layer, and then display the fourth interface based on the third image frame. Regardless of whether the second operation triggers a switch to the first or third interface of the first application, it will trigger the electronic device to implement the interface processing method described in the preceding and following sections of this application. The scenario of switching to the third interface will not be specifically described here; please refer to the corresponding description of switching back to the first interface.

[0110] Next, referring to the method flow shown in Figure 6, we will describe in detail the interface processing method provided in this application. The interface processing method provided in this application includes optimizing the layer composition involved in the interface processing. Optionally, the interface processing method provided in this application may also include optimizing the drawing and rendering involved in the interface processing. The method flow shown in Figure 6 only uses the optimization of the layer composition mechanism as an example for introduction. For the optimization of the drawing and rendering mechanism, please refer to the optimization of the layer composition mechanism.

[0111] As shown in Figure 6, the interface processing method provided in this application includes the following steps:

[0112] Optional S61, which determines whether the application running in the foreground is a whitelisted application.

[0113] Specifically, the electronic device can have a pre-configured whitelist. When a foreground application is switched, the electronic device can determine whether the currently running foreground application is a whitelisted application. If it is a whitelisted application, the interface processing method provided in this application will continue to be used, that is, the method described in S62-S67 will be executed for that application. If it is not a whitelisted application, the interface processing method provided in this application will no longer be used, that is, the method described in S62-S67 will no longer be executed for that application, until the next application switched to the foreground is a whitelisted application, at which point the method described in S62-S67 will be executed for that next application.

[0114] The whitelisted applications can be applications frequently used by the user of the electronic device, or applications currently widely used in commercial applications, etc., and this application embodiment does not impose any restrictions on this. Typically, whitelisted applications can provide multiple interfaces, such as a homepage, a settings page, and other detail pages.

[0115] In this context, foreground applications are those that are running and displaying their interface, and that can respond to user actions. For example, if an electronic device displays the interface of a social media application, and the user can see and interact with the interface, then the social media application is in a foreground running state. Conversely, background applications include those that are running on an electronic device but are not visible to the user. For example, if an electronic device is playing music but displays the interface of a social media application, then the music application is in a background running state. Compared to background applications, foreground applications provide an interface for users to interact with.

[0116] In this embodiment, S61 is an optional step, meaning that the electronic device may not need to have a pre-set whitelist, and the interface processing method provided in this application can be used for all applications installed on the electronic device. This embodiment does not limit this.

[0117] After determining in S61 that the foreground application belongs to the whitelist, the electronic device enables an optimized layer composition mechanism for that application. In one possible implementation, this optimized layer composition mechanism includes: the electronic device periodically executing subsequent steps S65-S67, i.e., determining whether the first layer of the application meets the optimization conditions (i.e., whether the first layer participating in composition is invisible). If so, the first layer is set not to participate in composition; otherwise, it is set to participate in composition. This simple method of periodic detection controls the invisible first layer to stop participating in composition, reducing the resources occupied by invisible layer composition, lowering the power consumption of the electronic device, and improving application running speed. In another possible implementation, the optimized layer composition mechanism includes: first executing S62-S64, i.e., predicting whether the user wants to switch application interfaces based on user actions; when this is predicted, promptly setting the first layer to participate in composition; then executing S65-S67, i.e., further determining whether the first layer of the application meets the optimization conditions. In this way, the invisible first layer is prevented from participating in the composition, and the user's actions are used to predict whether the user may return to continue displaying the first interface. Therefore, the first layer corresponding to the first interface is controlled to continue compositing in time, so as to ensure that the switching animation and the first interface after the return to the first interface are actually triggered can be displayed in time.

[0118] In this embodiment, an invisible layer refers to a layer that is displayed at the bottom during the layer compositing process. Furthermore, the lower layer is covered by the upper layer, and the upper layer does not include transparent portions. Taking the first layer corresponding to the first interface and the second layer corresponding to the second interface as an example, in response to an operation to switch from the first interface to the second interface, when a composite image frame is created based on the first and second layers for display, the first layer is below the second layer, meaning the first layer is covered by the second layer. In this case, the first layer is invisible, and the second layer is visible.

[0119] In this embodiment, an invisible interface refers to an interface that cannot be seen by the user during the display process. Typically, the layers corresponding to an invisible interface are all in an invisible state. Taking the correspondence between a first interface and a second interface as an example, in response to an operation to switch from the first interface to the second interface, the electronic device displays the second interface instead of the first interface. In this case, the first interface is in an invisible state, and the second interface is in a visible state.

[0120] Option S62 can be used to obtain the operation event. If the operation event is a press event, then execute S63-1; if the event is a release event, then execute S63-2.

[0121] Specifically, after determining in S61 that the application running in the foreground belongs to the whitelist, since the user can input interactive operations such as clicking, long pressing, and swiping (long swipe, short swipe) when the electronic device displays the interface of the foreground application, in order to control the application to switch interfaces or perform other tasks, the electronic device can obtain the user's interaction events with the interface. This allows it to predict whether the user wants to open a lower-level interface (e.g., switching from the first interface to the second interface as described in Figures 2A-2B above), and whether the user wants to return to the previous interface (e.g., switching from the second interface back to the first interface as described in Figures 3A-3B and 4A-4B above, and interrupting the switch from the second interface back to the first interface and continuing to display the second interface as described in Figures 5A-5B above).

[0122] Analysis of common operations such as tapping, long pressing, and swiping reveals that these operations all involve two actions: pressing down and releasing (touch up). Furthermore, touch down and touch up each have the following characteristics:

[0123] (1) Touch down may trigger the application to switch interfaces.

[0124] In one possible implementation, a touch down might be an operation encompassed by a click on a control used to switch interfaces within the current screen. Therefore, immediately following a touch up after such a touch down, the application might display the switched interface, which could be a newly opened sub-interface or a return to a previously opened previous interface. Optionally, this touch down might be encompassed by a click on control 221 in Figure 2B, as described earlier; or, the touch down might also be encompassed by a click on control 241 in Figure 3A, as described earlier.

[0125] In another possible implementation, the touch-down action might be an operation included in the swipe gesture used to switch interfaces within the current screen. Therefore, after such a touch-down followed by a touch-up, the application will display an animation of the interface switching during the swipe, and then after the touch-up (equivalent to ending the swipe), the application will display the switched interface. Optionally, this touch-down action might be the action included in the swipe gestures described above in Figures 4A and 5A.

[0126] Regardless of whether it's a click or swipe operation, if the user intends to switch interfaces, the duration of the touch-down action is typically between 10 and 80 milliseconds (ms). That is, when the user's touch-down duration reaches 10-80 ms, the user will usually touch up to trigger the application to display the switched interface. Conversely, if the user's touch-down duration exceeds 80 ms, the user is highly likely to continue to undo the interface switching action by swiping backwards, or the user may simply be swiping upwards to view a long article. Optionally, this application refers to a swipe operation with a touch-down duration of the first duration (e.g., 80 ms) as a long swipe operation, and a swipe operation with a touch-down duration shorter than the first duration (e.g., 80 ms) as a short swipe operation. Generally, short swipe operations are more likely to trigger an application interface switch, while long swipe operations are more likely to trigger the application to scroll through a long article or undo the interface switch.

[0127] (2) The application switching interface may not be triggered after touch up.

[0128] Similar to the analysis of the aforementioned touch-down scenario, taking click and swipe operations as examples, after a touch-up occurs within the click or swipe operation, an application switching interface may or may not be triggered. In the scenarios described in Figures 2A-2C, 3A-3B, and 4A-4B, a touch-up triggers an application switching interface; however, in the scenario described in Figures 5A-5B, a touch-up does not trigger an application switching interface.

[0129] Furthermore, after a touch-up, the device may directly display the new screen, or it may first display a screen transition animation and then display the new screen after the animation ends. Alternatively, it may display both a screen transition animation and a screen reversal animation, and then display the un-transformed screen after the reversal animation ends. Regardless of whether a screen transition animation is displayed, the electronic device will typically display the fixed application screen steadily within 1-3 seconds after the touch-up.

[0130] Based on the foregoing analysis, receiving a touch-down event may trigger an application switch from the second screen to the first screen. The first layer corresponding to the switched first screen may have previously been set not to participate in the composition. To ensure effective playback of the screen transition animation and the switched first screen, it is necessary to promptly enable the first layer corresponding to the first screen to participate in the composition. Therefore, upon receiving a touch-down event, the first layer can be promptly set to participate in the composition. Furthermore, receiving a touch-down event may not trigger an application screen transition. Therefore, after the first duration of the touch-down event, further layer detection is needed to determine whether any invisible screen is being covered. Additionally, after the second duration of the touch-up event, the application typically displays a stable, fixed screen. Therefore, layer detection can continue to accurately determine whether any invisible screen is being covered.

[0131] Therefore, this application can use the method of obtaining touch down event or touch up event to execute different subsequent strategies respectively. That is, if touch down event is obtained, the electronic device starts to execute subsequent S63-1 to S67, and if touch up event is obtained, the electronic device starts to execute subsequent S63-2 to S67.

[0132] S63-1, if the first layer is set to stop participating in compositing, then set the first layer to participate in compositing.

[0133] Specifically, based on the analysis of the touch down event in S62 above, it can be seen that after a touch down event is received, a switch from the second interface to the first interface may be triggered. Furthermore, the first layer corresponding to the switched first interface may have been previously set not to participate in the composition. Therefore, in order to ensure that the interface switching animation and the switched first interface are displayed promptly after responding to the user's operation, the first layer should be set to participate in the composition in a timely manner, thereby compositing the first layer promptly.

[0134] In this embodiment, step S63-1 is optional. The electronic device may skip step S63-1 and instead execute steps S65-S67 directly after step S62. This means that the electronic device does not need to set the first layer to participate in the composition after receiving the touch-down event. Instead, after the touch-down event, it directly performs the layer detection described in steps S65-S67 on the foreground application and then determines whether to exclude the first layer from the composition based on the detection results.

[0135] Option S64 can be used to determine when the press event has reached the first duration.

[0136] Specifically, determining that a touch-down event has reached its first duration includes: after receiving a touch-down event, starting a timer for the first duration; if a release event is received during the timer's countdown for the first duration, interrupting the timer's countdown and restoring it to its initial countdown state; if no release event is received during the timer's countdown for the first duration, determining that the touch-down event has reached its first duration until the timer has reached its first duration.

[0137] Based on the analysis results of the press event in S62 above, it can also be seen that even after obtaining the press event, the interface switching may not be triggered. For example, if the press duration reaches a first duration (which is, for example, greater than or equal to the aforementioned 80ms duration), it may indicate that the user's intention is not to switch interfaces. Therefore, the electronic device needs to continue executing the subsequent S65-S67, that is, to further detect whether the first layer meets the optimization conditions (i.e., whether the first layer participating in the composition is invisible), and then accurately determine whether to set the first layer not to participate in the composition, thereby improving the accuracy of the judgment and reducing the power consumption of the electronic device.

[0138] Optionally, scenarios where the press event reaches the first duration can include the scenario described in Figures 5A-5B above, where the interface is switched by swiping right and then swiping left in reverse to undo the interface switch.

[0139] In this embodiment, step S64 is optional. The electronic device may skip step S64 and instead execute steps S65-S67 directly after step S63-1. This means the electronic device does not need to determine whether the duration of the press event has reached the first duration. Instead, after acquiring the press event, it directly sets the first layer to participate in the composition and directly performs the layer detection described in steps S65-S67 on the foreground application. Then, based on the detection results, it determines whether to set the first layer not to participate in the composition, thus reducing the power consumption of the electronic device.

[0140] Option S63-2 can be used to determine when the lift-up event reaches the second duration.

[0141] Specifically, determining that the touch-up event has reached the second duration includes: after acquiring the touch-up event, starting a timer for the second duration until the timer reaches the second duration, then determining that the touch-up event has reached the second duration, and then executing step S65.

[0142] Specifically, based on the analysis in S62 above, the screen switching might not be triggered after the lift event is received. Regardless of whether the lift triggers screen switching, the electronic device may be in the process of displaying a screen switching animation during the second duration after the lift (this second duration is, for example, greater than or equal to the aforementioned 3 seconds). After this second duration, a fixed screen is usually displayed, which may be the switched screen or the current screen. Therefore, to avoid misjudgment due to inaccurate layer detection results during the screen switching animation, S65-S67 can be executed after waiting for the second duration and after the screen is stably displayed. That is, further determine whether the detected layers meet the optimization conditions (i.e., whether the first layer participating in the composition is invisible), and then precisely set the first layer not to participate in the composition, thereby reducing the power consumption of the electronic device.

[0143] Optionally, if the event obtained when executing S62 is a lift event, S65 can be executed directly without executing S63-2. That is, there is no need to wait for a second duration after the lift event before checking whether the first layer is covered. Instead, the check for whether the first layer is covered can be started directly after the lift operation. This can avoid the abnormal problem that the first application needs to switch back to the first interface while waiting for the second duration, but the first interface cannot be successfully displayed because the first layer is set not to participate in the composition.

[0144] In this embodiment, the electronic device executes the aforementioned S62 to S63-1 to predict whether the user wants to switch the application interface. Then, after predicting that the user wants to switch the application interface, it promptly controls the layers that were not previously involved in the compositing to continue to participate in the compositing, so as to avoid problems such as black screen and frame drops when playing the interface switching animation or even the interface after the switch.

[0145] In this embodiment, the electronic device executes the aforementioned S64 and S63-2 to determine whether the displayed application interface has reached a temporarily stable state after the user operation, that is, whether it has reached the state of ending the interface switching animation and displaying the fixed interface before / after the switch. Then, in this stable state, the layer detection results are used to accurately identify whether there is a covered layer, thereby controlling the covered layer not to participate in the synthesis. This improves the accuracy of the detection results and avoids the electronic device from synthesizing unnecessary layers, thus saving the power consumption of the electronic device.

[0146] S65 determines whether the total number of layers of the application running in the foreground is greater than the preset number of layers.

[0147] Specifically, electronic devices can preset the number of layers corresponding to each application in the whitelist. The preset number of layers is usually the number of layers corresponding to each application in a baseline scenario. For example, the baseline scenario includes: a scenario where only the main interface of the application is displayed, excluding system-level interface elements; that is, a scenario where only the main Activity corresponding to the main interface of the application is run, and only the layer corresponding to the main interface is composed.

[0148] Optionally, taking the first application mentioned above as an example, the baseline scenario can be the scenario of only running Activity A in Figure 1, or the scenario of displaying the first interface in Figure 2A. Based on the previous description of Figures 1, 2A-2B, during the process of displaying the main interface of the first application, if the user opens the second interface of the first application, the application will add a layer of the second interface, and the layers of the first interface and the second interface will participate in the composition. Therefore, by determining whether the total number of layers of the current foreground application is greater than the number of layers in the baseline scenario (the first interface scenario), it can be determined whether the electronic device has opened a new interface, and then whether any layers are covered (i.e., whether the old interface is not visible). If so, the covered layers are controlled not to participate in the composition.

[0149] Typically, the number of layers corresponding to an interface is greater than or equal to one. An interface can be obtained by compositing multiple layers in a predetermined order and along the same direction to form an image frame, such as a background layer, an information layer, and a control layer. Therefore, this application embodiment does not impose a specific limit on the number of layers in the baseline scene.

[0150] S66, determine whether the first layer is covered.

[0151] Specifically, taking the first application as an example, after determining in S65 that the total number of layers in the foreground application is greater than the preset total number of layers, it is necessary to further determine whether the first layer is covered. If it is determined that the first layer is covered, it means that the first layer is invisible. The first interface corresponding to the first layer is actually an invisible interface, that is, the user cannot see the first interface. Therefore, the first layer does not need to continue to participate in the composition, and the subsequent S67 is executed.

[0152] Optionally, in one possible implementation, once it is determined that the first layer is covered, it is equivalent to the first layer being an invisible layer. In another possible implementation, after determining that the first layer is covered, it is necessary to further determine whether other layers covering the first layer (e.g., the second layer) include portions with transparency. Only when it is determined that other layers covering the first layer do not include transparent portions is the first layer determined to be an invisible layer.

[0153] One method to determine whether the first layer is covered includes: first, obtaining the layers above the first layer (e.g., the second layer above the first layer), and then comparing the coordinates of the first and second layers. Usually, it is only necessary to obtain the coordinates of the two opposite corners of the first layer and the two opposite corners of the second layer, and compare the two opposite corner coordinates of the two layers respectively. For example, if the horizontal and vertical values ​​of the top left corner coordinate of the second layer are both greater than the horizontal and vertical values ​​of the top left corner coordinate of the first layer, and the horizontal and vertical values ​​of the bottom right corner coordinate of the second layer are both greater than the horizontal and vertical values ​​of the bottom right corner coordinate of the first layer, then it is determined that the first layer is covered by the second layer.

[0154] Optionally, in the scenarios shown in Figures 2C and 5B described above, after executing S66, the electronic device can determine that the first layer corresponding to the first interface is covered. In the scenarios shown in Figures 3B and 4B described above, after executing S66, the electronic device can determine that the first layer corresponding to the first interface is not covered.

[0155] S67, set the first layer to not participate in compositing.

[0156] Specifically, once the electronic device determines in S66 that the first layer is covered, it can set the first layer to not participate in compositing. This prevents the first layer from being subsequently composited in SF. Setting the first layer to not participate in compositing can be achieved by setting its visibility attribute to invisible. Conversely, setting the first layer to participate in compositing can be achieved by setting its visibility attribute to visible.

[0157] Optionally, in the scenarios shown in Figures 2C and 5B above, after executing S67, the electronic device stops compositing the layer corresponding to the first interface. However, in the scenarios shown in Figures 3B and 4B above, after executing S67, the electronic device can compose the layer corresponding to the first interface.

[0158] The following section will introduce the OS interaction involved in the interface processing method provided in this application using specific application scenarios.

[0159] Figure 7 shows the OS interaction process for switching whitelisted applications to the foreground provided in this application.

[0160] In this embodiment of the application, the OS interaction process of switching the whitelist application to the foreground as shown in Figure 7 described below corresponds to a specific implementation of S61 in Figure 6 above.

[0161] As shown in Figure 7, in the scenario where a whitelisted application switches to the foreground, the interface processing method provided in this application involves modules including, but not limited to: the launcher and first application in the application (APP) layer, AMS, View, and SF in the framework layer, as well as the adaptive power saving (APS) module and the advanced graphics platform (AGP). The interaction between these modules includes the following steps:

[0162] S71, the launcher in the APP layer notifies AMS to start the first application.

[0163] Specifically, when a user inputs an action to launch the first application, the Launcher in the electronic device can send an event to AMS via an intent to notify AMS to launch the first application.

[0164] Optionally, the operation for launching the first application is, for example, the operation on the icon 211 of the first application shown in Figure 2A above. In this embodiment of the application, the first application may also be referred to as the first application.

[0165] S72, AMS in the Framework layer starts the process of the first application.

[0166] As AMS is a core service in electronic device systems, responsible for important tasks such as application process management, the application startup process needs to be coordinated and managed by AMS to ensure that the application can be loaded and run. AMS startup of the first application includes creating the first application's process, obtaining information about the first application from the Package Manager Service (PMS), and then executing the start process to launch the first application's process.

[0167] In S73, the AMS in the Framework layer reports the first application foreground running events to the APS through stub functions.

[0168] Specifically, APS can pre-mark points in AMS using stub functions (such as APP_RUN_FRONT), which allows AMS to send the foreground running event of the first application to APS when the first application is launched.

[0169] Optionally, APS can also use methods other than stub functions to obtain the event of the application switching to the foreground. For example, by registering an application switching to the foreground detection service in AMS, after successful registration, AMS can notify APS to switch the application to the foreground when it detects that the application has started.

[0170] Optionally, the execution order of S73 can be that during the execution of S72, the stub function can be pre-stamped in the process of AMS starting the first application. When AMS executes the process of starting the first application, it can trigger AMS to report the foreground running event of the first application to APS through the stub function.

[0171] S74, the first application launches the first activity.

[0172] Specifically, after the process of the first application is started, the first application can start its first activity by executing the onStart() method. This first activity is the main activity of the first application. After the first activity is started, AMS can manage the lifecycle of the first activity to implement the subsequent display of the first screen (also known as the main screen).

[0173] Optionally, the main activity of the first application launched by AMS can be Activity A, which launches the first application as described in the UI embodiment above. Correspondingly, the first interface can be the first interface described in the UI embodiment above.

[0174] S75, SF synthesizes the first image frame based on the first layer.

[0175] Specifically, the first application adds the drawn first layer to the buffer, SF reads the first layer from the buffer, and then synthesizes the first image frame (the image frame can also be called image data) based on the first layer.

[0176] S76, SF sends the synthesized first image frame to the display driver to display the first interface.

[0177] Specifically, after SF synthesizes the first image frame based on the first layer, the synthesized first image frame can be passed to the display driver through the hardware composer (HWC). The display driver then controls the display screen to display the first interface based on the first image frame. This display screen can be a display screen installed in an electronic device, or a display device independent of the electronic device.

[0178] S77, APS determines whether the first application is a whitelisted application. If so, it sets the whitelist flag of the first application to true.

[0179] Specifically, the APS has a pre-defined whitelist of applications. Applications on the whitelist can utilize the layer optimization mechanism provided in this application. Therefore, after the APS receives information that the first application is running in the foreground, it can determine whether the first application is a whitelisted application. If so, the whitelist flag of the first application is set to true. The whitelist flag being true indicates that the layer optimization mechanism can be applied to the first application subsequently.

[0180] Optionally, the layer optimization mechanism includes: performing layer detection on the application to determine whether the application meets the optimization conditions; if so, the application is placed under control. Optionally, entering control includes controlling the invisible layers of the application to stop participating in compositing; for details, please refer to the description of the scenario shown in Figure 8 later.

[0181] Optionally, the layer optimization mechanism also includes: performing operation detection on the application, and controlling the application to exit management upon detecting a touch-down operation. Optionally, exiting management includes controlling layers in the application that have stopped compositing to continue participating in compositing; for details, please refer to the description of the scenario shown in Figure 9 later.

[0182] S78, APS records information about the first layer (layer number, layer coordinates).

[0183] Specifically, after APS sets the whitelist flag of the first application to true, it can record the information of the first layer of that first application, which can then be used for subsequent layer detection based on this information. The first layer information refers to the number of layers and layer coordinates of the whitelisted application in the baseline scene.

[0184] Figure 8 shows the OS interaction flow for triggering whitelisted applications to enter control provided in this application.

[0185] In this embodiment of the application, the OS interaction process for whitelisted applications entering control shown in Figure 8 described below corresponds to a specific implementation of S63-2 and S65-S67 in Figure 6 above.

[0186] As shown in Figure 8, after determining that the whitelisted application has switched to the foreground, the interface processing method provided in this application further includes obtaining the user's operation event on the foreground application and setting the whitelisted application to enter a controlled state based on the operation event. The OS interaction involved in this controlled state scenario includes the following steps:

[0187] S801, ViewRootImpl in the Framework layer receives the touch up event.

[0188] Specifically, when a user inputs a first action to open the second interface of the first application, the input can obtain the lift-up event corresponding to the first action, and then notify the ViewRootImpl in the Framework layer of the lift-up event.

[0189] ViewRootImpl is the top-level class responsible for managing the view hierarchy. As the root node (parent class) of all views, ViewRootImpl periodically traverses the entire view tree and performs necessary update operations to maintain the real-time performance and responsiveness of the user interface. ViewRootImpl is also responsible for distributing click, long press, touch, and swipe operations to the corresponding view objects for processing. Therefore, after detecting the first operation, the parent class's process is processed first (such as reporting events to the APS through stub functions as described in S802-S803), and then the application's view class process is processed (such as sending the first operation to the first application as described in S804).

[0190] Optionally, the release event in S801 corresponds to the release event obtained when S62 is executed in Figure 6 above. Furthermore, the release event in S801 can be, for example, the event corresponding to the operation performed on control 221 as shown in Figure 2B above. Specifically, the events corresponding to the operation performed on control 221 as shown in Figure 2B above include, but are not limited to: press events, release events, and the time corresponding to the press operation and the time corresponding to the release operation.

[0191] In S802, ViewRootImpl in the Framework layer reports the lift event to APS through the stub function.

[0192] Specifically, APS can pre-mark points in ViewRootImpl using stub functions (such as EVENT_TOUCHEVENT_UPLOAD), so that when the View receives a lift event, it sends the lift event to APS.

[0193] S803, the View in the Framework layer sends a lift event to the first application.

[0194] Specifically, after processing the aforementioned process of the ViewRootImpl parent class, ViewRootImpl can also dispatch the first operation to the corresponding View, so that the View sends the lift event corresponding to the first operation to the first application.

[0195] Optionally, if the first operation is, for example, the operation performed on control 221 as shown in Figure 2B above, then ViewRootImpl will notify the View corresponding to control 221 of the first operation, and then the View will send the first operation to the first application.

[0196] S804, the first application launches the second activity.

[0197] Specifically, after the first application receives a raise-to-open event, it analyzes the event and determines that it corresponds to opening a second interface. The first application can then call `StartActivity()` to launch the second activity. This second activity corresponds to the second interface of the first application.

[0198] S805, AMS creates a second activity.

[0199] The second activity is the activity corresponding to the second interface.

[0200] Specifically, AMS can use the onCreate() method to create a second activity and then manage the lifecycle of that second activity.

[0201] S806, SF synthesizes the second image frame based on the second layer.

[0202] The second layer is the layer corresponding to the second interface.

[0203] Specifically, the first application adds the drawn second layer to the buffer, SF reads the second layer from the buffer, and then synthesizes the second image frame (the image frame can also be called image data) based on the second layer.

[0204] S807, SF sends the synthesized second image frame to the display driver to display the second interface.

[0205] Specifically, after SF synthesizes the second image frame based on the second layer, the synthesized second image frame can be passed to the display driver through HWC, and the display driver controls the display screen to display the second interface based on the second image frame.

[0206] Optionally, the second interface is as described above, as shown in Figure 2B.

[0207] Following S802, the electronic device also executes S808-S813.

[0208] S808, APS obtains information about the first application layer from SF.

[0209] Specifically, after receiving a lift event, APS can obtain information about the current layer of the first application from SF, including but not limited to the number and coordinates of the current layer.

[0210] S809, APS determines whether the current number of layers in the first application is greater than the preset number of layers.

[0211] Specifically, after obtaining the information of the current layer of the first application, APS will determine whether the current layer number is greater than the preset layer number. If the current layer number is greater than the preset layer number, then S810 will continue to be executed.

[0212] The preset layer information is the same as the layer information in the baseline scene introduced earlier. For more information on the preset layer information, please refer to the introduction of S65 and S78 earlier, which will not be repeated here.

[0213] Taking the scenarios shown in Figures 2A-2C as an example, the preset layer is the first layer corresponding to the first interface, and other layers besides the preset layer are, for example, the second layer corresponding to the second interface.

[0214] Optionally, after receiving the lift-up operation event, APS can wait for a second duration before performing layer number detection.

[0215] S810, APS determines whether the first layer is covered.

[0216] Specifically, when S809 determines that the number of layers in the first application is greater than the preset number of layers, it is equivalent to adding a layer on top of the first layer, such as adding a second layer. Therefore, APS can determine whether the first layer is covered by the second layer based on one or more of the following parameters: layer coordinates, layer opacity, etc. For details on this determination method, please refer to the description of S66 above; it will not be repeated here.

[0217] Taking the scenario shown in Figures 2A-2C as an example, the touch up event is the hand-raising event included in the operation on control 221 shown in Figure 2B. After the touch up event, the electronic device will play an animation of the first application switching from the first interface to the second interface. Usually, the playback time of this animation will not exceed 3 seconds. Therefore, when APS executes S810, the electronic device has finished playing the interface switching animation and fully displayed the second interface as shown in Figure 2C. After the electronic device fully displays the second interface as shown in Figure 2C, the electronic device will detect that the newly added layer (the second layer corresponding to the second interface) completely covers the base layer (the first layer corresponding to the first interface).

[0218] S811, if the first layer is covered, set the control flag of the first application to true.

[0219] Specifically, the control flag is used to indicate whether the layer compositing optimization mechanism is used for the corresponding application. The value of the control flag can be true, false, or 1, 0. Among them, true or 1 can be used to indicate that the optimized layer compositing mechanism is used for the corresponding application (that is, the first layer is set to stop participating in compositing); false or 0 can be used to indicate that the optimized layer compositing mechanism is not used for the corresponding application (that is, the first layer is set to participate in compositing).

[0220] S812, APS sends information to SF via AGP that the control flag of the first application is set to true, as well as information about the first layer.

[0221] Specifically, after setting the control flag of the first application to true, the AGP can be notified that the control flag of the first application is set to true. The APS also sends the information of the first layer to the AGP. This causes the AGP to notify the SF that the control flag of the first application is set to true, and the first layer is set not to participate in compositing.

[0222] Optionally, after setting the control flag of the first application to 1, the AGP can be notified of the information that the control flag of the first application is set to 1 and the identification information of the first layer, etc.

[0223] S813, SF stops compositing the first layer.

[0224] Once SF receives the information sent by AGP, it stops compositing the first layer.

[0225] Optionally, taking the scenarios shown in Figures 2A-2C above as an example, the first layer that SF stops compositing can be the layer corresponding to the first interface after the first interface is fully displayed as shown in Figure 2C.

[0226] This application does not impose specific restrictions on the order in which S806-S808 and S809-S813 are executed. Generally, S813 is executed later than S808.

[0227] Figure 9 shows the OS interaction flow for triggering the exit control of whitelisted applications provided in this application.

[0228] In the embodiments of this application, the OS interaction process for whitelist application exit control shown in Figure 9 described below corresponds to a specific implementation of S63-1, S64-S67 in Figure 6 above.

[0229] As shown in Figure 9, after a whitelisted application switches to the foreground, the interface processing method provided in this application further includes detecting the user's operation on the foreground application and setting the whitelisted application to exit the control state based on the operation. The OS interaction involved in this exit control state scenario includes the following steps:

[0230] S901, ViewRootImpl in the Framework layer receives the touch down event.

[0231] Specifically, when the user inputs a second operation to return to the first interface of the first application, the input can obtain the press event corresponding to the second operation, and then notify the ViewRootImpl in the Framework layer of the press event. In addition, the input can obtain the release event corresponding to the second operation, and then notify the ViewRootImpl in the Framework layer of the press event, as described in S911-S912 below, which will not be elaborated here.

[0232] ViewRootImpl is the top-level class responsible for managing the view hierarchy. As the root node (parent class) of all views, ViewRootImpl periodically traverses the entire view tree and performs necessary update operations to maintain the real-time performance and responsiveness of the user interface. ViewRootImpl is also responsible for distributing click, long press, touch, and swipe operations to the corresponding view objects for processing. Therefore, after detecting a second operation, the parent class's process is processed first (e.g., reporting events to the APS via stub functions as described in S902), and then the application's view class process is processed (e.g., sending the press and release events corresponding to the second operation to the first application as described in S905).

[0233] Optionally, ViewRootImpl can also obtain the time of the press event corresponding to the second operation, as well as the time of the release event, etc. Here, we will only take the scenario of the press event triggering the first application exit management as an example. For details about the release event, please refer to the description in S911-S912 below, which will not be elaborated here.

[0234] Optionally, the second operation may be, for example, a click operation on control 241 as shown in Figure 3A above, or an operation on the second interface as shown in Figure 4A above, involving pressing, sliding to the right, and lifting the hand, or an operation on the second interface as shown in Figure 5A above, involving pressing, sliding to the right, not lifting the hand after sliding to the right, and then sliding to the left and lifting the hand.

[0235] S902, ViewRootImpl in the Framework layer reports the press event to APS through the stub function.

[0236] Specifically, the APS can pre-mark points in the View using stub functions (such as EVENT_TOUCHEVENT_UPLOAD), so that when the View receives a press event, it sends the press event to the APS.

[0237] S903, if the control flag of the first application is true, set the control flag of the first application to false, and also notify the View to delay reporting the event corresponding to the second operation to the first application.

[0238] Specifically, after receiving the press event corresponding to the second operation, the first application may switch from the second interface to the first interface. If the first application was previously in a controlled state, the first layer corresponding to the first interface of the first application would be set not to participate in compositing. This could lead to abnormal display issues such as black blocks / white blocks / disappearances during the interface switch due to the first layer not being composed in time. Therefore, after the press event, the control flag of the first application needs to be set to false, so that the first application exits control and the first layer is set to participate in compositing. In addition, a delayed notification of the event corresponding to the second operation to the first application can be sent to the View in the Framework layer. This can avoid the aforementioned abnormal display issues.

[0239] S904, APS sends information via AGP to SF that the control flag of the first application is set to true, as well as information about the first layer.

[0240] After S903, the electronic device executes S905-S909 in addition to S904. This application does not restrict the execution order of S905-S909 and the aforementioned S904. However, if the View reports the event corresponding to the second operation to the application after a third delay, then S909 is usually later than S906, that is, the first layer is synthesized by SF before the application opens the first interface.

[0241] S905, the View in the Framework layer sends the event corresponding to the second operation to the first application after a third delay.

[0242] Specifically, after processing the aforementioned process of the ViewRootImpl parent class, ViewRootImpl can also dispatch the event corresponding to the second operation to the corresponding View, so that the View sends the event corresponding to the second operation to the first application. Optionally, after the View receives the APS delayed reporting notification, the View can delay for a third period of time before sending the event corresponding to the second operation to the first application. This third period of time can be set to, for example, 1-8ms. Typically, within 1-8ms, the electronic device has already completed the exit control process described in S904 above. This can avoid the problem of black blocks / white blocks / disappearances, etc., appearing when the application wants to return to the first screen because the first layer of the first screen failed to participate in the composition in time due to the previous control.

[0243] Optionally, if the first operation is, for example, the operation performed on control 221 as shown in Figure 2B above, then ViewRootImpl will notify the View corresponding to control 221 of the first operation, and then the View will send the first operation to the first application.

[0244] S906, the first application resumes the first activity.

[0245] Specifically, after the first application receives the event corresponding to the second operation, it analyzes the event that corresponds to the second operation and returns to the first interface. Then, the first application can call the onResume() method to make AMS resume the first activity.

[0246] Optionally, when the second operation is the operation on control 241 shown in Figure 3A above, or the right swipe operation on the second interface shown in Figure 4A above, or the right swipe operation on the second interface shown in Figure 5A above, the relevant information of the second operation includes, but is not limited to: touch down, touch up, and the time corresponding to touch down and touch up.

[0247] S907, AMS destruction second activity.

[0248] Specifically, AMS can call the onDestroy() method to destroy the second activity. This is because the second interface corresponding to the second activity is not the main interface of the first application, and users usually do not frequently switch to this second interface for extended periods. Therefore, after obtaining the operation used to switch from the second interface to the first interface, it is usually necessary to destroy the second activity corresponding to the second interface.

[0249] Optionally, in other embodiments, AMS may not destroy the second activity, but instead switch the first activity to the focus activity and continue running. This application does not limit this.

[0250] S908, SF synthesizes the first image frame based on the first layer.

[0251] Specifically, the first application adds the drawn first layer to the buffer, SF reads the first layer from the buffer, and then synthesizes the first image frame (the image frame can also be called image data) based on the first layer.

[0252] S909, SF sends the synthesized first image frame to the display driver to display the first interface. SF first layer sends the synthesized first layer to the display driver to control the display screen to show the first interface.

[0253] Specifically, after SF synthesizes the first image frame based on the first layer, the synthesized first image frame can be passed to the display driver through HWC, and the display driver controls the display screen to display the first interface based on the first image frame.

[0254] Optionally, taking the scenario shown in Figures 3A-3B as an example, the first interface displayed in the first image frame includes the incomplete first interface during the interface switching animation.

[0255] Taking the scenarios shown in Figures 4A-4B and 5A-5B as examples, the first interface displayed in the first image frame includes the incomplete first interface during the interface switching animation.

[0256] S910 determines that the first duration has been reached after the press event, and executes a method similar to the aforementioned S809-S813.

[0257] Specifically, after S902, APS starts a timer for the first duration. After the timer reaches the first duration, it executes a method similar to that described in S809-S813, which will not be elaborated here.

[0258] Optionally, taking the scenarios shown in Figures 5A-5B as an example, the second operation is that after pressing down, the user first swipes right, then doesn't lift their hand after swiping right, and then continues to swipe left and lift their hand. That is, the touch-down duration in this second operation is relatively long, usually reaching the duration of the first operation. Therefore, in this scenario, after the APS executes a method similar to S809-S813, because the electronic device has not actually switched to fully displaying the first interface (it has not successfully switched back to the first interface), the layer detection result is that the second layer covers the first layer. Therefore, it will re-enter control, i.e., execute a method similar to S809-S813.

[0259] S911, the View in the Framework layer receives the lift event.

[0260] Specifically, during the execution of S901, the View in the Framework layer can obtain not only the press event corresponding to the second operation, but also the release event. The times corresponding to the release events are all later than the times corresponding to the press events. Optionally, after the View obtains the release event, it will end the first duration of the timer involved in S911, which is equivalent to resetting the timer of the first duration to its initial state.

[0261] Optionally, the lift event may be triggered by an operation on control 241 as shown in Figure 3A above, or by a right swipe operation on the second interface as shown in Figure 4A above, or by a right swipe operation on the second interface as shown in Figure 5A above.

[0262] In S912, ViewRootImpl reports the lift event to APS through the stub function.

[0263] Specifically, APS can pre-mark points in ViewRootImpl using stub functions (such as EVENT_TOUCHEVENT_UPLOAD), so that when ViewRootImpl detects a lift event, it sends the lift event to APS.

[0264] S913, after determining that the second duration has been reached after the lift event, execute a method similar to the aforementioned S809-S813.

[0265] Specifically, after S912, APS starts a timer for the second duration. After the timer reaches the second duration, it executes a method similar to that described in S809-S813, which will not be elaborated here.

[0266] Optionally, taking the scenarios shown in Figures 3A-3B and 4A-4B as examples, the second operation will trigger a touch-up. Therefore, in these scenarios, after the APS executes a method similar to S811-S813, since the electronic device switches to the full display of the first interface (successfully switching back to the first interface), the layer detection result is that the second layer does not cover the first layer. Therefore, the exit control will remain in place, that is, the method similar to S809-S813 will not continue to be executed.

[0267] Optionally, taking the scenarios shown in Figures 5A-5B as an example, the second operation will trigger a touch-up. Therefore, in these scenarios, after the APS executes a method similar to S811-S813, since the electronic device has not actually switched to fully displaying the first interface (failed to switch back to the first interface), the layer detection result is that the second layer covers the first layer. Therefore, it will re-enter the control process, that is, execute a method similar to S809-S813.

[0268] Based on the preceding introduction to the camera application launch method provided in this application, the form and hardware / software architecture of the electronic device involved in this method will be described next.

[0269] Electronic devices can be equipped with Or other portable terminal devices with different operating systems, such as mobile phones, tablets, desktop computers, laptops, handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices and / or smart city devices, etc.

[0270] Figure 10 shows a schematic diagram of the structure of the electronic device 100.

[0271] Electronic device 100 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, sensor module 180, camera 193, and display screen 194, etc. The sensor module 180 may include pressure sensor 180A and touch sensor 180K, etc.

[0272] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0273] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0274] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0275] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0276] In this embodiment, the processor 110 is used in conjunction with the display screen 194, the touch sensor 180K, etc., to jointly implement the interface processing method provided in this application. For the specific implementation of this method, please refer to the previous description of Figures 6-9, which will not be repeated here.

[0277] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include a Universal Serial Bus (USB) interface, etc. The USB interface 130 is an interface compliant with the USB standard specification, specifically a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used for data transfer between the electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0278] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0279] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0280] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0281] In this embodiment of the application, the electronic device 100 may store an implementation program of the interface processing method in the aforementioned memory.

[0282] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0283] In this embodiment, the pressure sensor 180A can be used to detect various click, swipe, and other operations involved in Figures 2A-2C, 3A-3B, 4A-4B, and 5A-5B.

[0284] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual displays related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0285] In this embodiment, the touch sensor 180K is used to detect various click, swipe and other operations involved in Figures 2A-2C, 3A-3B, 4A-4B and 5A-5B.

[0286] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0287] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0288] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP displays the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0289] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0290] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0291] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0292] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be manufactured using organic light-emitting diodes (OLEDs), active-matrix organic light-emitting diodes (AMOLEDs), flexible light-emitting diodes (FLEDs), miniled, microLEDs, micro-OLEDs, quantum dot light-emitting diodes (QLEDs), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0293] In this embodiment of the application, the electronic device can control the display screen 194 to display corresponding feedback information based on the detected operation. Specifically, it can display user interfaces including but not limited to those shown in Figures 2A-2C, 3A-3B, 4A-4B, and 5A-5B.

[0294] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0295] Figure 11 is a software structure block diagram of an electronic device 100 according to an embodiment of this application.

[0296] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system libraries, and the kernel layer.

[0297] The application layer can include a series of application packages.

[0298] As shown in Figure 11, the application package may include a first application, an adaptive power saving module (APS), a camera, a gallery, a calendar, a caller ID, a map, a navigation system, WLAN, Bluetooth, and other applications.

[0299] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0300] As shown in Figure 11, the application framework layer may include the Activity Manager, Surface Flinger (SF), Advanced Graphics Platform (AGP), Windows Manager Service (WMS), and View System, etc.

[0301] The Activity Manager is responsible for managing the lifecycle of Activities in an application, including creation, startup, pause, resumption, and destruction, as well as events such as foreground / background switching.

[0302] SF is primarily responsible for compositing and displaying. Specifically, SF can use OpenGL and Hardware Composer to composite image frames based on layers to generate the final user interface.

[0303] AGP can be used to provide layer detection services to control whether an application enters management. For example, as described earlier in this application, if a whitelisted application is detected to meet the condition that a layer is covered, the application is controlled to enter management, that is, the compositing of the covered layer of the application is stopped; if a whitelisted application is detected not to meet the condition that a layer is covered, the application is controlled to exit management, that is, the compositing of the layer that the application had previously stopped is resumed.

[0304] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0305] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0306] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0307] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0308] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0309] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0310] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0311] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0312] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0313] A 2D graphics engine is a graphics engine for 2D drawing.

[0314] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver; the role of the display can be found in the previous method flow description, which will not be repeated here.

[0315] The following example, using a scene of capturing a photograph, illustrates the workflow of the software and hardware of the electronic device 100.

[0316] When touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking a touch click as an example, where the corresponding control is the camera application icon, the camera application calls the application framework layer's interface to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through camera 193.

[0317] It should be understood that the steps in the above-described method embodiments provided in this application can be implemented by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0318] This application also provides an electronic device that may include a memory and a processor. The memory may be used to store a computer program; the processor may be used to invoke the computer program in the memory to cause the electronic device to perform the method in any of the above embodiments.

[0319] This application also provides a chip system including at least one processor for implementing the functions involved in the methods performed by the electronic device in any of the above embodiments.

[0320] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0321] The chip system can consist of chips or include chips and other discrete components.

[0322] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0323] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0324] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0325] This application also provides a computer program product comprising: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the method executed by the electronic device in any of the above embodiments.

[0326] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed by the electronic device in any of the above embodiments.

[0327] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0328] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk).

[0329] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0330] In summary, the above description is merely an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made according to the disclosure of the present invention should be included within the scope of protection of the present invention.

Claims

1. An interface processing method, characterized in that, The method is applied in an electronic device, and the method includes: Launch the first application, display the first interface of the first application, and display the first layer as a composite system SF composite; the first interface corresponds to the first layer. In response to the first operation, a second interface of the first application is displayed. During the display of the second interface, the activity corresponding to the first interface is being run, and the first layer is not composited by the SF.

2. The method according to claim 1, characterized in that, In response to the first operation, a second interface of the first application is displayed, specifically including: In response to the first operation, a second image frame is synthesized based on the second layer, and a second interface is displayed based on the second image frame, wherein the second layer corresponds to the second interface.

3. The method according to claim 1 or 2, characterized in that, The method further includes: In response to the second operation, a third interface of the first application is displayed. During a period of time during the execution of the second operation, the first layer is composited by the SF. After the second operation, the first layer is not composited by the SF.

4. The method according to claim 3, characterized in that, The second operation includes a press operation and a release operation, the third interface is displayed in response to the release operation, and the method further includes: In response to the press operation, a first image frame is synthesized based on the first layer and the second layer; the second interface is displayed based on the first image frame; the second layer corresponds to the second interface.

5. The method according to claim 3, characterized in that, The second operation includes a press operation, a slide operation, and a release operation, the third interface is displayed in response to the release operation, and the method further includes: In response to the press operation, a first image frame is synthesized based on the first layer and the second layer, and the second interface is displayed based on the first image frame; the second layer corresponds to the second interface; In response to the sliding operation, a third image frame is synthesized based on the first layer, the second layer, and the third layer; a fourth interface is displayed based on the third image frame; the third layer corresponds to the third interface.

6. The method according to claim 5, characterized in that, The method further includes: In response to the press operation, a first timer is started. When the duration of the first timer reaches a first duration, it is detected whether the first layer is covered. If the first layer is covered, the first layer is not composited by the SF. If the first layer is not covered, the first layer is composited by the SF.

7. The method according to claim 6, characterized in that, The method further includes: If the first timer has not reached the first duration, the first timer is turned off in response to the lift-up operation.

8. The method according to any one of claims 5-7, characterized in that, The method further includes: In response to the lift-up operation, a second timer is started. When the duration of the second timer reaches a second duration, it is detected whether the first layer is covered. If the first layer is covered, the first layer is not composited by the SF. If the first layer is not covered, the first layer is composited by the SF.

9. The method according to any one of claims 6-8, characterized in that, Detecting whether the first layer is covered specifically includes: Based on the layer order and vertex coordinates of the first layer, and the layer order and vertex coordinates of the third layer, it is determined whether the first layer is covered. The first layer is determined to be covered if the following conditions are met: the first layer has a lower layer order than the third layer; and the vertex coordinates of the first layer are located in the region contained by the vertex coordinates of the third layer.

10. The method according to any one of claims 6-9, characterized in that, If the first layer is covered, and before the first layer is composited by the SF, the method further includes: It is determined that the third layer does not contain any transparent portions.

11. The method according to any one of claims 6-10, characterized in that, The method further includes: Since the first layer is covered, the control flag of the first application is set to true; Since the control flag of the first application is true, the first layer is not synthesized by the SF.

12. The method according to any one of claims 1-11, characterized in that, The method further includes: The system detects whether the first application belongs to the preset whitelist of applications. Based on the fact that the first application belongs to the whitelist of applications, the first layer is not composited by SF during the display of the second interface.

13. The method according to any one of claims 1-12, characterized in that, The first interface is the main interface of the first application, and the second interface is a non-main interface of the first application.

14. An electronic device, characterized in that, The electronic device includes: one or more memories, one or more processors, and a display screen; the memories are coupled to the one or more processors, the memories are used to store computer program code, the computer program code including computer instructions, and the one or more processors invoke the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-13.

15. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-13.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-13.

Images