Layer composition method, electronic device, and chip system

Abstract

The embodiments of the present application disclose a layer synthesis method, electronic device, and chip system applicable to the field of computer technology, wherein the method includes: the electronic device switches the layer synthesis mode from the first synthesis mode to the second synthesis mode in response to a first operation, and stores the first layer image of the target application in the target cache; in the second synthesis mode, the first animation effect of the target application is displayed, the first animation effect includes a first image frame, the image of the layer to be analyzed in the animation layer data corresponding to the first image frame is a black image, and the first image frame is obtained by the GPU performing layer synthesis on each layer to be synthesized based on the first layer image and layer properties in the target cache. The embodiments of the present application can effectively solve the problem of the application interface flashing black during the display process.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a layer synthesis method, electronic equipment, and chip system. Background Art

[0002] With the development of electronic technology, more and more electronic devices, such as mobile phones and tablets, have image display functions. The display interface displayed by electronic devices is usually synthesized from multiple layers, for example, it can be synthesized from a status bar layer and an application interface layer. The status bar layer can include multiple status icons, such as a status icon for indicating network status, a status icon for indicating device connection status, a status icon for indicating battery status, a status icon for indicating time, and so on.

[0003] In actual applications, it is found that during the operation of the application, due to the instability of the application software itself or the incompatibility of the application software with the system of the electronic device, the application interface may flash black during the display process, resulting in a poor user experience. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a layer synthesis method, an electronic device, and a chip system, which can effectively solve the problem of the application interface flashing black during the display process.

[0005] In a first aspect, an embodiment of the present application provides a layer synthesis method applied to an electronic device.

[0006] First, in response to a first operation by a user, the electronic device may switch the layer synthesis mode from the first synthesis mode to the second synthesis mode, and store the first layer image of the target application into the target cache. The first layer image is the layer image of the layer corresponding to the target application at the moment before the first operation starts. The first synthesis mode is performed by at least the hardware synthesizer HWC for layer synthesis, and the second synthesis mode is performed by the graphics processor GPU for layer synthesis. The first operation is usually used to trigger the electronic device to display the first animation effect of the target application.

[0007] Afterwards, the electronic device can display the first animation effect of the target application in the second synthesis mode. In the process of the electronic device displaying the first animation effect, the first animation effect may include a first image frame, the layer image to be analyzed in the animation layer data corresponding to the first image frame is a black image, and the first image frame is obtained by the GPU performing layer synthesis on each layer to be synthesized based on the first layer image and the layer attributes in the target cache. The animation layer data is the layer data of the target application at any moment between the first moment and the second moment, and the animation layer data includes layer attributes and the layer image to be analyzed. The above-mentioned first moment is the moment when the electronic device detects that the above-mentioned first operation begins to act on the electronic screen. The above-mentioned second moment is the moment when the electronic device ends displaying the first animation effect.

[0008] The electronic screen is a screen of an electronic device. The target application is an application running in the foreground.

[0009] The first operation mentioned above refers to an operation that may cause the application interface to flash black. The first animation effect mentioned above is the animation effect corresponding to the first operation mentioned above. Flashing black generally refers to the application interface layer displaying normally in a certain frame, then being black in one or more frames thereafter, and then returning to normal display. For example, the application interface layer displays normally in frame N, is black in frame N+1, and then returns to normal display in frame N+2.

[0010] Among them, there can be one or more layers corresponding to the target application. The embodiment of the present application does not limit the number of layers corresponding to the target application.

[0011] The layer attributes are used to describe the distribution characteristics of the layer on the display interface. The layer attributes may include but are not limited to the position, size, level, and transparency of the layer in the display interface.

[0012] The first synthesis mode may also be referred to as the conventional synthesis mode. The second synthesis mode may also be referred to as the GPU synthesis mode. In the GPU synthesis mode, the GPU performs layer synthesis. In the conventional synthesis mode, at least the HWC performs layer synthesis. It should be noted that the conventional synthesis mode is generally the layer synthesis method used in the prior art. In the conventional synthesis mode, layer synthesis can be performed with reference to the existing synthesis method, which will not be described in detail here.

[0013] In practice, the layer data of the application interface is usually updated in real time, for example, 60 times per second. That is to say, when the target application is running in the foreground, the target application can write the layer data of each layer into the corresponding cache queue in real time. For each layer, there is a corresponding cache queue. The electronic device (specifically, it can be SurfaceFlinger in the electronic device) can obtain the layer data of each layer from the cache queue, including the layer data of the layer corresponding to the target application and the layer data of other layers. As an example, the other layer can be the layer corresponding to the status bar. For ease of description, the layer corresponding to the status bar can also be referred to as the status bar layer. In the second synthesis mode, the electronic device can pass the obtained layer data of each layer to the GPU in the electronic device through SurfaceFlinger, and the GPU performs layer synthesis. At this time, the image synthesized by the GPU (or display interface) can be used for display on the electronic screen of the electronic device.

[0014] In an embodiment of the present application, since the application interface of the electronic device usually flashes black when responding to the first operation, the electronic device, when detecting the first operation, caches the first layer image of the target application, that is, the layer image generated before the first operation, in the target cache. In this way, in the second synthesis mode where the GPU performs layer synthesis, when the target application's layer image to be analyzed obtained in real time is a black image, the GPU in the electronic device can use the cached first layer image for layer synthesis, that is, it can avoid using the black image for layer synthesis, which helps to effectively prevent the application interface from flashing black during display. In addition, when the electronic device detects the first operation, it switches the layer synthesis mode to the second synthesis mode, so that only the GPU performs synthesis on each layer. Since the GPU has very strong parallel computing capabilities and is highly efficient in processing data such as graphics and images, the GPU performs layer synthesis, which can ensure the efficiency of layer synthesis. Among them, the target application's layer image to be analyzed obtained in real time is the layer image in the dynamic effect layer data corresponding to the target application obtained in real time.

[0015] Optionally, the target cache is typically a GPU cache. Caching the first layer image in the GPU allows the GPU to quickly obtain the first layer image and perform layer compositing when performing layer compositing, thereby further improving the efficiency of the GPU in performing layer compositing.

[0016] Optionally, in some application scenarios, in GPU synthesis mode, the electronic device can also pass the layer data of each layer corresponding to the target application to the GPU through SurfaceFlinger, and pass other layer data to the HWC. The GPU performs synthesis on the layers of the target application, and then the HWC continues to synthesize the image synthesized by the GPU with other layers to obtain a display interface for display on the electronic screen.

[0017] In some optional embodiments of the present application, SurfaceFlinger in the electronic device may send operation indication information to the GPU in response to the first operation, wherein the operation indication information is used to indicate that the electronic device has detected a side-slide gesture operation in the first operation stage.

[0018] Optionally, the operation instruction information may be a first label whose corresponding value is a first value.

[0019] The first tag is used to indicate whether the electronic device has detected a side-slide gesture operation in the first operation stage. The value of the first tag may include a first value and a second value. The first value indicates that the electronic device has detected a side-slide gesture operation in the first operation stage. The second value may indicate that the electronic device has detected a side-slide gesture operation that is not in the first operation stage.

[0020] In a first possible implementation manner of the first aspect, the first operation may be a side-slide gesture operation. When the first operation is a side-slide gesture operation, the first animation effect is an exit animation effect.

[0021] The side-slide gesture operation is used to exit the target application.

[0022] In practice, in response to the side-slide gesture operation, the electronic device may display an exit animation effect.

[0023] Among them, the exit animation effect (also known as application exit animation effect) is the animation effect when exiting the target application.

[0024] The exit animation effect may include the first image frame, which is obtained by performing layer synthesis based on the first layer image, and the layer image to be analyzed in the animation layer data corresponding to the first image frame is a black image.

[0025] Optionally, the exit animation effect may further include a second image frame, wherein the second image frame is synthesized based on the image of the layer to be analyzed in the animation layer data. In other words, the image of the layer to be analyzed in the animation layer data is not black.

[0026] It can be understood that if, from the time the electronic device detects the first operation and starts to execute it, to the time the electronic device finishes displaying the animation effect corresponding to the first operation, the layer images to be analyzed in the animation layer data obtained by the electronic device at each moment are not black images, then the exit animation effect displayed by the electronic device may also not include the above-mentioned first image frame.

[0027] The side-slide gesture operation (or side-slide exit operation) is generally an operation of sliding from one side of the left or right side of the electronic device to the other side to exit the target application. For example, it can be an operation of sliding from the right side to the left side of the mobile phone.

[0028] It is understandable that the first operation may also be other operations that may cause the application interface to flash black. This embodiment does not specifically limit the first operation.

[0029] In a second possible implementation of the first aspect, the electronic device switches the layer synthesis mode from the first synthesis mode to the second synthesis mode in response to the user's first operation, and stores the first layer image of the target application into the target cache. It can be that the electronic device switches the layer synthesis mode from the first synthesis mode to the second synthesis mode in response to the side-sliding gesture operation in the first operation stage, and stores the first layer image of the target application into the target cache.

[0030] The first operation stage is a stage from the first moment to the third moment.

[0031] The first moment is the moment when the electronic device detects that the first operation begins to act on the electronic screen.

[0032] The third moment is the moment when the electronic device detects that the first operation stops acting on the electronic screen.

[0033] In an embodiment of the present application, since the first operation usually requires a process from the start to the end of execution, that is, it takes a period of time, such as 0.1 seconds, to execute a complete first operation. Furthermore, since the layer data of the application interface is usually updated in real time and quickly, for example, it is updated 60 times per second. In the process of updating the layer data of the application interface, if the application is not compatible with the function corresponding to the first operation, the application interface may flash black before the first operation is completed, that is, before the operation gesture of the first operation leaves the electronic screen. Therefore, when the electronic device detects a side-slide gesture operation in the first operation stage, it switches the layer synthesis mode to the second synthesis mode in which the GPU performs layer synthesis, and caches the first layer image of the target application. This can avoid the application interface flashing black before the operation gesture leaves the electronic screen, and can also avoid the application interface flashing black after the operation gesture leaves the electronic screen, which helps to further improve the user experience.

[0034] In a third possible implementation of the first aspect, the electronic device may display an exit animation effect in response to a side-sliding gesture operation, and the specific response process may be: the electronic device displays an exit animation effect in response to a side-sliding gesture operation in the second operation stage.

[0035] The second operation stage is from the third moment to the second moment.

[0036] The third moment is the moment when the electronic device detects that the first operation stops acting on the electronic screen.

[0037] The second moment is the moment when the electronic device ends displaying the first animation effect.

[0038] In practice, when the electronic device displays the exit animation effect, if the layer image to be analyzed in the acquired animation layer data is a black image, the electronic device can switch the first layer image in the target cache to the acquired layer image to be analyzed.

[0039] In an embodiment of the present application, since the target application is executing the exit animation effect, the application interface is first scaled, then the application interface and the application icon are scaled together, and finally the application icon is scaled, that is, during the target application executing the exit animation effect, the layer images of each layer corresponding to the target application may change in real time. During the process of the target application executing the exit animation effect, the electronic device promptly uses the latest layer image to update the first layer image (or cached layer image) cached in the target cache when the layer image to be analyzed is not a black image. That is, the electronic device can cache the latest layer image, which helps to ensure the accuracy of layer synthesis using the cached layer image. That is, it can ensure that the display interface obtained by layer synthesis using the cached layer image is more consistent with the changes between the display interface at the previous moment, which helps to further improve the user experience.

[0040] In a fourth possible implementation of the first aspect, before the electronic device instructs the GPU in the electronic device to perform layer synthesis on the required layers based on the first layer image and the layer properties in the target cache, the electronic device may also perform a black image judgment on the layer image to be analyzed based on the pixel data of each pixel point in the layer image to be analyzed.

[0041] In an embodiment of the present application, before each layer synthesis is performed, a black image judgment is performed on the real-time layer image to be analyzed, which can avoid using a black image to perform layer synthesis, and helps to effectively avoid the application interface flashing black during the display process.

[0042] In a fifth possible implementation of the first aspect, the target cache is a GPU cache. In electronic devices, the GPU is usually used to perform black image determination on the image of the layer to be analyzed.

[0043] Specifically, the process by which the GPU determines whether an image in the layer to be analyzed is black is as follows:

[0044] First, the GPU reduces the image of the layer to be analyzed to a preset size to obtain the target analysis layer image. In practice, the preset size is usually very small.

[0045] Then, the GPU determines a normalized pixel value of the target analysis layer image, where the normalized pixel value is a ratio of an average pixel value to a maximum pixel value of the target analysis layer image.

[0046] Afterwards, the GPU determines that the image of the layer to be analyzed is a black image when the normalized pixel value is less than a preset normalized pixel threshold.

[0047] Finally, when the normalized pixel value is greater than or equal to the normalized pixel threshold, it is determined that the image of the layer to be analyzed is not a black image.

[0048] In the embodiments of the present application, since the image layer to be analyzed typically has a large data volume, the computational effort required to determine whether the image layer to be analyzed is black is substantial. Furthermore, since the GPU possesses flexible and efficient graphics and image data processing capabilities, having the GPU perform black image determination on the image layer to be analyzed ensures the efficiency and accuracy of black image determination, thereby ensuring the efficiency and accuracy of layer synthesis. Furthermore, the GPU first reduces the image layer to be analyzed to a relatively small size. Then, it analyzes the pixel data of each pixel in the small target analysis layer image and determines whether the target analysis layer image is black, thereby determining whether the image layer to be analyzed is black. This significantly reduces the amount of data processing required, further improving the efficiency of black image determination.

[0049] In a sixth possible implementation manner of the first aspect, the electronic device may also promptly switch the layer synthesis mode from the second synthesis mode to the first synthesis mode after finishing displaying the first animation effect.

[0050] In an embodiment of the present application, the display of the first animation effect ends, indicating that the electronic device has completed the response to the first operation. At this time, the electronic device can switch the layer synthesis mode from the second synthesis mode (also called GPU synthesis mode) back to the first synthesis mode (also called conventional synthesis mode). In conventional synthesis mode, the GPU and HWC collaborate to perform layer synthesis, which can reduce the data processing capacity of the GPU. Since the GPU can perform layer synthesis and can also be used to complete other graphics processing tasks, the GPU and HWC collaborate to perform layer synthesis, which can reduce the data processing capacity of the GPU, thereby improving the overall data processing efficiency of the electronic device.

[0051] In a seventh possible implementation manner of the first aspect, the electronic device may further release the target cache in a timely manner after finishing displaying the first animation effect.

[0052] In the embodiment of the present application, timely releasing the target cache can improve the utilization of storage resources. In addition, when the target cache is a GPU cache, it helps to improve the utilization of storage resources in the GPU.

[0053] In an eighth possible implementation of the first aspect, the first animation effect also includes a second image frame, and the image of the layer to be analyzed in the animation layer data corresponding to the second image frame is not a black image. The second image frame is obtained by the GPU performing layer synthesis on the required synthesized layers based on the image of the layer to be analyzed and the layer properties.

[0054] In an embodiment of the present application, in the second synthesis mode in which layer synthesis is performed by the GPU, when the image of the layer to be analyzed of the target application obtained in real time is not a black image, the GPU in the electronic device can use the image of the layer to be analyzed obtained in real time to perform layer synthesis, thereby ensuring the accuracy of layer synthesis.

[0055] It can be understood that if, from the time when the electronic device detects the first operation and starts to execute it to the time when the electronic device finishes displaying the animation effect corresponding to the first operation, the layer images to be analyzed in the animation layer data obtained by the electronic device at each moment are not black images, then the first animation effect displayed by the electronic device may also not include the first image frame.

[0056] In a second aspect, an embodiment of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program that can be run on the processor. When the processor executes the computer program, it implements any method as described in the first aspect above.

[0057] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of any one of the first aspects above.

[0058] In a fourth aspect, embodiments of the present application provide a chip system, comprising a processor coupled to a memory, the processor executing a computer program stored in the memory to implement any of the methods described in the first aspect. The chip system may be a single chip or a chip module consisting of multiple chips.

[0059] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute any one of the methods of the first aspect.

[0060] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of a display interface of a mobile phone provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram showing the effect of an application interface flashing black on the display interface of a mobile phone provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the animation effect when exiting an application provided in an embodiment of the present application;

[0064] Figure 4AA schematic diagram showing the effect of the application interface provided in an embodiment of the present application flashing black in the first operation stage;

[0065] Figure 4B A schematic diagram showing the effect of the application interface provided in an embodiment of the present application flashing black in the second operation stage;

[0066] Figure 5A A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0067] Figure 5B A flowchart of a layer synthesis method provided in an embodiment of the present application;

[0068] Figure 5C A flowchart of a GPU performing black image judgment on an image of a layer to be analyzed provided in an embodiment of the present application;

[0069] Figure 6 A schematic structural diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0071] The following describes some concepts that may be involved in the embodiments of this application:

[0072] (1) Multiple: Unless otherwise specified, in the embodiments of the present application, multiple refers to two or more.

[0073] (2) Surface: In the embodiments of the present application, the display interface displayed by the electronic device is usually synthesized from multiple layers.

[0074] See also Figure 1 , Figure 1 This is a schematic diagram of a display interface of a mobile phone provided in an embodiment of the present application.

[0075] like Figure 1 As shown in the figure on the left, the display interface of the mobile phone can be synthesized by the layer corresponding to the status bar 101 and the layer corresponding to the application interface 102. Among them, the status bar 101 is used to indicate the status of the mobile phone. The status bar 101 usually includes multiple status icons.

[0076] like Figure 1As shown in the figure on the right side, the status bar 101 may include status icons for indicating network status, device connection status, battery status, time, etc. The application interface 102 may be an interface of a "settings" application (an application specifically used to perform system setting functions).

[0077] It is understandable that a mobile phone is only an example of an electronic device, and the electronic device may also be other devices, for example, the electronic device may also be a tablet. In addition, the display interface of the electronic device may also be composed of other layers. The embodiment of this application only uses Figure 1 Take this as an example to illustrate.

[0078] Combine Figure 1 Each layer in the display interface has its own position, size, and content on the screen. During the application running process, the position, size, and content of the corresponding layer on the screen may vary at different times.

[0079] Each layer has corresponding layer properties and a layer image. The layer image represents the content that the layer will display on the screen. The layer image can be the image obtained by drawing and rendering the layer. Layer properties describe the distribution characteristics of the layer on the display interface. Layer properties can include information such as the position, size, and level of the corresponding layer. Layer properties indicate the position, size, level, and transparency of the layer on the display interface. The level indicates the vertical order of the layers on the screen.

[0080] (3) Graphics Processing Unit (GPU): In the embodiment of the present application, the GPU is a general-purpose image processing device that can be used to synthesize layers and perform other graphics processing tasks.

[0081] (4) Hardware Composer (HWC): In the embodiment of the present application, the HWC is a dedicated image processing device that can be used to synthesize layers.

[0082] In practice, at least one of a GPU and a HWC may be used to synthesize the layers.

[0083] (5) Layer management module (SurfaceFlinger): In the embodiment of the present application, SurfaceFlinger is a system service, which is mainly used to implement functions such as layer creation, control, and management.

[0084] In practice, an application can send a layer creation request to SurfaceFlinger after startup. After receiving the application's layer creation request, SurfaceFlinger can create the corresponding layer for the application. In practice, an application can correspond to one layer or multiple layers.

[0085] In addition, SurfaceFlinger can also allocate a corresponding buffer queue (BufferQueue) for the created layer, which can be used to store layer data of the layer, where the layer data can include layer properties and layer images.

[0086] In practice, applications can draw and render the content to be displayed in the layer through canvas (Canvas) or open graphics library (Open Graphics Library, OpenGL), and transfer the rendered layer image to the corresponding cache queue, and also transfer the layer properties of the layer to the cache queue. SurfaceFlinger can obtain the layer data of the corresponding layer from the cache queue corresponding to each layer, and then combine the layer data of each layer to synthesize each layer through GPU and / or HWC. Among them, synthesizing multiple layers usually refers to synthesizing the layer images of multiple layers into one image based on the layer properties of each layer.

[0087] (6) Application interface flashing black: In the embodiments of the present application, application interface flashing black generally refers to the application interface layer displaying normally in a certain frame, then being black in one or more frames thereafter, and then displaying normally again. For example, the application interface layer displays normally in frame N, is black in frame N+1, and then displays normally again in frame N+2.

[0088] (7) Window Manager Service (WMS): In the embodiments of this application, the WMS service is generally used to start, add, or delete windows. A window corresponds to a layer. The layer is used to draw the content to be displayed, and the window is used to display the content of the layer.

[0089] The WMS service can allocate a window for the application layer and send the window parameters, such as window position and window size, to the application to instruct the application to draw and render the layer.

[0090] In practice, the WMS service can also respond to touch operations on the electronic screen and send the operation information of the touch operation to the application. In this way, the application can draw and render the application interface layer according to the touch operation.

[0091] The touch operation may be an operation triggered by a user's finger, knuckle, etc., or an operation triggered by a stylus.

[0092] (8) Target Application: In the embodiments of the present application, the target application is usually the application running in the foreground. The target application is often an application that is unstable or does not match the system of the electronic device.

[0093] With the development of electronic technology, more and more electronic devices such as mobile phones and tablets have image display functions. The display interface of electronic devices is usually synthesized by multiple layers, such as Figure 1 The display interface can be synthesized by the status bar layer 101 and the application interface layer 102.

[0094] In electronic devices, modules related to layer synthesis may include but are not limited to: SurfaceFlinger, GPU, and HWC.

[0095] In practice, SurfaceFlinger can maintain a cache queue for each layer. Figure 1 , a cache queue can be maintained for the layer corresponding to the status bar 101 , and a cache queue can also be maintained for the layer corresponding to the application interface 102 .

[0096] When the target application is running in the foreground, the target application can write the layer data of each layer to the corresponding cache queue in real time. SurfaceFlinger can obtain the layer data of each layer from the cache queue, including the layer data of the layer corresponding to the target application and the layer data of other layers. Afterwards, SurfaceFlinger can combine the layer data of each layer obtained to instruct the GPU and / or HWC to synthesize each layer based on the layer properties of each layer.

[0097] In actual applications, it has been found that during the operation of an application, due to reasons such as instability of the application software itself or incompatibility between the application software and the electronic device system, messenger applications are prone to instability when running on the Android system, and the application interface is prone to flashing black during display, resulting in a poor user experience. Among them, the application interface flashing black usually means that the application interface layer displays normally in a certain frame, and then one or more frames are black, and then it displays normally again. For example, the application interface layer displays normally in frame N, turns black in frame N+1, and displays normally again in frame N+2.

[0098] Figure 2 The figure shows an effect diagram of the application interface displayed on the mobile phone display interface flashing black. Figure 2As shown in (a) of FIG, the application interface 202 on the icon interface 201 is displayed normally. Figure 2 As shown in (b), the application interface 202 on the icon interface 201 changes from normal display to black. Figure 2 As shown in (c) , the application interface 202 on the icon interface 201 changes from a black image back to a normal display.

[0099] It should be noted that if the layers of the application interface are displayed as black images, it usually means that the application did not draw the corresponding layers at the corresponding time, resulting in the black images being placed in the corresponding cache queues. In this way, SurfaceFlinger obtains the layer data from each cache queue and instructs the GPU and / or HWC to synthesize each layer. The resulting application interface in the display interface is black.

[0100] In practice, in order to achieve visual continuity in interface switching, electronic devices usually respond to user operations and switch interfaces by displaying animation effects, so that a coherent connection between different interfaces can be achieved. As an example, an electronic device can exit the target application running in the foreground by displaying an exit animation effect. Among them, the exit animation effect generally refers to the animation effect when exiting the application. In some embodiments, the exit animation effect is also called the application exit animation effect.

[0101] The following uses the application exit animation as an example to illustrate the scenarios in which electronic devices display animation effects. The application exit animation is the animation effect displayed by the electronic device in response to an exit operation, such as a side swipe exit operation or a swipe up exit operation.

[0102] In practice, the electronic device can respond to a user's side swipe gesture (also known as a side swipe gesture operation or a side swipe exit operation) or an upward swipe gesture (also known as an upward swipe gesture operation or an upward swipe exit operation) to exit the target application currently running in the foreground. In other words, the target application is exited from the application interface to the desktop interface.

[0103] A side-swipe exit operation typically involves swiping from one side of an electronic device to the other to exit the target application. For example, this might involve swiping from the right side of a phone to the left. A swipe-up exit operation typically involves swiping from the bottom to the top of an electronic device to exit the target application. In practice, a side-swipe exit operation typically occurs on an application interface displayed full screen.

[0104] Figure 3 This is a schematic diagram of the animation effect when exiting the "Settings" application provided in the embodiment of this application. It should be understood that Figure 3 This description only takes the Settings app as an example.

[0105] like Figure 3 As shown, the application exit animation can include the following three parts: the first part is the application interface zoom part, for example, from Figure 3 (a) in the equation changes to Figure 3 (b) in the second part, the application interface and application icon are scaled together, for example, from Figure 3 (c) in the equation changes to Figure 3 (d) in the third part, the application icon scaling part, for example, from Figure 3 (d) in the equation changes to Figure 3 (e) in .

[0106] Combine Figure 3 In (a), the mobile phone can respond to the user's side swipe gesture from the right edge to the left on the application interface of the "Settings" application, entering the application exit animation. During the process of exiting the "Settings" application, the application interface 301 begins to shrink.

[0107] Combine Figure 3 In (b), the application interface 301 is reduced from full-screen display to non-full-screen display.

[0108] Combine Figure 3 In (c), the mobile phone further reduces the application interface 301 of the "Settings" application from a non-full screen display to a superimposed display of the application interface 301 and the application icon 302.

[0109] Here, the electronic device may display application icon 302 in a layer below application interface 301, with the transparency of application interface 301 being higher than that of application icon 302, and application icon 302 being visible through application interface 301. It is understood that in some application scenarios, the electronic device may also display application icon 302 in a layer above application interface 301, in which case the transparency of application interface 301 is lower than that of application icon 302, and application interface 301 is visible through application icon 302.

[0110] Understandably, Figure 3 The intersection of the image and text between the application interface 301 and the application icon 302 shown in (c) is only to indicate that the upper one is transparent, and does not actually occur.

[0111] Combine Figure 3 In (d), the mobile phone displays the "Settings" application from the application interface 301 and the application icon 302 in an overlaid manner, and further reduces the display to only the application icon 302 in an enlarged manner.

[0112] Combine Figure 3 In (e), the mobile phone can reduce the application icon 302 to a normal size for display.

[0113] Combine Figure 3 (a) to Figure 3 As can be seen from (e) in the figure, when an application is exited, the mobile phone exits from the application interface to the icon interface (also called the desktop interface), and by gradually shrinking the application interface toward the application icon position of the application, a coherent connection between the interfaces when the application is exited is achieved.

[0114] It should be noted that when the electronic device displays any image frame in the animation effect, the application interface may appear black. Figure 3 , that is, when the phone displays the application exit animation, Figure 3 (a) to Figure 3 At any stage in (e), the application interface may flash black.

[0115] In actual applications, it is also found that the electronic device usually flashes black when the application interface is in the process of responding to the first operation. Since the first operation usually takes a process from the start to the end, that is, it takes a period of time to perform a complete first operation. For the sake of description, the time from the start of the first operation to the end of the first operation can be recorded as the operation duration. In practice, the operation duration is usually very short, for example, 1 second.

[0116] Because the layer data in the app interface is usually updated in real time, for example, 60 times per second, the update cycle of the layer data in the app interface is usually shorter than the duration of the above operation. During the process of updating the layer data in the app interface, if the application is not compatible with the functions corresponding to certain operations, such as the side-swipe exit function, the app interface may flash black before the first operation is completed.

[0117] That is to say, in electronic devices, the application interface may flash black in the following two situations.

[0118] In case 1, the first operation is executed and the operation gesture has not left the electronic screen, and the application interface flashes black.

[0119] In scenario 2, the operation gesture of the first operation has left the electronic screen and the animation effect corresponding to the first operation has not yet finished displaying, and the application interface flashes black.

[0120] In an embodiment of the present application, for the sake of convenience of description, the stage in which the first operation is executed and the operation gesture has not yet left the electronic screen can be recorded as the first operation stage, and the stage in which the operation gesture of the first operation has left the electronic screen and the display of the animation effect corresponding to the first operation has not yet ended can be recorded as the second operation stage.

[0121] In this application, the first operation stage can also be described as the stage from the first moment to the third moment. The second operation stage can also be described as the stage from the third moment to the second moment. The first moment is the moment when the electronic device detects that the first operation begins to act on the electronic screen. The second moment is the moment when the electronic device stops displaying the first animation effect. The third moment is the moment when the electronic device detects that the first operation stops acting on the electronic screen.

[0122] The following combination Figure 4A and Figure 4B , taking the first operation as the side-slide exit operation (or side-slide gesture operation) as an example, a schematic diagram of the effect of the application interface flashing black at different operation stages is described. Among them, the side-slide gesture operation is used to trigger the electronic device to display the application exit effect (or called the exit animation effect). In the embodiment of the present application, the above-mentioned first operation refers to an operation that may cause the application interface to flash black. It can be understood that the first operation can also include other operations that may cause the application interface to flash black, and this embodiment does not specifically limit the first operation.

[0123] Figure 4A This is a schematic diagram of the effect of the application interface provided in an embodiment of the present application flashing black in the first operation stage.

[0124] Figure 4B Schematic diagram of the effect of the application interface provided in an embodiment of the present application flashing black in the second operation stage.

[0125] like Figure 4A As shown in (a), the mobile phone detects that a side-slide gesture operation has just been performed. At this time, the user's finger acts on the right side of the mobile phone, and the application interface 301 is displayed normally.

[0126] like Figure 4A As shown in (b), the mobile phone detects that the user's finger is still on the mobile phone screen and has not left, and the user's finger slides from right to left, and the application interface 301 is displayed as a black image.

[0127] like Figure 4A As shown in (c), the mobile phone detects that the user's finger is still on the mobile phone screen and has not left, and the user's finger continues to slide from right to left, and the application interface 301 returns to normal display.

[0128] Combine Figure 4A (a) to Figure 4A As can be seen from (c) in the figure, the application interface may appear to be black when the first operation is being executed and the operation gesture of the first operation has not yet left the electronic screen. In other words, the application interface may appear to be black when the first operation is in the first operation stage.

[0129] Understandably, from Figure 4A (a) to Figure 4A (c) in the figure takes a very short time, for example, 0.1 seconds.

[0130] like Figure 4B As shown in (a), the mobile phone detects that the side-slide gesture operation is completed. At this time, the user's finger has just left the mobile phone screen and the application interface 301 is displayed normally. It can be understood that Figure 4B The operation gesture shown in (a) is located outside the mobile phone, which is only used to indicate that the operation gesture has left the mobile phone screen, and is not a limitation on the specific location of the operation gesture.

[0131] like Figure 4B As shown in (b), the mobile phone responds to the side-slide gesture operation, displays the exit animation effect, and the application interface 301 is reduced from full-screen display to non-full-screen display, and the application interface 301 is displayed normally.

[0132] like Figure 4B As shown in (c), when the mobile phone is displaying the exit animation effect, the displayed application interface 301 turns black.

[0133] like Figure 4B As shown in (d), the mobile phone enlarges and displays the application icon 302 during the process of displaying the exit animation effect, and the display is normal.

[0134] like Figure 4B As shown in (e), the mobile phone reduces the application icon 302 to a normal size and displays it normally during the process of displaying the exit animation effect.

[0135] Combine Figure 4B (a) to Figure 4B As can be seen in (e), the application interface may appear to flash black from the time the first operation gesture leaves the electronic screen to the end of the animation effect corresponding to the first operation. In other words, the application interface may appear to flash black during the second operation phase of the first operation.

[0136] Figure 4A and Figure 4B The black image is shown for example only. It is understood that when the phone is in the first operation stage, the application interface may be black in any image frame of the first animation effect displayed. When the phone is in the second operation stage, the application interface may also be black in any image frame of the first animation effect displayed.

[0137] In view of the above-mentioned technical problems in the related art, in order to effectively resolve the issue of an application interface flashing black when an electronic device responds to a first operation, in an embodiment of the present application, first, in response to a user's first operation, the electronic device can switch its layer compositing mode from a first compositing mode to a second compositing mode and store a first layer image of a target application in a target cache. The first layer image is the layer image of the layer corresponding to the target application immediately before the first operation begins. In the first compositing mode, layer compositing is performed by at least a hardware compositor (HWC), while in the second compositing mode, layer compositing is performed by a graphics processing unit (GPU). The first operation is typically used to trigger the electronic device to display a first animation effect of the target application. Subsequently, the electronic device can display the first animation effect of the target application in the second compositing mode. During the display of the first animation effect, the first animation effect can include a first image frame, wherein the image to be analyzed in the animation layer data corresponding to the first image frame is black, and the first image frame is obtained by the GPU performing layer compositing on each layer to be composited based on the first layer image and layer properties in the target cache. The animation layer data is layer data of the target application at any time between a first moment and a second moment, and includes layer attributes and a layer image to be analyzed. The first moment is the moment when the electronic device detects that the first operation begins to act on the electronic screen. The second moment is the moment when the electronic device stops displaying the first animation effect.

[0138] The embodiments of the present application have at least the following beneficial effects:

[0139] 1. In an embodiment of the present application, since the application interface of an electronic device usually flashes black when responding to a first operation, the electronic device, upon detecting the first operation, caches the first layer image of the target application, that is, the layer image generated before the first operation, in a target cache. Thus, in the second synthesis mode in which the GPU performs layer synthesis, when the target application's layer image to be analyzed, acquired in real time, is black, the GPU in the electronic device can use the cached first layer image for layer synthesis. That is, it can avoid using a black image for layer synthesis, which helps to effectively prevent the application interface from flashing black during display.

[0140] 2. When the electronic device detects the first operation, it switches the layer synthesis mode to the second synthesis mode, so that only the GPU performs synthesis on each layer. Since the GPU has very strong parallel computing capabilities and is highly efficient in processing data such as graphics and images, the GPU performs layer synthesis to ensure layer synthesis efficiency.

[0141] The following describes the usage scenarios of the embodiments of this application:

[0142] The embodiments of the present application may be applicable to a scenario where an application interface displayed on an electronic device flashes black when a user performs a touch operation on the electronic device.

[0143] The layer synthesis method provided in the embodiments of the present application can be applied to electronic devices, which can be terminals or servers. The terminals can be mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other terminals. The wearable devices can be smart watches or smart bracelets, etc., which are not limited in the embodiments of the present application.

[0144] Figure 5A A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5A ,The system of an electronic device may include an application layer, a framework layer, a kernel layer and a hardware layer.

[0145] like Figure 5A As shown, the application layer can include a drawing and rendering module and a series of applications. For example, the application layer can include applications such as camera, gallery, calendar, map, navigation, and Bluetooth. Applications in the application layer can call the drawing and rendering module to render the corresponding layers of the application. For example, the drawing and rendering module can be a Canvas. The application layer is also called the application layer.

[0146] The framework layer provides an application programming interface (API) and programming framework for the application layer's applications. The framework layer includes some predefined functions. The framework layer is also called the application framework layer or the system service framework layer.

[0147] like Figure 5A As shown, the framework layer may include the input system, WMS service, and SurfaceFlinger. The input system is used to obtain touch information acting on the electronic screen. Touch information can be information triggered by the user's fingers, knuckles, etc., or information triggered by a stylus.

[0148] The WMS service can distribute the touch information obtained by the input system to the application. The application in the application layer can call the drawing and rendering module to draw and render the layer corresponding to the application based on the touch information, such as the touch starting point, touch end point, touch type, etc. The WMS service can also send indication information to SurfaceFlinger based on the touch information obtained by the input system. For example, when the touch information indicates that a side-slide exit operation is detected in the first operation stage, the WMS service can send operation indication information to SurfaceFlinger, wherein the operation indication information is used to indicate that a side-slide exit operation is detected in the first operation stage.

[0149] SurfaceFlinger can create corresponding layers for applications in the application layer. While applications in the application layer are running, SurfaceFlinger can also obtain layers rendered by applications in the application layer and synthesize the obtained layers. For example, SurfaceFlinger can synthesize the obtained layers through the GPU and / or HWC.

[0150] The kernel layer is the layer between hardware and software. The kernel layer includes at least the central processing unit (CPU) driver, GPU driver, HWC driver, touch driver, display driver, sensor driver, etc.

[0151] The CPU driver is used to drive the CPU; the GPU driver is used to drive the GPU; the HWC driver is used to drive the HWC; the touch driver is used to drive the touch panel; the display driver is used to drive the display panel; and the sensor driver is used to drive the sensor.

[0152] In other embodiments, the system of the electronic device may further include a system library layer, which is located between the framework layer and the kernel layer, and is not limited here. The system library may include multiple functional modules. For example, the system library may include a surface manager, a media library, a three-dimensional graphics engine (for example, OpenGLES), and a 2D graphics engine. Among them, the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static graphic files, etc. The three-dimensional graphics engine is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0153] The hardware layer includes physical devices such as CPU, GPU, HWC, touch panel, display panel, and sensors.

[0154] The touch panel includes: a touch sensor and a touch integrated circuit (IC) chip, and the display panel may include: a display screen and a display IC chip. In one embodiment, the touch IC chip and the display IC chip can be integrated into one or independently set. The display screen may include but is not limited to: a liquid crystal display (LCD), a light emitting diode (LED) display screen, etc., and the embodiments of the present application are not limited to this. It can be understood that the electronic screen described in the embodiments of the present application may include: a touch panel and a display panel.

[0155] The sensors may include: pressure sensors, gyroscope sensors, acceleration sensors, distance sensors, proximity light sensors, fingerprint sensors, etc. The functions of each sensor are not described in detail in the embodiments of the present application.

[0156] Based on the above Figure 5A The system framework of the electronic device shown in the figure is described in detail below with reference to the interactive flow diagram, taking the first operation as the side-slide exit operation as an example. Figure 5A The electronic device shown is a specific interaction process between various structures in the system of the electronic device during the process of implementing the layer synthesis method of the present application.

[0157] It should be noted that the framework layer software interacts with the corresponding hardware in the hardware layer through the hardware drivers in the kernel layer. For example, SurfaceFlinger interacts with the GPU through the GPU driver, and SurfaceFlinger interacts with the HWC through the HWC driver, and so on.

[0158] For further example, SurfaceFlinger sending the operation instruction information to the GPU means that SurfaceFlinger sends the operation instruction information to the GPU through the GPU driver.

[0159] Figure 5B This is a flowchart of a layer synthesis method provided by an embodiment of the present application, which is applied to the above Figure 5A In the electronic device shown.

[0160] S501, SurfaceFlinger responds to a slide-out operation in a first operation phase, switches the layer composition mode of a target application to a GPU composition mode, and caches a first layer image of a corresponding layer of the target application in a GPU.

[0161] The GPU synthesis mode is also called the second synthesis mode.

[0162] Here, the first layer image cached in the GPU may also be referred to as a cached layer image.

[0163] In the embodiment of this application, combined with Figure 5A , when the electronic device detects an exit gesture (such as Figure 3 When the side-swipe gesture operation (also called the side-swipe exit operation) begins to execute, the input system can obtain touch information acting on the electronic screen and send the touch information to the WMS service. The WMS service can send indication information to SurfaceFlinger based on the touch information obtained by the input system. For example, when the touch information indicates that the side-swipe exit operation in the first operation stage is detected, the WMS service can send operation indication information to SurfaceFlinger, where the operation indication information is used to indicate that the side-swipe exit operation in the first operation stage is detected.

[0164] Optionally, the operation instruction information may be a first label whose corresponding value is a first value.

[0165] Among them, the first tag is used to indicate whether the electronic device detects a side-sliding exit operation in the first operation stage. The value of the first tag may include a first value and a second value. Among them, the first value indicates that the electronic device detects that the side-sliding exit operation is in the first operation stage. The second value may indicate that the electronic device detects that the side-sliding exit operation is not in the first operation stage. It can be understood that when the value of the first tag is the second value, it may indicate that the electronic device detects that the side-sliding exit operation is in the second operation stage, and it may also indicate that the electronic device has finished displaying the application exit animation.

[0166] In practice, the input system of an electronic device can preliminarily determine the purpose of the first operation or the type of the first operation by analyzing the touch starting point. For example, when the first operation is a swipe to exit, the touch starting point is usually a side edge of the electronic device, such as the left edge or the right edge.

[0167] It is understood that when the electronic device detects that the slide-to-exit operation is in the first operation stage, it generally indicates that the slide-to-exit operation has begun and the operation gesture has not yet left the electronic screen. As an example, the operation gesture from acting on the electronic screen to leaving the electronic screen can be considered as a complete execution of the first operation.

[0168] It is understood that the operation gesture leaving the electronic screen can mean that after the user's finger leaves the electronic screen, there is no further action on the electronic screen within a preset time interval, such as 0.5 seconds. The embodiments of the present application do not specifically limit how the electronic device determines whether the operation gesture of the first operation has left the electronic screen.

[0169] In practice, the electronic device may determine that a slide-to-exit operation in the first operation phase is detected when the first operation acting on the side edge of the electronic device is just detected. When the electronic device detects the slide-to-exit operation in the first operation phase, the WMS service in the electronic device may send operation indication information to SurfaceFlinger.

[0170] In practice, layer compositing modes can include GPU compositing mode (also called second compositing mode) and regular compositing mode (also called first compositing mode). GPU compositing mode indicates that layer compositing is performed only by the GPU. That is, when compositing each layer of the target application, only the GPU is used for compositing. Regular compositing mode performs layer compositing at least by the HWC.

[0171] It should be pointed out that the conventional synthesis mode is usually a layer synthesis method used in the prior art. In the conventional synthesis mode, layer synthesis can be performed with reference to the existing synthesis method, which will not be described in detail here.

[0172] In some application scenarios, in GPU compositing mode, SurfaceFlinger obtains the layer data of each layer in real time and sends all the obtained layer data to the GPU, which performs compositing on all the layers. At this time, the image (or display interface) synthesized by the GPU can be used to display on the electronic screen of the electronic device.

[0173] In other application scenarios, in GPU synthesis mode, SurfaceFlinger can also pass the layer data of each layer corresponding to the target application to the GPU, and pass other layer data to the HWC. The GPU performs synthesis on the layers of the target application, and then the HWC continues to synthesize the image synthesized by the GPU with other layers to obtain a display interface for display on the electronic screen.

[0174] It should be noted that SurfaceFlinger caches the first layer image it has already acquired, that is, the layer image generated before the first operation, in the GPU cache. In this way, in GPU compositing mode, when the layer image to be analyzed acquired by the GPU in real time is a black image, the GPU can use the cached first layer image (or cached layer image) for layer compositing. In other words, it can avoid using a black image for layer compositing, which helps to effectively prevent the application interface from flashing black during display.

[0175] S502, SurfaceFlinger sends the operation instruction information to the GPU.

[0176] Here, SurfaceFlinger sends the operation instruction information to the GPU in a timely manner, so that the GPU can enter the state of cooperating with the GPU synthesis mode as soon as possible.

[0177] It is understandable that SurfaceFlinger interacts with the GPU through the GPU driver. For example, SurfaceFlinger sends operation instruction information to the GPU, which means that SurfaceFlinger sends the operation instruction information to the GPU through the GPU driver.

[0178] S503, SurfaceFlinger obtains layer data of each layer in real time in GPU synthesis mode, and sends the layer data to the GPU.

[0179] The layer data includes layer images and layer properties. For ease of description, the layer image of the target application's corresponding layer is referred to as the layer image to be analyzed.

[0180] In an embodiment of the present application, SurfaceFlinger can obtain the layer data of each layer to be synthesized in real time in the GPU synthesis mode, wherein the layers to be synthesized may include the layers corresponding to the target application and other layers. It can be understood that the above-mentioned other layers are layers other than the layers corresponding to the target application. As an example, other layers may include the status bar layer.

[0181] In practice, in GPU compositing mode, the layers corresponding to the target application are only composited by the GPU.

[0182] Optionally, SurfaceFlinger can send all the acquired layer data to the GPU, which performs synthesis of all the layers. At this time, the image (or display interface) synthesized by the GPU can be used for display on the electronic screen of the electronic device.

[0183] Optionally, SurfaceFlinger can also send only the layer data of each layer corresponding to the target application to the GPU, and the GPU will perform synthesis on each layer corresponding to the target application. It can be understood that when only the GPU performs synthesis on each layer corresponding to the target application, other layers and layers synthesized by the GPU need to be further synthesized by the HWC.

[0184] S504: The GPU performs black image determination on the received layer image to be analyzed.

[0185] In an embodiment of the present application, before the GPU performs layer synthesis on each layer corresponding to the target application, it is necessary to first perform a black image judgment on the layer image to be analyzed in the layer to avoid using a black image for layer synthesis, thereby preventing the application interface from flashing black during the display process.

[0186] As an example, the GPU can read the pixel value of each pixel in the image layer to be analyzed, then calculate the pixel mean of all pixels, and finally compare the pixel mean with a preset pixel threshold. If the calculated pixel mean is greater than the preset pixel threshold, it means that the image layer to be analyzed is not black. If the calculated pixel mean is less than or equal to the preset pixel threshold, it means that the image layer to be analyzed is black. The preset pixel threshold is typically a pre-set pixel value. In practice, the preset pixel threshold is usually small.

[0187] As another example, after calculating the pixel mean of the image layer to be analyzed, the GPU may divide the pixel mean by the maximum possible value of the image layer to be analyzed to obtain a quotient. In practice, the maximum possible value of the image layer to be analyzed is typically 255. It is understood that the GPU may also divide the pixel mean by a value greater than the maximum value to obtain a quotient.

[0188] Finally, the GPU compares this quotient with a pre-set threshold. If the quotient is greater than or equal to the threshold, the image in the layer being analyzed is not black. If the quotient is less than the threshold, the image in the layer being analyzed is black. The threshold is a pre-set value, typically greater than 0 and less than 1. In practice, the threshold is typically smaller, such as 0.1.

[0189] It should be noted that since the data volume of the image layer to be analyzed is usually very large, that is, the amount of calculation required to determine whether the image layer is black is very large. Since the GPU has flexible and efficient graphics and image data processing capabilities, using the GPU to determine whether the image layer is black can ensure the efficiency and accuracy of black image determination, thereby ensuring the efficiency and accuracy of layer synthesis.

[0190] Considering that images typically contain a large number of pixels, directly obtaining the pixel values of all pixels in the image layer to be analyzed and performing calculations based on these values may consume a significant amount of computing resources and easily reduce the overall data processing efficiency of the electronic device. Therefore, in an embodiment of the present application, the GPU can first reduce the image layer to be analyzed to a certain set size, such as 30×40, and then perform black image analysis based on the pixel values of each pixel in the reduced image layer to be analyzed, which can significantly reduce the amount of computation required.

[0191] Further references Figure 5C , Figure 5C A flowchart of a GPU performing black image determination on an image of a layer to be analyzed provided in an embodiment of the present application. Figure 5C In the process, the GPU can perform black image determination on the image of the layer to be analyzed by executing the following operations S5041 to S5045. Figure 5C In the process, the GPU can first reduce the size of the image of the layer to be analyzed, and indirectly determine whether the image of the layer to be analyzed is a black image by determining whether the image after the size reduction is a black image.

[0192] S5041: The GPU reduces the image of the layer to be analyzed to a preset size to obtain the target analysis layer image.

[0193] The target analysis layer image is the image obtained by reducing the image of the target analysis layer.

[0194] The preset size is usually a pre-set size, for example, 20×10. In practice, the preset size is usually smaller.

[0195] Optionally, the GPU can scale the image of the layer to be analyzed down to the preset size by calling the drawImageRect function in Canvas. The drawImageRect function is used to draw an image onto the canvas. Specifically, if the original image is larger than the canvas size, the original image is scaled down and drawn onto the canvas. If the original image is smaller than the canvas size, the original image is scaled up and drawn onto the canvas.

[0196] S5042: The GPU extracts pixel data of each pixel in the target analysis layer image to obtain a pixel value of each pixel.

[0197] In practice, pixel data is typically color data. This color data can be red, green, and blue (RGB) data, which includes color values for three channels: R, G, and B. For each pixel, the GPU can determine the average of these three color values as the pixel value.

[0198] Optionally, the GPU can extract the color data of each pixel in the target analysis layer image by calling the readPixels function in Canvas. The readPixels function is a function in Canvas that reads pixel data from the canvas.

[0199] S5043: The GPU calculates the average pixel value of the target analysis layer image.

[0200] Here, the GPU may divide the sum of the pixel values of all pixels in the target analysis layer image by the total number of pixels in the target analysis layer image to calculate the average pixel value of the target analysis layer image.

[0201] At S5044, the GPU normalizes the average pixel value to obtain a normalized pixel value.

[0202] The normalized pixel value is typically a value obtained by dividing the average pixel value obtained in step S5033 by the maximum pixel value in the target analysis layer image. The normalized pixel value is typically greater than 0 and less than 1.

[0203] S5045: The GPU determines whether the image of the layer to be analyzed is a black image based on the normalized pixel values.

[0204] In practice, the GPU can use the above-mentioned normalized pixel value to compare with a pre-set normalized pixel threshold, and based on the comparison result, determine whether the target analysis layer image is black, and thus determine whether the image of the layer to be analyzed is black. For example, if the normalized pixel value is greater than or equal to the normalized pixel threshold, it means that the image of the layer to be analyzed is not black. If the normalized pixel value is less than the normalized pixel threshold, it means that the image of the layer to be analyzed is black.

[0205] The normalized pixel threshold is usually a preset value, which is greater than 0 and less than 1.

[0206] Since the color data of each pixel in the black image is usually (0, 0, 0), that is, the R value, G value, and B value are usually 0. Therefore, the normalized pixel threshold is usually close to 0, for example, it can be 0.1.

[0207] It is understandable that when the target application has multiple layers, for the layer image to be analyzed corresponding to each layer, the GPU can perform black image judgment on the layer image to be analyzed by executing the above operations S5041-S5045.

[0208] In the embodiment of the present application, when the GPU performs black image judgment on the target analysis layer image, the target analysis layer image is first reduced to a relatively small size, and then the pixel data of each pixel point in the small-sized target analysis layer image is analyzed to determine whether the target analysis layer image is a black image, thereby determining whether the target analysis layer image is a black image. This can greatly reduce the amount of data processing, thereby further improving the efficiency of black image judgment.

[0209] S505: The GPU performs layer synthesis based on the black image judgment result.

[0210] The black image determination result indicates whether the image layer being analyzed is black. For example, the black image determination result can be "1," indicating that the image layer being analyzed is black, or "not-black," indicating that the image layer being analyzed is not black. It is understood that the embodiments of this application do not specifically limit the implementation of the black image determination result.

[0211] Optionally, if the black image determination result indicates that the image of the layer to be analyzed is black, the GPU can use the cached layer image pre-cached on the GPU and other layers to be displayed, such as the status bar layer, to perform layer compositing. The other layers are layers to be displayed other than the layer corresponding to the target application.

[0212] Optionally, when the black image determination result indicates that the image of the layer to be analyzed is not a black image, the GPU may perform layer synthesis using the image of the layer to be analyzed and other layers to be displayed.

[0213] It can be understood that the GPU can synthesize the layer images of each layer according to the layer properties of each layer.

[0214] For example, when the target application has one layer corresponding to layer 1, and there are two other layers, namely layer 2 and layer 3, before performing layer synthesis, the GPU can first determine whether the layer image 1 to be analyzed corresponding to layer 1 is a black image; then, the GPU can perform layer synthesis based on the black image judgment result of the layer image 1 to be analyzed. Specifically, if the layer image 1 to be analyzed is a black image, the GPU can use the cached layer image cached in the GPU, the layer image corresponding to layer 2, the layer image corresponding to layer 3, and the layer properties of each layer to perform synthesis on the three layers. If the layer image 1 to be analyzed is not a black image, the GPU can use the layer image 1 to be analyzed corresponding to layer 1, the layer image corresponding to layer 2, the layer image corresponding to layer 3, and the layer properties corresponding to each layer to perform synthesis on the three layers.

[0215] In an embodiment of the present application, when the image of the layer to be analyzed obtained by the GPU in real time is not a black image, the GPU uses the obtained image of the layer to be analyzed to perform layer synthesis; when the image of the layer to be analyzed obtained by the GPU in real time is a black image, the GPU uses the pre-cached cached layer image to perform layer synthesis, which can avoid the application interface flashing black during the display process while ensuring the accuracy of layer synthesis.

[0216] In some optional implementations of the embodiments of the present application, when the black image judgment result indicates that the layer image to be analyzed is not a black image, the GPU can switch the above-mentioned cached layer image to the layer image to be analyzed. For example, the cached layer image that has been cached in the GPU can be deleted first, and then the layer image to be analyzed can be cached in the GPU as a new cached layer image. In this way, on the one hand, when the layer image to be analyzed is a black image at the next moment, the cached layer image can be used for layer synthesis to avoid the electronic device flashing black during the display process. On the other hand, the GPU can cache the latest layer image, which helps to ensure the accuracy of layer synthesis using the cached layer image, that is, it can ensure that the display interface obtained by layer synthesis using the cached layer image is more consistent with the change between the display interface at the previous moment, which helps to further improve the user experience.

[0217] Optionally, when the electronic device detects that the first operation is a slide-out operation, Figure 3 Since the content of the application interface typically remains unchanged during the first phase of the slide-out operation, there's generally no need to update the cached layer image in the GPU. This saves computing resources consumed by updating the GPU cache, further improving the data processing efficiency of the electronic device.

[0218] S506 , SurfaceFlinger responds to the slide-out operation in the second operation phase and sends a motion effect start indication message to the GPU.

[0219] The animation start indication information is used to instruct the target application to start executing the application exit animation. The target application starts executing the application exit animation, that is, the electronic device starts displaying the application exit animation.

[0220] The second operation phase occurs when the electronic device detects that the first operation gesture has left the electronic screen but has not yet finished displaying the animation effect corresponding to the first operation. The application exit animation refers to the animation effect when the target application exits. A swipe-to-exit operation in the second operation phase typically indicates that the swipe-to-exit operation has concluded, meaning that the target application needs to execute the application exit animation.

[0221] In the embodiment of this application, Figure 5A , when the electronic device detects an exit gesture (such as Figure 3When the side-swipe gesture operation (also called the side-swipe exit operation) ends, the input system in the electronic device can determine that the side-swipe exit operation has ended by analyzing the touch information. As an example, if the touch information indicates that the electronic device detects that the operation gesture of the first operation has been applied to the electronic screen and then left the electronic screen, the first operation is considered to be completed, that is, the exit gesture has ended.

[0222] In practice, the electronic device may determine that a slide-out operation in the second operation phase has been detected upon just detecting that the slide-out operation has been completed. When the electronic device detects the slide-out operation in the second operation phase, the WMS service in the electronic device may send the above-mentioned animation effect activation indication information to SurfaceFlinger. SurfaceFlinger may send the above-mentioned animation effect activation indication information to the GPU upon receiving the above-mentioned animation effect activation indication information.

[0223] In some optional implementations, the GPU may actively switch the value of the first tag to the second value upon receiving the motion effect activation instruction information.

[0224] In some other optional implementations, SurfaceFlinger can send a first tag whose corresponding value is the second value to the GPU while sending the animation effect activation indication information to the GPU, so as to trigger the GPU to switch the value of the first tag to the second value.

[0225] S507 , while the target application is executing the application exit animation, the GPU continues to perform black image determination on the received layer image to be analyzed, and performs layer synthesis based on the black image determination result.

[0226] It can be understood that the process of the target application executing the application exit animation is also the process of the electronic device displaying the application exit animation.

[0227] Here, when the target application executes the application exit animation, the GPU can perform the above-mentioned S504-S505 operations to perform black image judgment on the layer image to be analyzed received in real time, and perform layer synthesis based on the black image judgment result, which will not be elaborated here.

[0228] S508 , when the target application is executing the application exit animation, if the layer image to be analyzed is black, the GPU switches the cached layer image in the GPU cache to the received layer image to be analyzed.

[0229] In the embodiment of the present application, since the target application is executing the application exit animation, the application interface is scaled first, then the application interface and the application icon are scaled together, and finally the application icon is scaled, that is, during the target application executing the application exit animation, the layer images of each layer corresponding to the target application may change in real time. During the target application executing the application exit animation, the GPU promptly updates the layer image (or cached layer image) in the GPU cache when the layer image to be analyzed is not a black image. That is, the GPU can cache the latest layer image, which helps to ensure the accuracy of layer synthesis using the cached layer image. That is, it can ensure that the display interface obtained by layer synthesis using the cached layer image is more consistent with the changes between the display interface at the previous moment, which helps to further improve the user experience.

[0230] S509: When the target application finishes executing the application exit animation, SurfaceFlinger switches the layer synthesis mode to the normal synthesis mode.

[0231] It can be understood that the above-mentioned target application ends executing the application exit animation, which means that the electronic device ends displaying the application exit animation.

[0232] The conventional synthesis mode at least includes HWC performing layer synthesis. It should be noted that the conventional synthesis mode is generally a layer synthesis method used in the prior art. In the conventional synthesis mode, layer synthesis can be performed with reference to the existing synthesis method, which will not be described in detail here.

[0233] In practice, when the target application finishes executing the application exit animation, the process managing the application usually sends a message to the WMS service indicating that the target application has exited. After receiving the message indicating that the target application has exited, the WMS service can send an animation end indication message to SurfaceFlinger.

[0234] The end-of-action indication information is used to indicate that the target application has finished executing the application exit action. It is understandable that the embodiment of the present application does not limit the implementation form of the end-of-action indication information. As an example, the end-of-action indication information can be "finished".

[0235] In an embodiment of the present application, when SurfaceFlinger receives the motion effect end indication information, it can switch the GPU synthesis mode to the normal synthesis mode.

[0236] In normal synthesis mode, the GPU and HWC can jointly synthesize the layers corresponding to the target application. For example, if the target application corresponds to three layers, namely layer 1, layer 2 and layer 3, in addition to the three layers corresponding to the target application, there are two layers, such as layer 4 and layer 5, which need to be synthesized. At this time, the GPU can synthesize layer 1, layer 2 and layer 4 to obtain a preliminary synthesized layer; after that, the GPU sends the preliminary synthesized layer to the HWC, and the HWC further synthesizes layer 3, layer 5 and the above-mentioned preliminary synthesized layer synthesized by the GPU to obtain the final synthesized layer. It can be understood that the final synthesized layer can be processed by hardware display and pushed to the electronic screen for display.

[0237] It is understandable that in the conventional synthesis mode, layer synthesis can be performed with reference to existing synthesis methods. This embodiment does not specifically limit the layer synthesis process in the conventional synthesis mode.

[0238] In an embodiment of the present application, the target application finishes executing the application exit animation, that is, the electronic device finishes displaying the application exit animation, indicating that the electronic device has completed the response to the slide exit operation. At this time, SurfaceFlinger can switch the layer synthesis mode from the second synthesis mode (also called GPU synthesis mode) back to the first synthesis mode (also called regular synthesis mode). In regular synthesis mode, the GPU and HWC collaborate to perform layer synthesis, which can reduce the data processing capacity of the GPU. Since the GPU can perform layer synthesis and can also be used to complete other graphics processing tasks, the GPU and HWC collaborate to perform layer synthesis, which can reduce the data processing capacity of the GPU, thereby improving the overall data processing efficiency of the electronic device.

[0239] S510, SurfaceFlinger sends motion effect end indication information to the GPU.

[0240] Here, when SurfaceFlinger receives the motion effect end indication information, it can send the motion effect end indication information to the GPU to prompt the GPU to end the application exit motion effect for the target application.

[0241] In some optional implementations of the present application, the animation start indication information may be a second tag with a corresponding value of the third value, and the animation end indication information may be a second tag with a corresponding value of the fourth value.

[0242] The second tag is used to instruct the target application to start or stop executing the application exit animation. The value of the second tag may include a third value and a fourth value. The third value indicates that the target application starts executing the application exit animation. The fourth value indicates that the target application ends executing the application exit animation.

[0243] S511: The GPU releases the GPU cache used to cache the layer image.

[0244] Here, the GPU can delete the cached layer image for the target application and release the storage space used to cache the cached layer image. The GPU promptly releases the GPU cache used to cache layer images, which helps improve the utilization of storage resources in the GPU.

[0245] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0246] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0247] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0248] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0249] In addition, in the description of this application specification and the appended claims, the terms "first", "second", "third" etc. are only used to distinguish descriptions, and cannot be interpreted as indicating or suggesting relative importance. It should also be understood that although the terms "first", "second" etc. are used to describe various elements in some embodiments of the present application in the text, these elements should not be restricted by these terms. These terms are only used to distinguish an element from another element. For example, the first status code can be named as the second status code, and similarly, the second status code can be named as the first status code, without departing from the scope of the various described embodiments. The first status code and the second status code are both status codes, but they are not the same status code.

[0250] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0251] Figure 6 It is a structural diagram of an electronic device provided in one embodiment of the present application.

[0252] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a camera 193, a display screen 194, and a SIM card interface 195, etc.

[0253] The sensor module 180 may include a four-in-one (accelerometer, angular velocity meter, gyroscope, compass) motion sensor, a pressure sensor, a touch sensor, a temperature sensor, a light sensor, a proximity sensor, a 3D structured light sensor, and the like. Different sensors are used to sense different signals, allowing the processor 110 to perform operations based on different sensing signals to implement different functions of the electronic device. For example, a four-in-one motion sensor can be used to determine the motion posture and orientation information of the electronic device; a light sensor is used to sense the ambient light brightness.

[0254] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a GPU, a HWC, an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. Among them, the controller can be the nerve center and command center of the electronic device. The controller can generate an operation control signal based on the instruction opcode and the timing signal to complete the control of instruction fetching and execution.

[0255] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0256] Processor 110 can execute the layer synthesis method provided in the embodiments of the present application to facilitate quick and accurate schedule creation and enhance the user experience. Processor 110 can include different components. For example, when integrated with a CPU and a GPU, the CPU and GPU can cooperate to execute the layer synthesis method provided in the embodiments of the present application. For example, some algorithms in the layer synthesis method can be executed by the CPU, while others can be executed by the GPU, to achieve higher processing efficiency.

[0257] It should be understood that the illustrated electronic device is merely an example, and that the electronic device may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0258] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device.

[0259] In addition, those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0260] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0261] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps of the above-mentioned method embodiments when executing the computer program product.

[0262] An embodiment of the present application also provides a chip system, which includes a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the steps in the above-mentioned method embodiments.

[0263] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. Computer-readable storage media may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0264] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0265] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0266] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0267] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A layer synthesis method, characterized in that: Applied to electronic equipment, the method includes: In response to a first user operation, switching the layer composition mode from the first composition mode to the second composition mode, and storing a first layer image of the target application into a target cache, wherein the first layer image is a layer image of the layer corresponding to the target application immediately before the first operation begins, wherein the first composition mode is performed by at least a hardware compositor HWC, and the second composition mode is performed by a graphics processor GPU; In the case of the second synthesis mode, the first animation effect of the target application is displayed, and the first animation effect includes a first image frame. The GPU performs a black image judgment on the layer image to be analyzed in the animation layer data corresponding to the first image frame, and obtains the layer image to be analyzed as a black image, wherein the animation layer data is the layer data of the target application at any moment between the first moment and the second moment, and the animation layer data includes layer attributes and the layer image to be analyzed. The first image frame is obtained by the GPU performing layer synthesis on each layer to be synthesized based on the black image judgment result and the first layer image and the layer attributes in the target cache. The first moment is the moment when the electronic device detects that the first operation begins to act on the electronic screen, and the second moment is the moment when the electronic device ends displaying the first animation effect.

2. The layer synthesis method according to claim 1, characterized in that: The first operation is a side-slide gesture operation, and the side-slide gesture operation is used to exit the target application; the first animation effect is an exit animation effect.

3. The layer synthesis method according to claim 2, characterized in that: The step of switching the layer composition mode from the first composition mode to the second composition mode in response to the first user operation, and storing the first layer image of the target application into the target cache, includes: In response to the side-slide gesture operation in the first operation stage, the layer synthesis mode is switched from the first synthesis mode to the second synthesis mode, and the first layer image of the target application is stored in the target cache, wherein the first operation stage is the stage from the first moment to the third moment, and the third moment is the moment when the electronic device detects that the first operation stops acting on the electronic screen.

4. The layer synthesis method according to claim 2, characterized in that: The displaying of the first animation effect of the target application includes: In response to the sidesliding gesture operation in the second operation stage, displaying the exit animation effect, wherein the second operation stage is the stage from the third moment to the second moment; In the process of displaying the exit animation effect, when the layer image to be analyzed in the acquired animation layer data is a black image, the first layer image in the target cache is switched to the acquired layer image to be analyzed.

5. The layer synthesis method according to any one of claims 1 to 4, characterized in that: Before the GPU performs layer synthesis on each layer to be synthesized based on the first layer image and the layer properties in the target buffer, the method further includes: Based on the pixel data of each pixel point in the image of the layer to be analyzed, a black image determination is performed on the image of the layer to be analyzed.

6. The layer synthesis method according to claim 5, characterized in that: The target cache is a GPU cache, and the black image determination on the image of the layer to be analyzed is performed by the GPU, and the black image determination on the image of the layer to be analyzed by the GPU includes: Reducing the image of the layer to be analyzed to a preset size to obtain an image of the target analysis layer; Determining a normalized pixel value of the target analysis layer image, wherein the normalized pixel value is a ratio of an average pixel value to a maximum pixel value of the target analysis layer image; When the normalized pixel value is less than a preset normalized pixel threshold, determining that the image of the layer to be analyzed is a black image; When the normalized pixel value is greater than or equal to the normalized pixel threshold, it is determined that the image of the layer to be analyzed is not a black image.

7. The layer synthesis method according to any one of claims 1 to 4, characterized in that: After the first animation effect is displayed, the method further includes: The layer composition mode is switched from the second composition mode to the first composition mode.

8. The layer synthesis method according to any one of claims 1 to 4, characterized in that: After the first animation effect is displayed, the method further includes: Release the target buffer.

9. The layer synthesis method according to any one of claims 1 to 4, characterized in that: The first animation effect also includes a second image frame. The image of the layer to be analyzed in the animation layer data corresponding to the second image frame is not a black image. The second image frame is obtained by the GPU performing layer synthesis on the required synthesized layers based on the image of the layer to be analyzed and the layer properties.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the layer synthesis method according to any one of claims 1 to 9 is implemented.

11. A chip system, characterized in that: The chip system includes a processor coupled to a memory, and the processor executes a computer program stored in the memory to implement the layer synthesis method according to any one of claims 1 to 9.

Images