Layer composition method and electronic device

Abstract

Embodiments of the present application are suitable for the field of data processing technology, and provide a layer synthesis method and an electronic device. Whether a current frame image meets a preset condition is determined first, and the preset condition includes that the current frame image includes a first layer. When the current frame image includes the first layer, the current frame image is subjected to blur processing. If the current frame image meets the preset condition, a first strategy is used for layer synthesis, that is, if the current frame image is a first image, a blur layer corresponding to other layers in a previous frame image and the first layer are used for layer synthesis to generate a to-be-displayed image, and then the to-be-displayed image is displayed. In this way, a step of performing blur processing on other layers by a GPU is omitted, and the to-be-displayed image can be obtained only by calling the blur layer corresponding to the other layers in the previous frame image and the first layer for layer synthesis, thereby avoiding performance problems such as display process freezing or frame loss caused by excessive load of the GPU.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a layer composition method and an electronic device. Background Technology

[0002] To enhance data security when users use electronic devices and improve their visual experience, the interface of an electronic device can be blurred in full screen when possible.

[0003] In one possible scenario, electronic devices send each frame of the displayed image to the graphics processing unit (GPU). The GPU then blurs each frame before sending the blurred image to the display screen. Since electronic devices typically have high refresh rates—for example, 60Hz (60 frames per second)—the GPU needs to blur 60 frames per second, significantly consuming its resources. This can lead to performance issues such as stuttering or dropped frames, as well as excessive GPU power consumption.

[0004] Therefore, how to reduce the GPU load and avoid performance problems during the display process when electronic devices display in full-screen blur has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a layer compositing method that can reduce the GPU load when an electronic device displays a blurred full-screen image, avoid performance problems during the display process, and reduce the GPU's power consumption.

[0006] Firstly, a layer compositing method is provided, including:

[0007] Determine whether the current frame image meets the preset conditions. The preset conditions include that the current frame image includes the first layer. When the current frame image includes the first layer, the current frame image is blurred.

[0008] If the current frame image meets the preset conditions, the first strategy is used for layer compositing. The first strategy means that when performing layer compositing, if the current frame image is the first image, the first blurred layer and the first layer are used for layer compositing to generate the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to the layers in the previous frame image other than the first layer. The first image refers to the image that has a corresponding first blurred layer.

[0009] Display the image to be displayed.

[0010] The layer parameter acquisition submodule within the layer compositing module can collect layer-related parameters. For example, the layer acquisition submodule can collect the layer identification information in the image and determine whether the first layer exists in the current frame image based on the layer identification information. For instance, if the layer parameter acquisition submodule obtains the identification information of the power-off layer, it can determine that the current frame image includes the power-off layer, which is the first layer.

[0011] The layer compositing method provided in this application first determines whether the current frame image meets preset conditions. The preset conditions include that the current frame image includes a first layer. When the current frame image includes a first layer, the current frame image is blurred. If the current frame image meets the preset conditions, a first strategy is adopted for layer compositing. The first strategy means that when performing layer compositing, if the current frame image is the first image, a first blurred layer and the first layer are used for layer compositing to generate the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to layers in the previous frame image other than the first layer. The first image refers to the image with a corresponding first blurred layer. Then the image to be displayed is displayed. This is equivalent to omitting the step of the GPU blurring other layers. It is only necessary to call the blurred layer corresponding to other layers in the previous frame image and the first layer for layer compositing to obtain the image to be displayed, thereby reducing the GPU load and avoiding performance problems such as stuttering or frame dropping caused by excessive GPU load.

[0012] In conjunction with the first aspect, in some embodiments of the first aspect, the first strategy further includes: if the current frame image is a first image, and the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, performing layer compositing using the first blurred layer and the first layer to generate an image to be displayed.

[0013] The layer compositing method provided in this application employs a first strategy to merge the first blurred layer and the first layer of the current frame image when the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image. This avoids the problem that the image to be displayed after layer compositing is significantly different from the current frame image due to the large difference between the previous frame image and the current frame image.

[0014] In conjunction with the first aspect, in some embodiments of the first aspect, the first strategy further includes: if the current frame image is a first image and the number of first layers included in the current frame image is 1, then layer compositing is performed using the first blurred layer and the first layer to generate an image to be displayed.

[0015] The first image can be referred to as the skip frame image.

[0016] If the number of first layers in the current frame image is 0, or if the number of first layers in the current frame image is greater than or equal to 2, then the first strategy for layer compositing is abandoned, which is equivalent to the current frame image being the solution frame image that is turned off.

[0017] In conjunction with the first aspect, in some embodiments of the first aspect, the first strategy further includes: if the current frame image is a second image, blurring other layers in the current frame image to generate a first blurred layer and storing the first blurred layer, and the second image indicating an image that does not have a corresponding first blurred layer.

[0018] The second image can be referred to as the copy frame image.

[0019] In conjunction with the first aspect, in some embodiments of the first aspect, the first strategy further includes: if the current frame image is a second image, blurring other layers in the current frame image to generate a first blurred layer and storing the first blurred layer, performing layer compositing processing on the first blurred layer and the first layer in the second image to generate an image to be displayed.

[0020] It should be understood that when starting layer compositing using the first strategy, images are typically sorted in a frame-by-frame order, with one copy frame immediately preceding another. For example... Figure 10 As shown.

[0021] The layer compositing method provided in this application embodiment, if the current frame image is a second image, blurs other layers in the second image to generate a first blurred layer, and then performs layer compositing on the first blurred layer and the first layer in the second image to generate an image to be displayed. This avoids the situation where the second image does not generate an image to be displayed. That is, not only is the first layer in the first image merged to obtain the image to be displayed, but the first layer in the second image is also merged to obtain the image to be displayed. This is equivalent to the first layer in the image to be displayed after each frame is layer composited being the updated first layer, avoiding the situation where two adjacent frames are layered using the same first layer and the first blurred layer, thereby avoiding the situation where two adjacent frames are displayed the same, and improving the display effect of electronic devices.

[0022] In conjunction with the first aspect, in some embodiments of the first aspect, the first strategy further includes: if the current frame image is the first image, performing layer compositing of the first blurred layer and the first layer using a hardware synthesizer (HWC) to generate an image to be displayed.

[0023] The layer compositing method provided in this application uses HWC instead of GPU to perform layer merging processing on the first layer and the first blurred layer. This can further reduce the GPU load and thus further avoid performance problems such as display stuttering or frame dropping caused by excessive GPU load.

[0024] In conjunction with the first aspect, in some embodiments of the first aspect, the first layer is the top layer in the current frame image.

[0025] Among them, the power-off layer and the drop-down window layer are usually displayed at the top level, that is, the top layer in the current frame image.

[0026] In conjunction with the first aspect, in some embodiments of the first aspect, the aforementioned preset conditions further include the inclusion of a first layer in the previous frame image.

[0027] In conjunction with the first aspect, in some embodiments of the first aspect, the aforementioned preset condition further includes that the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0028] The layer parameter acquisition submodule within the layer compositing module can collect layer-related parameters. For example, if the size of the first layer in the current frame image has changed compared to the first layer in the previous frame image, the parameter indicating the size change is 1; if the size of the first layer in the current frame image has not changed compared to the first layer in the previous frame image, the parameter indicating the size change is 0. If the parameter indicating the size change of the layer obtained by the layer parameter acquisition submodule is 0, then the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0029] The layer compositing method provided in this application determines whether to use the first strategy for layer compositing if the size of the first layer in the current frame image and the size of the first layer in the previous frame image are the same. If the size of the first layer in the current frame image and the size of the first layer in the previous frame image are the same, it means that the current frame image is a stable display of the first layer, rather than the first time the first layer is displayed. This can avoid the situation where the display effect of the current frame image is not good due to the reuse of the first blurred layer of the previous frame image when the first layer is displayed for the first time.

[0030] In conjunction with the first aspect, in some embodiments of the first aspect, the aforementioned preset condition further includes: the size of the first layer in the current frame image is the same as the size of the first layer in N previous frame images.

[0031] It should be noted that N previous frame images refer to the N images preceding the current frame image. If the size of the first layer in the current frame image is the same as the size of the first layer in the previous N images, it means that the current frame image is displaying the first layer image stably, rather than during the initial display of the first layer image where the size of the first layer image is fine-tuned.

[0032] The layer compositing method provided in this application determines whether to use the first strategy for layer compositing if the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images. If the size of the first layer in the current frame image is the same as the size of the first layer in the previous N images, it means that the current frame image is displaying the first layer image stably, rather than during the initial display of the first layer, when the size of the first layer image needs to be fine-tuned. This avoids the situation where the display effect of the current frame image is poor due to the reuse of the first blurred layer of the previous frame image when fine-tuning the first layer.

[0033] In conjunction with the first aspect, in some embodiments of the first aspect, the image to be displayed is a dynamically displayed image.

[0034] It should be understood that the generated image to be displayed is a dynamically displayed image. Therefore, during the layer compositing process, the layer compositing is performed according to the refresh rate of the electronic device. Compared with static images, dynamically displayed images undergo more layer compositing and consume more system resources. The layer compositing method provided in this application embodiment can effectively reduce the system resources consumed when blurring dynamically displayed images.

[0035] Secondly, a layer compositing apparatus is provided, including a unit for performing any of the methods in the first aspect. The apparatus may be a server, a terminal device, or a chip within a terminal device. The apparatus may include an input unit and a processing unit.

[0036] When the device is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may also include a memory for storing computer program code, which, when the processor executes the computer program code stored in the memory, causes the terminal device to perform any of the methods in the first aspect.

[0037] When the device is a chip within a terminal device, the processing unit can be an internal processing unit of the chip, and the input unit can be an output interface, pin, or circuit, etc.; the chip may also include a memory, which can be an internal memory of the chip (e.g., a register, cache, etc.) or an external memory (e.g., a read-only memory, random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip performs any of the methods in the first aspect.

[0038] In one possible implementation, the memory is used to store computer program code; the processor executes the computer program code stored in the memory. When the computer program code stored in the memory is executed, the processor performs the following: determining whether the current frame image meets preset conditions, the preset conditions including that the current frame image includes a first layer, wherein, if the current frame image includes a first layer, the current frame image is blurred; if the current frame image meets the preset conditions, a first strategy is used for layer compositing, the first strategy being that, when performing layer compositing, if the current frame image is a first image, a first blurred layer and a first layer are used for layer compositing to generate an image to be displayed, the first blurred layer being a blurred layer corresponding to other layers in the previous frame image, other layers being layers in the previous frame image other than the first layer, and the first image being an image with a corresponding first blurred layer; and displaying the image to be displayed.

[0039] Thirdly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer program code, which, when run by a layer compositing apparatus, causes the layer compositing apparatus to perform any of the layer compositing methods in the first aspect.

[0040] Fourthly, a computer program product is provided, the computer program product comprising: computer program code, which, when run by a layer compositing apparatus, causes the layer compositing apparatus to perform any of the apparatus methods in the first aspect.

[0041] The layer compositing method and electronic device provided in this application, before performing layer compositing, first determine whether the current frame image meets preset conditions. The preset conditions include that the current frame image includes a first layer. When the current frame image includes a first layer, the current frame image is blurred. If the current frame image meets the preset conditions, a first strategy is adopted for layer compositing. The first strategy means that when performing layer compositing, if the current frame image is the first image, a first blurred layer and the first layer are used for layer compositing to generate the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to layers in the previous frame image other than the first layer. The first image refers to the image with a corresponding first blurred layer. Then the image to be displayed is displayed. This is equivalent to omitting the step of the GPU blurring other layers. It is only necessary to call the blurred layer corresponding to other layers in the previous frame image and the first layer for layer compositing to obtain the image to be displayed, thereby reducing the GPU load and avoiding performance problems such as display stuttering or frame dropping caused by excessive GPU load. At the same time, it reduces the GPU's performance. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of a dynamic, blurred interface;

[0043] Figure 2 This is a flowchart illustrating the process of layer compositing;

[0044] Figure 3 This is a schematic diagram of a hardware system for an electronic device applicable to this application;

[0045] Figure 4 This is a schematic diagram of a software system applicable to an electronic device of this application;

[0046] Figure 5 This is a schematic flowchart of a layer compositing method provided in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram illustrating a layer arrangement order provided in an embodiment of this application;

[0048] Figure 7 This is a diagram illustrating the time required for layer compositing using HWC and GPU.

[0049] Figure 8 This is a flowchart illustrating another layer compositing method provided in an embodiment of this application;

[0050] Figure 9 This is a schematic diagram of the process of layer composition by skipping frames in an embodiment of this application;

[0051] Figure 10This is a schematic diagram illustrating the arrangement order of different frames in an embodiment of this application;

[0052] Figure 11 This is a schematic diagram of the process of layer compositing by copying frames in an embodiment of this application;

[0053] Figure 12 This is a diagram illustrating the time taken for blurring during layer processing;

[0054] Figure 13 This is a flowchart illustrating another layer compositing method provided in an embodiment of this application;

[0055] Figure 14 This is a schematic diagram of a layer compositing device provided in this application;

[0056] Figure 15 This is a schematic diagram of an electronic device for image processing provided in this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0058] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0059] For ease of understanding, the examples provided are for reference only and are related to the concepts in the embodiments of this application.

[0060] 1. Real-time blurring.

[0061] To enhance data security and improve the visual experience for users when using electronic devices, the display interface can be blurred to hide the content. This type of blurred display is called a blurred scene.

[0062] Blur processing can refer to adding a mask layer to the top of an image to blur the content displayed within it. Understandably, blur processing is typically performed on the GPU.

[0063] Blurring of the display interface can be divided into static blurring and dynamic blurring. Static blurring refers to blurring a single frame of an image; the electronic device only displays that blurred frame and does not display subsequent blurred frames over time. Dynamic blurring, also known as real-time blurring, refers to blurring every frame of the electronic device's image, with the device displaying different blurred images as time progresses. Compared to static blurring, real-time blurring consumes a significant amount of GPU memory, impacting the GPU's normal image processing capabilities.

[0064] For example, such as Figure 1 As shown, the wallpaper displayed on the shutdown interface 10 is a live wallpaper. In this case, the shutdown interface 10 includes a shutdown layer and a wallpaper layer. The shutdown layer includes a shutdown icon 11 and a restart icon 12. It is understood that the shutdown layer may also include more components to display more information, such as a date component (not shown in the figure) displaying the date, or text prompting the user to force a restart. This embodiment does not limit this.

[0065] It should be understood that the wallpaper displayed on the lock screen 10 is a live wallpaper; for example, the heart in the wallpaper can shrink and move. These dynamic effects are not shown in the illustration.

[0066] 2. Full-screen blur.

[0067] Full-screen blur refers to the entire display interface being displayed in a blurry state, for example, as shown in the image. Figure 1 The shutdown screen 10 is shown. It can be understood that in a full-screen, blurry display, the topmost layer can be clearly displayed, while the wallpaper layer occupies the entire display area and is displayed blurry. For example, Figure 1 The shutdown layer, consisting of the shutdown icon 11 and the restart icon 12, is clearly displayed without any blurring. The wallpaper layer below the shutdown layer is blurred.

[0068] 3. Frame.

[0069] A frame refers to a single, still image, the smallest unit in a user interface display. A frame can be understood as a still image; rapidly displaying multiple consecutive frames creates the illusion of motion. Frame rate refers to the number of frames refreshed per second, or the number of times the graphics processor in a terminal device refreshes the screen per second. A higher frame rate results in smoother and more realistic animation. The more frames per second, the smoother the displayed motion.

[0070] Currently, to enhance data security and improve the visual experience for users of electronic devices, full-screen blurring can be applied to the display interface. It should be understood that each frame is typically composed of multiple layers, for example... Figure 2 As shown, the displayed screen is a composite of wallpaper layer 10, desktop control layer 20, and dropdown window layer 30. Typically, the GPU can composite these multiple layers to generate the displayed image. In one possible scenario, the GPU can add a mask layer 40 during layer compositing to blur the resulting image. The mask layer 40 can refer to the icon attached to wallpaper layer 10 and desktop control layer 20; its properties are not layer properties. Figure 2 As shown, a mask layer 40 is added above the wallpaper layer 10 and desktop control layer 20, and below the dropdown window layer 30, to blur the area below the dropdown window layer 30 in the composited display. As users' demands for display interfaces increase, the refresh rate of electronic devices also increases. Currently, the refresh rate of electronic devices is 60Hz, meaning 60 frames per second need to be displayed. In this situation, the GPU needs to blur 60 frames per second, significantly consuming GPU resources. The GPU cannot perform layer compositing and blurring in a timely manner, leading to stuttering or frame drops during display, as well as excessive GPU power consumption.

[0071] In view of this, the layer compositing method provided in this application first determines whether the current frame image meets preset conditions. The preset conditions include that the current frame image includes a first layer. When the current frame image includes a first layer, the current frame image is blurred. If the current frame image meets the preset conditions, a first strategy is adopted for layer compositing. The first strategy means that when performing layer compositing, if the current frame image is the first image, a first blurred layer and the first layer are used for layer compositing to generate the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to layers in the previous frame image other than the first layer. The first image refers to the image with a corresponding first blurred layer. Then the image to be displayed is displayed. This is equivalent to omitting the step of the GPU blurring other layers. It is only necessary to call the blurred layer corresponding to other layers in the previous frame image and the first layer for layer compositing to obtain the image to be displayed, thereby reducing the GPU load and avoiding performance problems such as display stuttering or frame dropping caused by excessive GPU load, and reducing GPU power consumption.

[0072] The layer composition method provided in this application can be applied to electronic devices. Optionally, the electronic device includes a terminal device, which can also be called a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device can be a mobile phone, smart TV, wearable device, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc. The embodiments of this application do not limit the specific technology or device form used in the terminal device.

[0073] For example, Figure 3 A schematic diagram of the structure of electronic device 100 is shown. Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0074] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0075] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0076] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0077] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0078] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0079] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0080] In one possible scenario, the GPU can composite multiple layers to generate an image to be displayed. Furthermore, the GPU can add a blur layer during layer compositing to create a blurred image, improving rendering efficiency. Additionally, the image can be blurred further to obtain a blurred image, which is then sent to the display.

[0081] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0082] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0083] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0084] It should be noted that any electronic device mentioned in the embodiments of this application may include more or fewer modules in electronic device 100.

[0085] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0086] Figure 4 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0087] The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0088] The application layer can include a series of application packages.

[0089] like Figure 4 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0090] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0091] Applications can also typically perform image rendering on the image to be displayed, generate multiple layer data, and send the generated multiple layer data to the layer compositing module in the application framework layer.

[0092] like Figure 4 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, layer composition module, etc.

[0093] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0094] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0095] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0096] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0097] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0098] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0099] The layer compositing module is used to combine multiple layers obtained after the application renders an image to obtain the image to be displayed.

[0100] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0101] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0102] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0103] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0104] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0105] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0106] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0107] A 2D graphics engine is a graphics engine for 2D drawing.

[0108] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, Wi-Fi drivers, etc.

[0109] It should be noted that the electronic device mentioned in the embodiments of this application may include more or fewer modules of the aforementioned electronic device. For example, the electronic device may also include a memory, a timer, etc.

[0110] The following describes each step of the image display process.

[0111] An application in an electronic device acquires relevant data of an image to be displayed, processes this data according to a fixed procedure to obtain the image data to be displayed, and then sends the image data to the display screen so that the screen can display the image frame. This fixed procedure mainly includes a rendering process, a layer compositing process, and a display submission process. In other words, for a single image frame, the electronic device needs to sequentially execute the rendering process, the layer compositing process, and the display submission process before it can be displayed on the screen. The rendering process can be implemented through the application, hence it is also called the application rendering process. The layer compositing process can be implemented by the layer compositing module (SurfaceFlinger), and the layer compositing process can also be called the SurfaceFlinger layer compositing process. The display submission process can be implemented through hardware, hence it is also called the hardware display submission process.

[0112] Specifically, the rendering process involves acquiring data from multiple layers to be displayed, rendering and drawing these layers to generate composite layer data. The layer compositing process involves combining the composite layer data generated in the rendering process to generate image data to be displayed. The hardware display process involves processing the image data generated in the layer compositing process using hardware and pushing it to the display screen for display.

[0113] The layer compositing method provided in this application is mainly applied in the above-mentioned layer compositing process. It is understood that the layer compositing process can be executed by the GPU or by a hardware composer (HWC), and this application does not impose any restrictions on this.

[0114] The application scenarios provided by the embodiments of this application are described below with reference to the accompanying drawings.

[0115] Before performing layer compositing, to enhance data security and improve the user's visual experience when using electronic devices, images can be blurred. It should be understood that each frame of an image typically consists of multiple layers. Blurring can refer to adding a mask layer among the multiple layers during layer compositing, resulting in a blurred image. A single frame of an image to be displayed typically includes three layers: a wallpaper layer, a desktop control layer, and a dropdown window layer. These three layers are arranged from top to bottom, meaning the top layer is the dropdown window layer. During layer compositing, the blurred layer generated from the previous frame can be reused. This means reusing the blur layers from the previous frame, specifically the wallpaper and desktop control layers blurred by the GPU. The blurred layer from the previous frame is then merged with the dropdown window layer to obtain the image to be displayed. The blurred layer generated from the previous frame can refer to the blurred layer obtained by merging the wallpaper layer, desktop control layer, and mask layer from the previous frame. It should be noted that the layer compositing process is initiated by the layer compositing module in the application framework layer. These initiated processes call the GPU to perform layer compositing, or call the hardware composer (HWC) to perform layer compositing. This application embodiment does not limit this.

[0116] It should be understood that the above are illustrative examples of application scenarios and do not limit the application scenarios of this application in any way.

[0117] The following is combined Figures 5 to 13 The layer composition method provided in the embodiments of this application will be described in detail.

[0118] Figure 5 This is a flowchart illustrating a layer compositing method provided in an embodiment of this application, as shown below. Figure 5 As shown, the method includes:

[0119] S101. Determine whether the current frame image meets the preset conditions.

[0120] The preset conditions include that the current frame image contains the first layer. When the current frame image contains the first layer, the current frame image is blurred.

[0121] In some scenarios, such as when an electronic device is powered on or off, a power-on / off layer is typically added to the display. To improve the user's visual experience, layers below the power-on / off layer (such as wallpaper and desktop control layers) are usually blurred, resulting in a globally blurred display of the electronic device. The first layer can refer to the power-on / off layer.

[0122] In some scenarios, such as when an electronic device displays a drop-down window interface, the device adds a drop-down window layer to the display. To improve the user's visual experience, layers below the drop-down window layer (such as wallpaper layers and desktop control layers) are usually blurred, resulting in a globally blurred display on the electronic device. The first layer can refer to the drop-down window layer.

[0123] Based on the electronic device's settings, when the aforementioned first layer appears in the current frame image, the electronic device performs a global blur display. That is, when the first layer is detected in the current frame image, the electronic device blurs all other layers except the first layer. It should be understood that each layer has its corresponding identifier. When the electronic device detects the identifier of the first layer in the current frame image, it means it has detected the presence of the first layer and begins blurring the current frame image. This is equivalent to detecting the first layer being the condition that triggers the electronic device to perform a global blur display. For example, when the electronic device detects a power on / off layer in the current frame image, it blurs the layers below the power on / off layer, resulting in a global blur display. Similarly, when the electronic device detects a dropdown window layer in the current frame image, it blurs the layers below the dropdown window layer, resulting in a global blur display.

[0124] Optionally, the first layer can be the top layer in the current frame image.

[0125] It should be understood that each frame of an image typically comprises multiple layers. For example, such as... Figure 6 As shown, this frame image consists of three layers, arranged from top to bottom along the Z-axis as Layer 1, Layer 2, and Layer 3. The top layer refers to... Figure 6 Layer 1 in the image. Typically, when displaying a blurred image, the top layer is usually clearly visible. That is to say, the first layer is usually the layer that has not been blurred.

[0126] Determine whether the current frame image meets the preset conditions, and execute S102.

[0127] S102. Use the first strategy to perform layer compositing to obtain the image to be displayed.

[0128] The first strategy refers to using the first blurred layer and the first layer to composite the image if the current frame image is the first image, and generating the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, and other layers refer to the layers in the previous frame image other than the first layer. The first image refers to the image that has a corresponding first blurred layer.

[0129] Since the first blurred layer is the blurred layer corresponding to other layers in the previous frame image, that is to say, when using the first strategy to perform layer compositing on the current frame image, the step of blurring other layers is omitted. It is only necessary to call the blurred layer corresponding to other layers in the previous frame image and the first layer to perform layer compositing to obtain the image to be displayed.

[0130] Optionally, the electronic device can merge the first layer and the first blurred layer using a GPU, or it can merge the first layer and the first blurred layer using a hardware composer (HWC). The embodiments in this application are not limited in this respect.

[0131] For example, such as Figure 7 As shown in (a), when layer compositing is performed using HWC and the GPU does not perform blurring, the time taken for layer compositing is 4ms. Figure 7 As shown in (b), blurring and layer compositing are performed using the GPU, with the layer compositing process taking 10ms. It is evident that using HWC to merge the first layer and the first blurred layer effectively reduces the layer compositing time. It should be understood that because the layer compositing time is shortened, the power consumption of the electronic device is correspondingly reduced.

[0132] The layer compositing method provided in this application uses HWC instead of GPU to perform layer merging processing on the first layer and the first blurred layer. This can further reduce the GPU load and thus further avoid performance problems such as display stuttering or frame dropping caused by excessive GPU load.

[0133] S103. Display the image to be displayed.

[0134] The layer compositing method provided in this application first determines whether the current frame image meets preset conditions. The preset conditions include that the current frame image includes a first layer. When the current frame image includes a first layer, the current frame image is blurred. If the current frame image meets the preset conditions, a first strategy is adopted for layer compositing. The first strategy means that when performing layer compositing, if the current frame image is the first image, a first blurred layer and the first layer are used for layer compositing to generate the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to layers in the previous frame image other than the first layer. The first image refers to the image with a corresponding first blurred layer. Then the image to be displayed is displayed. This is equivalent to omitting the step of the GPU blurring other layers. It is only necessary to call the blurred layer corresponding to other layers in the previous frame image and the first layer for layer compositing to obtain the image to be displayed, thereby reducing the GPU load and avoiding performance problems such as stuttering or frame dropping caused by excessive GPU load.

[0135] Furthermore, during image compositing, if the size of the first layer in the current frame image differs from the size of the first layer in the previous frame image, it indicates that the first layer in the current frame image has not yet stabilized, and the content displayed on the interface has undergone some changes. If the first blurred layer from the previous frame image is still used to merge the first layer in the current frame image, the resulting image to be displayed will differ from the current frame image, resulting in poor display quality. Therefore, before performing layer compositing, it is necessary to further determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image. Only when the size of the current frame image is the same as the size of the previous frame image should the first strategy be used for layer compositing. The following will demonstrate... Figure 8 The embodiments shown will be described in detail below.

[0136] Figure 8 A flowchart illustrating another layer compositing method provided in this application embodiment is shown below. Figure 8 As shown, the method includes:

[0137] S201. Determine whether the current frame image meets the preset conditions.

[0138] The preset conditions include that the current frame image contains the first layer. When the current frame image contains the first layer, the current frame image is blurred.

[0139] If the current frame image meets the preset conditions, then S202 is executed. Alternatively, if the current frame image meets the preset conditions, then S203 is executed. This application embodiment does not limit this.

[0140] S202. Determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0141] It should be understood that when an electronic device begins displaying the first layer, its size is typically fine-tuned to better match the size of the device's display interface. If the size of the first layer in the current frame is the same as that in the previous frame, it indicates that the electronic device has completed the fine-tuning of the first layer's size and entered a stable display state. Let's take a power-off screen as an example. If the size of the first layer (power-off layer) in the current frame is the same as that in the previous frame, the electronic device is already in a stable state of displaying the power-off screen. In this case, if the first strategy is used for layer compositing, the problem of a large difference between the previous and current frame images, leading to a large discrepancy between the resulting image (the blurred layer from the previous frame and the first layer of the current frame) and the image to be displayed, will not occur.

[0142] The layer compositing method provided in this application employs a first strategy to merge the first blurred layer and the first layer of the current frame image when the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image. This avoids the problem that the image to be displayed after layer compositing is significantly different from the current frame image due to the large difference between the previous frame image and the current frame image.

[0143] If the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, S203 can be executed, that is, further determine whether the number of the first layer in the current frame image is 1; or S204 can be executed, that is, directly determine whether the current frame image is the first image; the embodiments of this application do not limit this.

[0144] S203. Determine whether the number of the first layer in the current frame image is 1.

[0145] It should be understood that the first layer is the layer that triggers the electronic device to display a global blur. In some cases, multiple first layers can be displayed simultaneously on the electronic device. For example, the drop-down window layer and the power-off layer are both layers that trigger the global blur. When a user triggers a power-off operation while the electronic device displays a drop-down window, both the drop-down window layer and the power-off layer are displayed on the screen, which is equivalent to having two first layers in the current frame image. In this case, the electronic device pauses the reuse of the first blur layer from the previous frame image and combines it with the first layer of the current frame image for layer compositing.

[0146] If the number of first layers in the current frame image is 1, then execute S204.

[0147] S204. Determine whether the current frame image is the first image.

[0148] Here, the first image can refer to an image with a corresponding first blur layer. In one possible case, the first image can also be called a skipped frame image. If the current frame image is the first image, then S205 is executed, that is, the first blur layer and the first layer of the previous frame image are merged to generate the image to be displayed.

[0149] If the current frame image is not the first image, then the current frame image is the second image, and S206 is executed, which means blurring the other layers in the current frame image to generate the first blurred layer.

[0150] In this process, the second image can composite its other layers with a mask layer, providing the first image with a first blurred layer for layer compositing. That is, the other layers in the second image are blurred (i.e., composited with the mask layer), generating the first blurred layer. This allows the first image to reuse the first blurred layer generated by the second image for layer compositing; therefore, the second image can be called a copy frame. When the current frame image meets the preset conditions, that is, when the electronic device begins to use the first strategy for layer compositing, the images to be processed are usually arranged in a sequence of one copy frame adjacent to one skipped frame. At this point, if the current frame image is not the first image (skipped frame image), then the current frame image is the second image (copy frame image).

[0151] S205. Use the first blurred layer and the first layer to perform layer compositing to generate the image to be displayed.

[0152] The first strategy refers to the following: when performing layer compositing, if the current frame image is the first image, the first blurred layer and the first layer are used to perform layer compositing to generate the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to the layers in the previous frame image other than the first layer, and the first image refers to the image that has a corresponding first blurred layer.

[0153] For example, such as Figure 9 As shown, if the current frame image is the first image (skipped frame image), then the first blurred layer of the previous frame image and the first layer of the skipped frame image (e.g., the power-off layer) are used to merge the layers to generate the image to be displayed.

[0154] S206. Blur other layers in the current frame image to generate the first blurred layer.

[0155] For example, such as Figure 10As shown, when the first layer is first displayed on the screen, its size is usually fine-tuned to better match the size of the electronic device's display. At this time, the image displayed by the electronic device is set to a normal frame image; that is, each frame image is directly composited by the GPU, without reusing the first blurred layer of the previous frame. When the current frame image meets preset conditions, it is determined as a copy frame image, and then layer compositing is performed sequentially, with one copy frame adjacent to one skipped frame. For skipped frame images, the first blurred layer of the previous frame image (i.e., the copy frame image) and the first layer of the skipped frame image can be merged to generate the image to be displayed. For copy frame images (i.e., the second image), other layers in the second image can be blurred to generate the first blurred layer. Other layers refer to layers in the copy frame image other than the first layer, and the first image refers to the image with the corresponding first blurred layer. Blurring other layers can refer to merging other layers with the mask layer. When merging other layers and mask layers, the GPU can be used to merge them.

[0156] S207, Store the first blurred layer.

[0157] After obtaining the first blur layer, the GPU can store the first blur layer in the GPU so that it can be retrieved from the GPU when processing skipped frames.

[0158] S208. Perform layer compositing on the first blurred layer and the first layer of the second image to generate the image to be displayed.

[0159] For example, such as Figure 11 As shown, if the current frame image is the second image (i.e., the copy frame image), the GPU first merges the other layers in the copy frame image with the mask layer to generate the first blurred layer, and then stores the first blurred layer in the GPU.

[0160] Optionally, during the process of merging other layers in the copied frame image with the mask layer, the layer parameters of the first layer can also be input into the GPU at the same time, so that the GPU can merge other layers in the copied frame image with the mask layer according to the layer parameters of the first layer, so that the size of the generated first blurred layer matches the size of the first layer more closely.

[0161] After generating the first blur layer, the GPU sends the first blur layer to the HWC, and then the HWC merges the first blur layer with the first layer of the copied frame image (e.g., the shutdown layer) to generate the image to be displayed.

[0162] For the copied frame image, it is necessary to first merge all layers except the first layer with the mask layer to obtain the first blurred layer, and save the first blurred layer. At the same time, it is also necessary to perform layer compositing processing on the first blurred layer and the first layer in the copied frame image to generate the image to be displayed.

[0163] If the layer compositing method provided in the embodiments of this application is not used, such as Figure 12 As shown in (a), each frame of the image needs to be blurred, which increases the time the GPU takes to process the image; if the layer compositing method provided in the embodiments of this application is used, such as Figure 12 As shown in (b), blurring is performed every frame, that is, blurring is performed on copied frame images, but not on skipped frame images. This can save the GPU time for blurring, thereby reducing the GPU time for image processing.

[0164] The layer compositing method provided in this application embodiment, if the current frame image is a second image, blurs other layers in the second image to generate a first blurred layer, and then performs layer compositing on the first blurred layer and the first layer in the second image to generate an image to be displayed. This avoids the situation where the second image does not generate an image to be displayed. That is, not only is the first layer in the first image merged to obtain the image to be displayed, but the first layer in the second image is also merged to obtain the image to be displayed. This is equivalent to the first layer in the image to be displayed after each frame is layer composited being the updated first layer, avoiding the situation where two adjacent frames are layered using the same first layer and the first blurred layer, thereby avoiding the situation where two adjacent frames are displayed the same, and improving the display effect of electronic devices.

[0165] S209. Display the image to be displayed.

[0166] Before determining whether the current frame image is a skipped frame or a copied frame image, when the current frame image includes the first layer, it's possible to first determine if the previous frame image also includes the first layer. If the previous frame image also includes the first layer, it indicates that the electronic device has begun to enter a stable blurred display state. Alternatively, it's possible to first determine if the previous frame image also includes the first layer. If the previous frame image also includes the first layer, further confirm whether the size of the first layer in the previous frame image is the same as the size of the first layer in the current frame image. If the size of the first layer in the previous frame image is the same as the size of the first layer in the current frame image, then the electronic device has begun to enter a stable blurred display state. After the electronic device has begun to achieve a stable blurred display, using the first strategy for layer compositing can improve the stability of layer compositing and reduce the situation where the difference between the blurred image and the current frame image is large due to large differences between the two frames. The following will explain... Figure 13 The embodiments shown will be described in detail.

[0167] Figure 13 A flowchart illustrating another layer compositing method provided in this application embodiment is shown below. Figure 13 As shown, the method includes:

[0168] S301. Determine that the current frame image has a first layer.

[0169] When the first layer appears in the current frame image of the electronic device, the electronic device performs global blurring. That is to say, when the first layer is detected in the current frame image, the electronic device blurs all other layers except the first layer. It should be understood that each layer has its corresponding identification information. When the electronic device detects the identification information of the first layer in the current frame image, that is, when the first layer is detected in the current image, it begins to blur the current frame image.

[0170] The layer compositing module in the application framework layer can include a layer parameter acquisition submodule. This submodule collects and saves the changes in layer-related parameters in each frame of the image. For example, the layer acquisition submodule can collect the layer identification information in the image and determine whether the first layer exists in the current frame based on the layer identification information. For instance, if the layer parameter acquisition submodule obtains the identification information of the power-off layer, it can determine that the current frame includes the power-off layer, which is the first layer.

[0171] If the current frame image has a first layer, then execute S302.

[0172] S302. Determine that the first layer exists in the previous frame image.

[0173] If the current frame image has a first layer, then execute S303.

[0174] S303. Determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0175] The parameters acquired by the layer parameter acquisition submodule may also include parameters indicating changes in layer size. For example, if the size of the first layer in the current frame image has changed compared to the first layer in the previous frame image, then the parameter indicating the change in layer size is 1; if the size of the first layer in the current frame image has not changed compared to the first layer in the previous frame image, then the parameter indicating the change in layer size is 0. If the parameter indicating the change in layer size acquired by the layer parameter acquisition submodule is 0, then the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, and S304 is executed.

[0176] The layer compositing method provided in this application determines whether to use the first strategy for layer compositing if the size of the first layer in the current frame image and the size of the first layer in the previous frame image are the same. If the size of the first layer in the current frame image and the size of the first layer in the previous frame image are the same, it means that the current frame image is a stable display of the first layer, rather than the first time the first layer is displayed. This can avoid the situation where the display effect of the current frame image is not good due to the reuse of the first blurred layer of the previous frame image when the first layer is displayed for the first time.

[0177] S304. Determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images.

[0178] Here, N previous frame images refer to the N images preceding the current frame image. If the size of the first layer in the current frame image is the same as the size of the first layer in the previous N images, it means that the current frame image is displaying the first layer image stably, rather than during the initial display of the first layer image where the size of the first layer image is fine-tuned.

[0179] If the size of the first layer in the current frame image is the same as the size of the first layer in N previous frame images, then execute S305.

[0180] The layer compositing method provided in this application determines whether to use the first strategy for layer compositing if the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images. If the size of the first layer in the current frame image is the same as the size of the first layer in the previous N images, it means that the current frame image is displaying the first layer image stably, rather than during the initial display of the first layer, when the size of the first layer image needs to be fine-tuned. This avoids the situation where the display effect of the current frame image is poor due to the reuse of the first blurred layer of the previous frame image when fine-tuning the first layer.

[0181] S305. Determine whether the number of the first layer in the current frame image is 1.

[0182] If the number of first layers in the current frame image is 1, then execute S306.

[0183] If the number of first layers in the current frame image is 0, or if the number of first layers in the current frame image is greater than or equal to 2, then the first strategy for layer compositing is abandoned, which is equivalent to the current frame image being the solution frame image that is turned off.

[0184] S306. Determine whether the current frame image is the first image.

[0185] Here, the first image can refer to an image that has a corresponding first blur layer. In one possible case, the first image can also be referred to as a skipped frame image.

[0186] Optionally, the layer compositing module also includes a decision submodule. This decision submodule can be used to determine whether the current frame image is a normal frame image, a copied frame image, or a skipped frame image based on the parameters obtained by the layer parameter acquisition submodule, and to disable the first strategy. Disabling the first strategy means that the GPU performs blurring and layer compositing on every frame image, that is, it does not distinguish the type of the current frame image and does not perform corresponding layer compositing based on the type of the current frame image.

[0187] For example, the layer parameter acquisition submodule obtains the identification information of the shutdown layer that exists in both the current frame image and the previous frame image. Furthermore, the N consecutive parameters indicating the size change of the shutdown layer obtained by the layer parameter acquisition submodule are 0, which means that the size of the shutdown layer in the current frame image is the same as that of the N shutdown layers in the previous frame image. Based on the above parameters, the decision submodule can determine that the current frame is converted from a normal frame to a copy frame, and use the first strategy to perform layer compositing, that is, to reuse the first blurred layer of the previous frame image and the shutdown layer of the current frame image to merge the layers and obtain the image to be displayed.

[0188] For example, the layer parameter acquisition submodule obtains the identification information of the shutdown layer that exists in both the current frame image and the previous frame image. Furthermore, if the N consecutive parameters indicating the size change of the shutdown layer obtained by the layer parameter acquisition submodule are 0, meaning the shutdown layer in the current frame image and the shutdown layers in the N previous frames have the same size, the decision submodule can determine to use the first strategy for image compositing based on these parameters. At this point, the decision submodule can further determine whether the current frame image is a skipped frame or a copied frame. The decision submodule can determine whether the current frame image contains a corresponding first blur layer based on the layer identification information obtained by the layer parameter acquisition submodule, that is, whether the previous frame image stored the first blur layer.

[0189] If the previous frame image obtained by the layer parameter acquisition submodule contains the identification information of the first blurred layer, then the current frame image is a skipped frame image, i.e., the first image.

[0190] If the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, then the current frame image is a skip frame image, i.e., the second image.

[0191] If the identification information of the first blurred layer is not found in the previous frame image obtained by the layer parameter acquisition submodule, then the current frame image is a copy frame image, i.e., the second image.

[0192] If the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a skipped frame, then the current frame image is a copy frame image, i.e., the second image.

[0193] Alternatively, if the identification information of the N previous frame images obtained by the layer parameter acquisition submodule indicates that all N previous frame images are normal frames, then the current frame image is a copy frame image, i.e., the second image.

[0194] Understandably, if the decision submodule has already decided to use the first strategy for layer compositing, that is, to start layer compositing in the order of copying frames and skipping frames, if it is detected that the size of the first layer of the current frame image is different from the size of the first layer of the previous frame image, it can exit the use of the first strategy for layer compositing, which is equivalent to the current frame being a normal frame; or, if it is detected that the current frame image includes at least two first layers, it can also exit the use of the first strategy for layer compositing, which is equivalent to the current frame image being a solution-off frame image.

[0195] Among the solution-off frames, normal frames, copy frames, and skip frames, the solution-off frames have the highest priority, followed by normal frames, and then copy frames and skip frames, which are in parallel.

[0196] For example, if the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, the current frame is indicated as a skip frame; at the same time, if it is detected that the current frame image includes two first layers, the current frame is indicated as a solution off frame; since the solution off frame has the highest priority, the current frame image is a solution off frame, not a skip frame image.

[0197] For example, if the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, the current frame is indicated as a skipped frame; at the same time, if it is detected that the size of the first layer in the current frame image is different from the size of the first layer in the first frame image in the previous frame image, the current frame is indicated as a normal frame; since the priority of normal frames is higher than that of skipped frame images, the current frame image is a normal frame, not a skipped frame image.

[0198] For example, if the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, the current frame is indicated as a skipped frame; and if it is detected that the size of the first layer in the current frame image is different from the size of the first layer in the first frame image of the previous frame, the current frame is indicated as a normal frame; at the same time, if it is detected that the current frame image includes two first layers, the current frame is indicated as a solution off frame; since the priority of solution off frames is higher than that of normal frames and skipped frames, the current frame image is a solution off frame, rather than a normal frame or a skipped frame image.

[0199] If the current frame image is the first image, then execute S307.

[0200] If the current frame image is not the first image, then the current frame image is the second image, and S308 is executed.

[0201] S307. Use the first blurred layer and the first layer to perform layer compositing to generate the image to be displayed.

[0202] The first strategy refers to the following: when performing layer compositing, if the current frame image is the first image, the first blurred layer and the first layer are used to perform layer compositing to generate the image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to the layers in the previous frame image other than the first layer. The first image refers to the image that has a corresponding first blurred layer.

[0203] Optionally, the layer compositing module may also include a frame skipping submodule, which can also be used to control the current frame image to skip the GPU compositing process and directly send the first blurred layer and the first layer to HWC so that HWC can perform layer compositing processing on the first blurred layer and the first layer.

[0204] S308. Blur other layers in the current frame image to generate the first blurred layer.

[0205] S309, Store the first blurred layer.

[0206] S310. Perform layer compositing on the first blurred layer and the first layer of the second image to generate the image to be displayed.

[0207] Optionally, the layer compositing module also includes a blur cache submodule. The blur cache submodule is used to control the other layers and mask layers in the copied frame image to perform layer compositing to obtain the first blur layer. Then, the parameters of the first blur layer are sent to the cache framebuffer in the GPU for storage so that the skip frame image can call the first blur layer generated by the copied frame image.

[0208] S311. Display the image to be displayed.

[0209] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0210] It is understood that, in order to achieve the above functions, the electronic device includes hardware and / or software modules that perform the respective functions. Based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0211] This application embodiment can divide an electronic device into functional modules based on the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one module. It should be noted that the module division in this application embodiment is illustrative and represents only one logical functional division; other division methods may be used in actual implementation. It should also be noted that the module names in this application embodiment are illustrative, and the names of the modules are not limited in actual implementation.

[0212] Figure 14 This is a schematic diagram of a layer compositing device provided in an embodiment of this application.

[0213] It should be understood that the layer composition device 600 can perform... Figures 5 to 13 The layer compositing method shown; the layer compositing apparatus 600 includes: an input unit 610 and a processing unit 620.

[0214] The processing unit 620 is used to determine whether the current frame image meets preset conditions. The preset conditions include that the current frame image includes a first layer. When the current frame image includes a first layer, the current frame image is blurred. If the current frame image meets the preset conditions, a first strategy is used for layer compositing. The first strategy means that when performing layer compositing, if the current frame image is the first image, a first blurred layer and the first layer are used for layer compositing to generate an image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to layers in the previous frame image other than the first layer. The first image refers to the image that has a corresponding first blurred layer. The image to be displayed is then displayed.

[0215] In one embodiment, the first strategy further includes: if the current frame image is a first image, and the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, performing layer compositing using the first blurred layer and the first layer to generate an image to be displayed.

[0216] In one embodiment, the first strategy further includes: if the current frame image is a first image and the number of first layers included in the current frame image is 1, then layer compositing is performed using the first blurred layer and the first layer to generate an image to be displayed.

[0217] In one embodiment, the first strategy further includes: if the current frame image is the second image, blurring other layers in the current frame image to generate a first blurred layer and storing the first blurred layer, and the second image indicating an image that does not have a corresponding first blurred layer.

[0218] In one embodiment, the first strategy further includes: if the current frame image is a second image, blurring other layers in the current frame image to generate a first blurred layer and storing the first blurred layer, and performing layer compositing processing on the first blurred layer and the first layer in the second image to generate an image to be displayed.

[0219] In one embodiment, the first strategy further includes: if the current frame image is the first image, performing layer compositing of the first blurred layer and the first layer using a hardware compositor HWC to generate an image to be displayed.

[0220] In one embodiment, the first layer is the top layer in the current frame image.

[0221] In one embodiment, the preset condition further includes the fact that the previous frame image includes a first layer.

[0222] In one embodiment, the preset condition further includes that the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0223] In one embodiment, the preset condition further includes: the size of the first layer in the current frame image is the same as the size of the first layer in N previous frame images.

[0224] In one embodiment, the image to be displayed is a dynamically displayed image.

[0225] The layer compositing apparatus provided in this embodiment is used to execute the layer compositing method of the above embodiment. The technical principle and technical effect are similar, and will not be described again here.

[0226] It should be noted that the aforementioned layer compositing device 600 is embodied in the form of a functional unit. The term "unit" here can be implemented in software and / or hardware, without specific limitations.

[0227] For example, a "unit" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, integrated logic circuitry, and / or other suitable components that support the described functions.

[0228] Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0229] Figure 15 A schematic diagram of the structure of an electronic device provided in this application is shown. Figure 15 The dashed lines indicate that the unit or module is optional. The electronic device 700 can be used to implement the layer composition method described in the above method embodiments.

[0230] The electronic device 700 includes one or more processors 701, which support the implementation of the layer composition method in the method embodiments. The processor 701 can be a general-purpose processor or a special-purpose processor. For example, the processor 701 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0231] The processor 701 can be used to control the electronic device 700, execute software programs, and process data from the software programs. The electronic device 700 may also include a communication unit 705 for inputting (receiving) and outputting (transmitting) signals.

[0232] For example, electronic device 700 may be a chip, communication unit 705 may be the input and / or output circuit of the chip, or communication unit 705 may be the communication interface of the chip, and the chip may be a component of terminal device or other electronic device.

[0233] For example, electronic device 700 can be a terminal device, communication unit 705 can be the transceiver of the terminal device, or communication unit 705 can be the transceiver circuit of the terminal device.

[0234] The electronic device 700 may include one or more memories 702, which store a program 704. The program 704 can be executed by the processor 701 to generate instructions 703, causing the processor 701 to execute the impedance matching method described in the above method embodiments according to the instructions 703.

[0235] Optionally, the memory 702 may also store data. Optionally, the processor 701 may also read the data stored in the memory 702, which may be stored at the same memory address as the program 704, or the data may be stored at a different memory address than the program 704.

[0236] The processor 701 and memory 702 can be configured separately or integrated together; for example, integrated on the system on chip (SOC) of the terminal device.

[0237] For example, the memory 702 can be used to store the relevant program 704 of the layer compositing method provided in the embodiments of this application, and the processor 701 can be used to call the relevant program 704 of the layer compositing method stored in the memory 702 when performing layer compositing, and execute the layer compositing method of the embodiments of this application; including: determining whether the current frame image meets the preset conditions, the preset conditions including that the current frame image includes a first layer, wherein when the current frame image includes a first layer, the current frame image is blurred; if the current frame image meets the preset conditions, a first strategy is adopted to perform layer compositing, the first strategy means that when performing layer compositing, if the current frame image is a first image, a first blurred layer and a first layer are used to perform layer compositing to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, other layers refer to layers other than the first layer in the previous frame image, and the first image refers to an image with a corresponding first blurred layer; and displaying the image to be displayed.

[0238] This application also provides a computer program product that, when executed by processor 701, implements the layer compositing method described in any of the method embodiments of this application.

[0239] The computer program product can be stored in memory 702, for example, program 704. Program 704 is finally converted into an executable object file that can be executed by processor 701 after processing such as preprocessing, compilation, assembly and linking.

[0240] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the layer compositing method described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

[0241] The computer-readable storage medium is, for example, memory 702. Memory 702 can be volatile memory or non-volatile memory, or memory 702 can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0242] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0243] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0244] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0245] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0246] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0247] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0248] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0249] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A layer compositing method, characterized in that, The method is applied to a dynamic blurring scene, which refers to a scene where each frame of the image to be blurred in the image to be blurred queue is blurred. The image to be blurred queue includes a first image and a second image. The first image is an image with a corresponding first blur layer, and the second image is an image without a corresponding first blur layer. The first blur layer is a blur layer corresponding to other layers in the previous frame image. The other layers are layers in the previous frame image other than the first layer. The first image and the second image are alternately arranged in the image to be blurred queue. The method includes: If the current frame image is the first image, then the first blurred layer and the first layer are used to perform layer compositing to generate the image to be displayed; If the current frame image is the second image, the other layers in the current frame image are blurred to generate the first blurred layer. The first blurred layer and the first layer in the second image are then combined to generate the image to be displayed. Display the image to be displayed.

2. The method according to claim 1, characterized in that, If the current frame image is the first image, and the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, the first blurred layer and the first layer are used to perform layer compositing to generate the image to be displayed.

3. The method according to claim 1 or 2, characterized in that, If the current frame image is the first image, and the number of first layers included in the current frame image is 1, then the first blurred layer and the first layer are used to perform layer compositing to generate the image to be displayed.

4. The method according to claim 1 or 2, characterized in that, If the current frame image is the second image, the other layers in the current frame image are blurred to generate the first blurred layer, and the first blurred layer is stored.

5. The method according to claim 1 or 2, characterized in that, The first blurred layer and the first layer are combined using a hardware compositor (HWC) to generate the image to be displayed.

6. The method according to claim 1 or 2, characterized in that, The first layer is the top layer in the current frame image.

7. The method according to claim 1 or 2, characterized in that, The size of the first layer in the current frame image is the same as the size of the first layer in N previous frame images.

8. The method according to claim 1 or 2, characterized in that, The image to be displayed is a dynamically displayed image.

9. An electronic device, characterized in that, The electronic device includes a module for performing the method as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored on the memory, and when the computer programs are executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 8.

11. A chip system, characterized in that, The chip system includes a processor for calling and running a computer program from memory, causing an electronic device on which the chip system is installed to perform the method as described in any one of claims 1 to 8.

12. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1 to 8.

Images