Layer synthesis method and electronic equipment

Abstract

The embodiment of the invention is suitable for the technical field of data processing, and provides a layer synthesis method and electronic equipment, whether a current frame image meets a preset condition is determined, the preset condition comprises that the current frame image comprises a first layer, when the current frame image comprises the first layer, the current frame image is subjected to fuzzy processing, and the first layer is obtained; and if the current frame image meets the preset condition, performing layer synthesis by adopting a first strategy, namely if the current frame image is the first image, performing layer synthesis by adopting the fuzzy layers corresponding to other layers in the previous frame image and the first layer to generate a to-be-displayed image, and then displaying the to-be-displayed image. Therefore, equivalently, the step of performing fuzzy processing on other image layers by the GPU is omitted, and the to-be-displayed image can be obtained only by calling the fuzzy image layers corresponding to other image layers in the previous frame of image and performing image layer synthesis on the first image layer, so that the performance problems of display process blockage or frame loss and the like caused by overlarge load of the GPU are avoided.

Description

Technical Field

[0001] The present application relates to the field of data processing, and more specifically, to a layer synthesis method and an electronic device. Background Art

[0002] In order to improve the security of data when users use electronic devices and enhance the user's visual experience, in some possible cases, the electronic device can blur the entire screen when displaying the interface.

[0003] In one possible scenario, the electronic device sends each displayed frame to a graphics processing unit (GPU), so that the GPU blurs each frame and then sends the blurred frame to the display screen for display. Since the display refresh rate of the electronic device is relatively high, for example, the display refresh rate is 60Hz, which means that 60 frames need to be displayed per second. In this case, the GPU needs to blur 60 frames per second, which greatly occupies the resources of the GPU, which may lead to performance problems such as freezes or frame drops during the display process, as well as the problem of excessive GPU power consumption.

[0004] Based on this, how to reduce the GPU load when the electronic device displays blurry full-screen and avoid performance problems during the display process has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a layer synthesis method, which can reduce the GPU load when the electronic device displays a full-screen blur, avoid performance problems during the display process, and reduce the power consumption of the GPU.

[0006] In a first aspect, a layer synthesis method is provided, comprising:

[0007] Determining whether the current frame image meets a preset condition, the preset condition comprising that the current frame image includes a first layer, wherein when the current frame image includes the first layer, blurring the current frame image;

[0008] If the current frame image meets the preset conditions, the first strategy is used for layer synthesis. The first strategy means that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used for layer synthesis to generate an image to be displayed. The first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, and other layers refer to the layers in the previous frame image other than the first layer. The first image refers to an image with a corresponding first blurred layer.

[0009] Displays the image to be displayed.

[0010] Among them, the layer parameter acquisition submodule in the layer synthesis module can acquire layer-related parameters. For example, the layer acquisition submodule can acquire identification information of the layer in the image, and determine whether there is a first layer in the current frame image according to the identification information of the layer. Exemplarily, if the identification information acquired by the layer parameter acquisition submodule is the identification information of the shutdown layer, it can be determined that the current frame image includes the shutdown layer, that is, the first layer.

[0011] The layer synthesis method provided in the embodiment of the present application first determines whether the current frame image meets the preset conditions, and the preset conditions include that the current frame image includes the first layer, wherein when the current frame image includes the first layer, the current frame image is blurred, and if the current frame image meets the preset conditions, the first strategy is adopted for layer synthesis, and the first strategy means that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used for layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, and the first image refers to the image with the corresponding first blurred layer, and then the image to be displayed is displayed, which is equivalent to omitting the step of the GPU blurring other layers, and only the blurred layers corresponding to other layers in the previous frame image and the first layer need to be called for layer synthesis to obtain the image to be displayed, thereby reducing the load of the GPU and avoiding performance problems such as display process freezes or frame drops due to excessive GPU load.

[0012] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned first strategy also includes: if the current frame image is the first image, and the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed.

[0013] The layer synthesis method provided in the embodiment of the present application adopts a first strategy to perform layer merging processing on the first blurred layer and the first layer of the current frame image when the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, thereby avoiding the problem that the image to be displayed after the layer synthesis processing is significantly different from the current frame image due to the significant difference between the previous frame image and the current frame image.

[0014] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned first strategy also includes: if the current frame image is the first image, and the number of first layers included in the current frame image is 1, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed.

[0015] Among them, the first image can be called a skipped frame image.

[0016] If the number of first layers in the current frame image is 0, or the number of first layers in the current frame image is greater than or equal to 2, then the first strategy for layer synthesis is exited, which is equivalent to the current frame image being a solution frame image for closing the solution.

[0017] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned first strategy also includes: if the current frame image is a second image, blurring other layers in the current frame image to generate a first blurred layer, and storing the first blurred layer, and the second image indicates that there is no image corresponding to the first blurred layer.

[0018] The second image may be referred to as a copy frame image.

[0019] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned first strategy also includes: if the current frame image is the second image, blurring other layers in the current frame image to generate a first blurred layer, storing the first blurred layer, and performing layer synthesis processing on the first blurred layer and the first layer in the second image to generate an image to be displayed.

[0020] It should be understood that when the first strategy is initially adopted for layer synthesis, the images are usually sorted in the order of a copy frame adjacent to a skip frame. Fig.10 shown.

[0021] The layer synthesis method provided by the embodiment of the present application, if the current frame image is the second image, then blurs the other layers in the second image to generate a first blurred layer, and synthesizes the first blurred layer with the first layer in the second image to generate an image to be displayed, thereby avoiding the situation where the second image does not generate an image to be displayed, that is, not only the first layer in the first image is layer-merged to obtain the image to be displayed, but also the first layer in the second image is layer-merged to obtain the image to be displayed, which is equivalent to the first layer in each frame of the image to be displayed after layer synthesis being the updated first layer, thereby avoiding the situation where two adjacent frames of images are layer-synthesized using the same first layer and the first blurred layer, thereby avoiding the situation where two adjacent frames of images display the same image, thereby improving the display effect of the electronic device.

[0022] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned first strategy also includes: if the current frame image is the first image, the first blurred layer and the first layer are layer-synthesized by the hardware synthesizer HWC to generate an image to be displayed.

[0023] The layer synthesis method provided in the embodiment of the present application uses HWC instead of GPU to perform layer merging processing on the first layer and the first blurred layer when merging the first layer and the first blurred layer. This can further reduce the load of the GPU, thereby further avoiding performance issues such as display process freezes or frame drops due to excessive GPU load.

[0024] In combination with the first aspect, in some embodiments of the first aspect, the first layer is a top layer in the current frame image.

[0025] Among them, the shutdown layer and the drop-down window layer are usually the layers displayed on the top layer, that is, the top layer in the current frame image.

[0026] In combination with the first aspect, in some embodiments of the first aspect, the preset condition further includes that the previous frame image includes the first layer.

[0027] In combination with the first aspect, in some embodiments of the first aspect, the preset condition further includes that the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0028] Among them, the layer parameter acquisition submodule in the layer synthesis module can acquire layer-related parameters. For example, if the size of the first layer in the current frame image has changed from that in the previous frame image, the parameter indicating the size change of the layer is 1; if the size of the first layer in the current frame image has not changed from that in the previous frame image, the parameter indicating the size change of the layer is 0. If the parameter indicating the size change of the layer obtained by the layer parameter acquisition submodule is 0, the size of the first layer in the current frame image is the same as that of the first layer in the previous frame image.

[0029] The layer synthesis method provided in the embodiment of the present application determines that both the current frame image and the previous frame image include a first layer, and when the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, then determines whether to adopt the first strategy for layer synthesis. If the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, it means that the current frame image is an image that stably displays the first layer, rather than the first time the first layer is displayed. This can avoid the situation where the display effect of the current frame image is poor due to the reuse of the first blurred layer of the previous frame image when the first layer is displayed for the first time.

[0030] In combination with the first aspect, in some embodiments of the first aspect, the above-mentioned preset condition also includes: the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images.

[0031] It should be noted that the N previous frames refer to the N images before the current frame. If the size of the first layer in the current frame is the same as the size of the first layer in the N previous images, it means that the current frame is a stable display of the first layer, rather than a period of fine-tuning the size of the first layer in the initial stage of displaying the first layer.

[0032] The layer synthesis method provided in the embodiment of the present application determines whether to adopt the first strategy to perform layer synthesis when it is determined that the size of the first layer in the current frame image is the same as the size of the first layer in N previous frame images. If the size of the first layer in the current frame image is the same as the first layer in the N previous images, it means that the current frame image is a stable display of the first layer, rather than a period of time when the size of the first layer image needs to be fine-tuned in the early stage of displaying the first layer. This can avoid the situation where the display effect of the current frame image is poor due to the reuse of the first blurred layer of the previous frame image when fine-tuning the first layer.

[0033] In combination with the first aspect, in some embodiments of the first aspect, the image to be displayed is a dynamically displayed image.

[0034] It should be understood that the generated image to be displayed is a dynamically displayed image. Therefore, in the process of layer synthesis, the layer synthesis is performed according to the refresh rate of the electronic device. Compared with static realistic images, dynamically displayed images undergo more layer synthesis and occupy more system resources. The layer synthesis method provided in the embodiment of the present application can effectively reduce the system resources occupied when blurring the dynamically displayed images.

[0035] In a second aspect, a layer synthesis device is provided, comprising a unit for executing any one of the methods in the first aspect. The device may be a server, a terminal device, or a chip in a terminal device. The device may include an input unit and a processing unit.

[0036] When the device is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may also include a memory, which is used to store computer program code. When the processor executes the computer program code stored in the memory, the terminal device executes any one of the methods in the first aspect.

[0037] When the device is a chip in a terminal device, the processing unit may be a processing unit inside the chip, and the input unit may be an output interface, a pin or a circuit, etc.; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the methods in the first aspect.

[0038] In one possible implementation, a memory is used to store computer program code; a processor executes the computer program code stored in the memory, and when the computer program code stored in the memory is executed, the processor is used to perform: determining whether a current frame image meets a preset condition, the preset condition includes that the current frame image includes a first layer, wherein when the current frame image includes the first layer, blurring the current frame image; if the current frame image meets the preset condition, adopting a first strategy for layer synthesis, the first strategy means that when performing layer synthesis, if the current frame image is the first image, adopting a first blurred layer and a first layer for layer synthesis to generate an image to be displayed, the first blurred layer refers to a blurred layer corresponding to other layers in a previous frame image, other layers refer to layers other than the first layer in the previous frame image, and the first image refers to an image having a corresponding first blurred layer; and displaying the image to be displayed.

[0039] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by a layer synthesis device, the layer synthesis device executes any one of the layer synthesis methods in the first aspect.

[0040] In a fourth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by a layer synthesis device, the layer synthesis device executes any one of the device methods in the first aspect.

[0041] The layer synthesis method and electronic device provided by the embodiment of the present application, before performing layer synthesis, first determine whether the current frame image meets the preset conditions, the preset conditions include that the current frame image includes the first layer, wherein when the current frame image includes the first layer, the current frame image is blurred, if the current frame image meets the preset conditions, the first strategy is adopted to perform layer synthesis, the first strategy means that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, the first image refers to the image with the corresponding first blurred layer, and then the image to be displayed is displayed, which is equivalent to omitting the step of GPU blurring other layers, and only the blurred layers corresponding to other layers in the previous frame image and the first layer need to be called to perform layer synthesis to obtain the image to be displayed, thereby reducing the load of the GPU, avoiding performance problems such as display process freeze or frame loss due to excessive GPU load, and at the same time reducing the GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of a dynamic fuzzy interface;

[0043] Figure 2 It is a schematic diagram of the process of layer synthesis;

[0044] Figure 3 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0045] Figure 4 is a schematic diagram of a software system for an electronic device applicable to the present application;

[0046] Figure 5 It is a flowchart of a layer synthesis method provided in an embodiment of the present application;

[0047] Figure 6 is a schematic diagram of a layer arrangement order provided in an embodiment of the present application;

[0048] Figure 7 It is a schematic diagram of the time taken for layer synthesis using HWC and GPU;

[0049] Figure 8 It is a flowchart of another layer synthesis method provided in an embodiment of the present application;

[0050] Fig. 9 It is a schematic diagram of the process of skipping frames to perform layer synthesis in an embodiment of the present application;

[0051] Fig.10is a schematic diagram of the arrangement order of different frames in the embodiment of the present application;

[0052] Fig.11 It is a schematic diagram of the process of copying frames for layer synthesis in an embodiment of the present application;

[0053] Fig.12 It is a schematic diagram of the time taken for blurring in layer processing;

[0054] Fig.13 It is a flowchart of another layer synthesis method provided in an embodiment of the present application;

[0055] Fig.14 is a schematic diagram of a layer synthesis device provided by the present application;

[0056] Fig.15 It is a schematic diagram of an electronic device for image processing provided by the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0058] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0059] To facilitate understanding, some of the examples given are provided for reference to the description of concepts related to the embodiments of the present application.

[0060] 1. Real-time blur.

[0061] In order to improve the security of data when users use electronic devices and enhance the user's visual experience, the display interface of the electronic device can be blurred so that the content displayed on the display interface is blurred and hidden. This scene of blurred display interface is called blurred scene.

[0062] The blurring process may refer to adding a mask layer to the upper layer of the image to blur the content displayed in the image. It is understandable that the blurring process is usually performed in a GPU.

[0063] Blurring the display interface can be divided into static blur and dynamic blur. Among them, static blur can refer to blurring a frame of image, and the electronic device only displays the blurred image of this frame, and does not display other blurred images as time changes. Dynamic blur can also be called real-time blur, which means blurring each frame of the electronic device, and the electronic device displays different blurred images as time changes. Compared with static blur, real-time blur will occupy a large amount of GPU memory and affect the normal image processing of the GPU.

[0064] For example, Figure 1 As shown, the wallpaper displayed on the shutdown interface 10 is a dynamic wallpaper. At this time, the shutdown interface 10 includes a shutdown layer and a wallpaper layer, wherein the shutdown layer includes a shutdown icon 11 and a restart icon 12. It is understandable that the shutdown layer may also include more components to display more information, for example, a date component (not shown in the figure) that displays a date, or text that prompts the user to force a restart, which is not limited in the present embodiment of the application.

[0065] It should be understood that the wallpaper displayed on the lock screen interface 10 is a dynamic wallpaper, for example, the love heart in the wallpaper can shrink and move. These dynamic effects are not shown in the schematic diagram.

[0066] 2.Full screen blur.

[0067] Full screen blur means that the entire display interface is blurred, for example, Figure 1 The shutdown interface 10 shown. It is understandable that the topmost layer displayed in the full-screen blurred display interface can be clearly displayed, and the wallpaper layer in the display interface occupies the entire area of ​​the display interface, and the wallpaper layer is blurred. For example, Figure 1 The shutdown layer composed of the shutdown icon 11 and the restart icon 12 is not blurred and is a clearly displayed layer, and the wallpaper layer under the shutdown layer is blurred.

[0068] 3. Frame.

[0069] Frame refers to the smallest unit of a single picture in the interface display. A frame can be understood as a still picture. Displaying multiple connected frames in rapid succession can create the illusion of motion. Frame rate refers to the number of frames that refresh the picture in 1 second. It can also be understood as the number of times the graphics processor in the terminal device refreshes the picture per second. A high frame rate can produce smoother and more realistic animations. The more frames per second, the smoother the displayed action will be.

[0070] At present, in order to improve the security of data when users use electronic devices and improve the user's visual experience, the interface can be blurred when the electronic device displays the interface. It should be understood that each frame is usually obtained by merging multiple layers, for example, Figure 2 As shown, the picture displayed on the display interface is obtained by merging the wallpaper layer 10, the desktop control layer 20 and the drop-down window layer 30. Usually, the GPU can synthesize the above multiple layers to generate a display picture. In one possible case, when the GPU performs layer synthesis, it can also add a mask layer 40 to make the synthesized display picture blurred. Among them, the mask layer 40 can refer to the icon attached to the wallpaper layer 10 and the desktop control layer 20, and its own attributes are not layer attributes. Figure 2 As shown, a mask layer 40 is added above the wallpaper layer 10 and the desktop control layer 20 and below the drop-down window layer 30, so that the area below the drop-down window layer 30 in the synthesized display is blurred. As users have higher and higher requirements for display interfaces, the display refresh rate of electronic devices has also increased. At present, the display refresh rate of electronic devices is 60Hz, which means that 60 frames of images need to be displayed per second. In this case, the GPU needs to blur 60 frames of images per second, which greatly occupies the resources of the GPU. The GPU cannot perform layer synthesis and blurring in time, resulting in problems such as freezes or frame drops in the electronic device during the display of the image, as well as the problem of excessive GPU power consumption.

[0071] In view of this, a layer synthesis method provided in an embodiment of the present application first determines whether the current frame image meets a preset condition, the preset condition includes that the current frame image includes a first layer, wherein when the current frame image includes the first layer, the current frame image is blurred, and if the current frame image meets the preset condition, a first strategy is adopted for layer synthesis, the first strategy means that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used for layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, the first image refers to the image with the corresponding first blurred layer, and then the image to be displayed is displayed, which is equivalent to omitting the step of the GPU blurring other layers, and only the blurred layers corresponding to other layers in the previous frame image and the first layer need to be called for layer synthesis to obtain the image to be displayed, thereby reducing the load of the GPU, avoiding performance problems such as display process freeze or frame loss due to excessive GPU load, and reducing the power consumption of the GPU.

[0072] The layer synthesis method provided in the embodiment of the present application can be applied to electronic devices. Optionally, the electronic device includes a terminal device, which can also be called a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The terminal device can be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiment of the present application does not limit the specific technology and specific device form adopted by the terminal device.

[0073] For example, Figure 3 1 shows a schematic diagram of the structure of the electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0074] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0075] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0076] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0077] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0078] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0079] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0080] In one possible case, the GPU can perform layer synthesis on multiple layers to generate an image to be displayed. Furthermore, the GPU can also add a blur layer when performing layer synthesis so that the generated image to be displayed is a blurred image. When rendering, the GPU can also perform blur processing on the image to obtain a blurred image, and then send the blurred image to the display for display.

[0081] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0082] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0083] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0084] It should be noted that any electronic device mentioned in the embodiments of the present application may include more or fewer modules in the electronic device 100.

[0085] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0086] Figure 4 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0087] The layered architecture of the electronic device 100 divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0088] The application layer can include a series of application packages.

[0089] like Figure 4 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0090] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0091] The application program may also generally perform image rendering on the image to be displayed, generate multiple layer data, and send the generated multiple layer data to the layer synthesis module in the application program framework layer.

[0092] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a layer synthesis module, and the like.

[0093] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0094] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0095] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0096] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0097] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0098] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0099] The layer synthesis module is used to synthesize multiple layers obtained after the application performs image rendering to obtain an image to be displayed.

[0100] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0101] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0102] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0103] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0104] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0105] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0106] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0107] A 2D graphics engine is a drawing engine for 2D drawings.

[0108] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, sensor driver, Wi-Fi driver, etc.

[0109] It should be noted that the electronic device mentioned in the embodiments of the present application may include more or fewer modules in the above electronic device. For example, the electronic device may also include a memory, a timer, etc.

[0110] The following is an explanation of the various processes in the image display process.

[0111] The application in the electronic device obtains the relevant data of the image to be displayed, and processes the relevant data of the frame image to be displayed according to a fixed process to obtain the image data to be displayed, and then sends the image data to be displayed to the display screen so that the display screen displays the frame image. The above-mentioned fixed process mainly includes a drawing and rendering process, a layer synthesis process and a display sending process. That is to say, for a frame of image, the electronic device needs to execute the drawing and rendering process, the layer synthesis process and the display sending process in sequence before it can be displayed on the display screen. Among them, the drawing and rendering process can be implemented by the application, so it is also called the application drawing and rendering process. The layer synthesis process can realize layer synthesis by the layer synthesis module (SurfaceFlinger), and the layer synthesis process can also be called the SurfaceFlinger layer synthesis process. The display sending process can be implemented by hardware (hardware), so it is also called the hardware display sending process.

[0112] Specifically, the drawing and rendering process refers to obtaining multiple layers of data to be displayed, drawing and rendering the multiple layers of data to be displayed, and generating the layers of data to be synthesized. The layer synthesis process refers to synthesizing the layers of data to be synthesized generated by the drawing and rendering process to generate the image data to be displayed. The hardware display process refers to performing hardware display processing on the image data to be displayed generated by the layer synthesis process, and pushing it to the display screen for display.

[0113] The layer synthesis method provided in the embodiment of the present application is mainly applied to the above-mentioned layer synthesis process. It is understandable that the layer synthesis process can be executed by a GPU or a hardware composer (HWC), and the embodiment of the present application does not limit this.

[0114] The application scenarios provided by the embodiments of the present application are described below with reference to the accompanying drawings.

[0115] Before the layer synthesis process is performed, in order to improve the data security of users using electronic devices and improve the user's visual experience, the image can be blurred. It should be understood that each frame of the displayed image usually includes multiple layers, and the blurring of the image can refer to adding a mask layer to the multiple layers when the multiple layers are layer synthesized, so that the image to be displayed is blurred. A frame of image to be displayed includes 3 layers, namely, a wallpaper layer, a desktop control layer, and a drop-down window layer. These three layers are arranged in order from top to bottom, that is, the top layer is the drop-down window layer. When performing layer synthesis, the blurred layer generated by the previous frame of the image can be reused, that is, the layer in which the wallpaper layer and the desktop control layer in the previous frame of the image are blurred by the GPU, and the blurred layer of the previous frame of the image is merged with the drop-down window layer to obtain the image to be displayed. Among them, the blurred layer generated by the previous frame of the image can refer to the blurred layer obtained by merging the wallpaper layer, the desktop control layer and the mask layer of the previous frame of the image. It should be noted that the process of layer synthesis is called up by the layer synthesis module in the application framework layer. These called processes call the GPU for layer synthesis, or call the hardware synthesizer (HWC) for layer synthesis. The embodiments of the present application do not limit this.

[0116] It should be understood that the above is an example of an application scenario and does not limit the application scenario of the present application.

[0117] Combine the following Figures 5 to 13 The layer synthesis method provided in the embodiment of the present application is described in detail.

[0118] Figure 5 A schematic diagram of a process of layer synthesis method provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the method includes:

[0119] S101: Determine whether the current frame image meets a preset condition.

[0120] The preset condition includes that the current frame image includes the first layer, and when the current frame image includes the first layer, blurring is performed on the current frame image.

[0121] In some scenarios, such as when an electronic device is turned on and off, the electronic device usually adds a power on and off layer on the display interface. At this time, in order to improve the user's visual experience, the layers below the power on and off layer (such as the wallpaper layer and the desktop control layer) are usually blurred, so that the electronic device performs a global blurred display. The first layer can refer to the power on and off layer.

[0122] In some scenarios, for example, when an electronic device displays a drop-down window interface, the electronic device adds a drop-down window layer on the display interface. In order to improve the user's visual experience, the layers below the drop-down window layer (such as the wallpaper layer and the desktop control layer) are usually blurred so that the electronic device performs a global blurred display. The first layer may refer to the drop-down window layer.

[0123] Based on the setting information of the electronic device, when the above-mentioned first layer appears in the current frame image of the electronic device, the electronic device performs global blurred display, that is, when the first layer is detected in the current frame image, the electronic device performs blurred processing on other layers other than the first layer. It should be understood that each layer has its corresponding identification information. When the electronic device detects that there is identification information of the first layer in the current frame image, it also detects that there is the first layer in the current image and starts to blur the current frame image. This is equivalent to detecting that the first layer is a condition that triggers the electronic device to perform global blurred display. For example, when the electronic device detects that the switch layer appears in the current frame image, the layer below the switch layer is blurred, so that the electronic device performs global blurred display. For another example, when the electronic device detects that a drop-down window layer appears in the current frame image, the layer below the drop-down window layer is blurred, so that the electronic device performs global blurred display.

[0124] Optionally, the first layer may be a top layer in the current frame image.

[0125] It should be understood that each frame of image generally includes multiple layers. Figure 6 As shown in the figure, the frame image consists of three layers, which are layer 1, layer 2 and layer 3 from top to bottom along the Z axis. The top layer refers to Figure 6 Layer 1 in the image. Usually when blurring is performed, the top layer is usually displayed clearly. In other words, the first layer is usually the layer that is not blurred.

[0126] Determine whether the current frame image meets a preset condition, and execute S102.

[0127] S102: Using the first strategy to perform layer synthesis to obtain an image to be displayed.

[0128] Among them, the first strategy means that if the current frame image is the first image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, and the first image refers to the image with the corresponding first blurred layer.

[0129] Since the first blurred layer is the blurred layer corresponding to other layers in the previous frame image, that is to say, when the first strategy is used to synthesize the layers of the current frame image, the step of blurring other layers is omitted. It is only necessary to call the blurred layers corresponding to other layers in the previous frame image and the first layer for layer synthesis to obtain the image to be displayed.

[0130] Optionally, the electronic device may merge the first layer and the first blurred layer through a GPU, or may merge the first layer and the first blurred layer through a hardware composer (HWC), and the embodiment of the present application does not limit this.

[0131] For example, Figure 7 As shown in (a) of FIG, when the layer synthesis process is performed by HWC and the GPU does not perform blur processing, the time taken for the layer synthesis process is 4ms. Figure 7 As shown in (b), the GPU performs blur processing and layer synthesis processing, and the time taken for layer synthesis processing is 10ms. It can be seen that using HWC to merge the first layer and the first blurred layer can effectively reduce the time for layer synthesis. It should be understood that since the time for layer synthesis is shortened, the power consumption of the electronic device is also shortened accordingly.

[0132] The layer synthesis method provided in the embodiment of the present application uses HWC instead of GPU to perform layer merging processing on the first layer and the first blurred layer when merging the first layer and the first blurred layer. This can further reduce the load of the GPU, thereby further avoiding performance issues such as display process freezes or frame drops due to excessive GPU load.

[0133] S103: Display the image to be displayed.

[0134] The layer synthesis method provided in the embodiment of the present application first determines whether the current frame image meets the preset conditions, and the preset conditions include that the current frame image includes the first layer, wherein when the current frame image includes the first layer, the current frame image is blurred, and if the current frame image meets the preset conditions, the first strategy is adopted for layer synthesis, and the first strategy means that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used for layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, and the first image refers to the image with the corresponding first blurred layer, and then the image to be displayed is displayed, which is equivalent to omitting the step of the GPU blurring other layers, and only the blurred layers corresponding to other layers in the previous frame image and the first layer need to be called for layer synthesis to obtain the image to be displayed, thereby reducing the load of the GPU and avoiding performance problems such as display process freezes or frame drops due to excessive GPU load.

[0135] Furthermore, when performing image synthesis, if the size of the first layer in the current frame image is different from the size of the first layer in the previous frame image, it means that the current frame image has not yet started to stably display the first layer, and there are some changes in the content displayed on the display interface. If the first blurred layer of the previous frame image is still used to merge the first layer in the current frame image, the resulting image to be displayed will be different from the current frame image, and the display effect will be poor. Therefore, before performing layer synthesis, it is possible to further determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, and if the size of the current frame image is the same as the size of the previous frame image, then use the first strategy for layer synthesis. Figure 8 The illustrated embodiment is described in detail.

[0136] Figure 8 A schematic diagram of a process flow of another layer synthesis method provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the method includes:

[0137] S201: Determine whether the current frame image meets a preset condition.

[0138] The preset condition includes that the current frame image includes the first layer, and when the current frame image includes the first layer, blurring is performed on the current frame image.

[0139] If the current frame image meets the preset condition, then S202 is executed. Alternatively, if the current frame image meets the preset condition, then S203 is executed. This embodiment of the present application does not limit this.

[0140] S202: Determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0141] It should be understood that when the electronic device starts to display the first layer, in order to make the first layer more compatible with the size of the display interface of the electronic device, the size of the first layer is usually fine-tuned. If the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, it means that the electronic device has completed the fine-tuning of the size of the first layer and has entered a stable display state. Take the current frame image as the shutdown screen as an example. If the size of the first layer (shutdown layer) in the current frame image is the same as the size of the first layer in the previous frame image, the electronic device is already in a state of stably displaying the shutdown screen and is in a stable state. At this time, if the first strategy is used for layer synthesis, there will be no problem that the image to be displayed obtained by using the blurred layer corresponding to the previous frame image and the first layer of the current frame image for layer synthesis is very different from the current frame image due to the large difference between the previous frame image and the current frame image.

[0142] The layer synthesis method provided in the embodiment of the present application adopts a first strategy to perform layer merging processing on the first blurred layer and the first layer of the current frame image when the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, thereby avoiding the problem that the image to be displayed after the layer synthesis processing is significantly different from the current frame image due to the significant difference between the previous frame image and the current frame image.

[0143] If the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, S203 can be executed, that is, further determining whether the number of first layers in the current frame image is 1; S204 can also be executed, that is, directly determining whether the current frame image is the first image; the embodiment of the present application does not limit this.

[0144] S203: Determine whether the number of the first layer in the current frame image is 1.

[0145] It should be understood that the first layer is a layer that triggers the electronic device to perform global blurred display. In one possible case, multiple first layers can be displayed on the electronic device at the same time. For example, the drop-down window layer and the shutdown layer are both layers that trigger the electronic device to perform global blurred display. When the user triggers the shutdown operation when the electronic device displays a drop-down window, the drop-down window layer and the shutdown layer are displayed on the display interface at the same time, which is equivalent to the number of first layers in the current frame image being 2. In this case, the electronic device suspends multiplexing of the first blurred layer of the previous frame image and the first layer of the current frame image for layer synthesis.

[0146] If the number of the first layer in the current frame image is 1, execute S204.

[0147] S204: Determine whether the current frame image is the first image.

[0148] The first image may refer to an image having a corresponding first blurred layer. In one possible case, the first image may also be referred to as a skipped frame image. If the current frame image is the first image, S205 is executed, that is, the first blurred layer of the previous frame image and the first layer are used to perform layer merging to generate an image to be displayed.

[0149] If the current frame image is not the first image, the current frame image is the second image, and S206 is executed, that is, blurring other layers in the current frame image to generate a first blurred layer.

[0150] Among them, the second image can synthesize its other layers with the mask layer, and provide the first image with a first blurred layer for layer synthesis. That is to say, other layers in the second image are blurred (that is, other layers and the mask layer are synthesized) to generate a first blurred layer, so that the first image can reuse the first blurred layer generated by the second image for layer synthesis, so the second image can be called a copy frame. When it is determined that the current frame image has met the preset conditions, that is, when the electronic device begins to adopt the first strategy for layer synthesis, the image to be processed is usually arranged in the order of one copy frame adjacent to one skip frame. At this time, if the current frame image is not the first image (skip frame image), the current frame image is the second image (copy frame image).

[0151] S205: Use the first blurred layer and the first layer to perform layer synthesis to generate an image to be displayed.

[0152] The first strategy refers to that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed, and the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image. Other layers refer to layers other than the first layer in the previous frame image, and the first image refers to an image with a corresponding first blurred layer.

[0153] For example, Fig. 9 As shown, if the current frame image is the first image (skipped frame image), the first blurred layer of the previous frame image and the first layer of the skipped frame image (such as the shutdown layer) are used to merge the layers to generate an image to be displayed.

[0154] S206: Blurring other layers in the current frame image to generate a first blurred layer.

[0155] For example, Fig.10As shown, when the display interface starts to display the first layer, in order to make the first layer more compatible with the size of the display interface of the electronic device, the size of the first layer is usually fine-tuned. At this time, the electronic device sets the displayed image to a normal frame image, that is, each frame image is directly layered through the GPU, and the first blurred layer of the previous frame image is not reused. When the current frame image meets the preset conditions, the current frame image is determined as a copy frame image, and then the layer synthesis process is performed in sequence in the order of one copy frame adjacent to one skip frame. For the skip frame image, the first blurred layer of the previous frame image (i.e., the copy frame image) of the skip frame image and the first layer of the skip frame image can be used to perform layer merging processing to generate an image to be displayed; for the copy frame image (i.e., the second image), other layers in the second image can be blurred to generate a first blurred layer, and other layers refer to layers in the copy frame image except the first layer, and the first image refers to an image with a corresponding first blurred layer. Blurring other layers can refer to layer merging processing of other layers and mask layers. When merging other layers with the mask layer, a GPU may be used to merge other layers with the mask layer.

[0156] S207: Store the first blurred layer.

[0157] After obtaining the first blur layer, the GPU may store the first blur layer in the GPU, so that when processing a skipped frame image, the first blur layer is called from the GPU.

[0158] S208: Perform layer synthesis processing on the first blurred layer and the first layer in the second image to generate an image to be displayed.

[0159] For example, Fig.11 As shown, if the current frame image is the second image (ie, the copy frame image), the other layers in the copy frame image are first merged with the mask layer through the GPU to generate a first blurred layer, and then the first blurred layer is stored in the GPU.

[0160] Optionally, in the process of merging other layers in the copied frame image with the mask layer, the layer parameters of the first layer can also be input into the GPU at the same time, so that the GPU can merge the other layers in the copied frame image with the mask layer according to the layer parameters of the first layer, so that the size of the generated first blurred layer can better match the size of the first layer.

[0161] After generating the first blurred layer, the GPU sends the first blurred layer to the HWC, and then merges the first blurred layer and the first layer of the copy frame image (eg, the shutdown layer) through the HWC to generate an image to be displayed.

[0162] For the copied frame image, it is necessary to first merge the layers except the first layer with the mask layer to obtain the first blurred layer and store the first blurred layer. At the same time, it is also necessary to perform layer synthesis processing on the first blurred layer and the first layer in the copied frame image to generate the image to be displayed.

[0163] If the layer synthesis method provided in the embodiment of the present application is not adopted, Fig.12 As shown in (a), each frame of the image needs to be blurred, which will increase the time it takes for the GPU to process the image. If the layer synthesis method provided in the embodiment of the present application is adopted, Fig.12 As shown in (b) in the figure, blur processing is performed on every other frame of the image, that is, blur processing is performed on the copied frame image, and blur processing is not performed on the skipped frame image. This can save the time for the GPU to perform blur processing, thereby reducing the time for the GPU to perform image processing.

[0164] The layer synthesis method provided by the embodiment of the present application, if the current frame image is the second image, then blurs the other layers in the second image to generate a first blurred layer, and synthesizes the first blurred layer with the first layer in the second image to generate an image to be displayed, thereby avoiding the situation where the second image does not generate an image to be displayed, that is, not only the first layer in the first image is layer-merged to obtain the image to be displayed, but also the first layer in the second image is layer-merged to obtain the image to be displayed, which is equivalent to the first layer in each frame of the image to be displayed after layer synthesis being the updated first layer, thereby avoiding the situation where two adjacent frames of images are layer-synthesized using the same first layer and the first blurred layer, thereby avoiding the situation where two adjacent frames of images display the same image, thereby improving the display effect of the electronic device.

[0165] S209: Display the image to be displayed.

[0166] When the current frame image includes the first layer, before determining whether the current frame image is a skipped frame image or a copied frame image, it is also possible to first determine whether the previous frame image also includes the first layer. If the previous frame image also includes the first layer, it indicates that the electronic device has begun to enter a stable blurred display state. Alternatively, it is also possible to first determine whether the previous frame image also includes the first layer. If the previous frame image also includes the first layer, it is further confirmed whether the size of the first layer in the previous frame image is the same as the size of the first layer in the current frame image. If the size of the first layer in the previous frame image is the same as the size of the first layer in the current frame image, the electronic device has begun to enter a stable blurred display state. After the electronic device has begun stable blurred display, adopting the first strategy for layer synthesis can improve the stability of layer synthesis and reduce the situation where the difference between the blurred display image and the current frame image is large due to the large difference between the previous and next frame images. The following is through Fig.13 The illustrated embodiment is described in detail.

[0167] Fig.13 A schematic diagram of a process flow of another layer synthesis method provided in an embodiment of the present application is shown in FIG. Fig.13 As shown, the method includes:

[0168] S301: Determine whether the current frame image has a first layer.

[0169] When the first layer appears in the current frame image of the electronic device, the electronic device performs global blur display, that is, when the first layer is detected in the current frame image, the electronic device performs blur processing on other layers except the first layer. It should be understood that each layer has its corresponding identification information. When the electronic device detects that the first layer's identification information is present in the current frame image, it means that the first layer is detected in the current image and starts to perform blur processing on the current frame image.

[0170] The layer synthesis module in the application framework layer may include a layer parameter acquisition submodule, which is used to collect parameter changes related to the layer in each frame image and save these parameters. For example, the layer acquisition submodule can acquire the identification information of the layer in the image, and determine whether the first layer exists in the current frame image according to the identification information of the layer. For example, if the identification information obtained by the layer parameter acquisition submodule is the identification information of the shutdown layer, it can be determined that the current frame image includes the shutdown layer, that is, the first layer.

[0171] If the current frame image has a first layer, execute S302.

[0172] S302: Determine whether a first layer exists in the previous frame image.

[0173] If the current frame image has a first layer, execute S303.

[0174] S303: Determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0175] The parameters collected by the layer parameter collection submodule may also include parameters indicating the size change of the layer. For example, if the size of the first layer in the current frame image has changed from that of the first layer in the previous frame image, the parameter indicating the size change of the layer is 1; if the size of the first layer in the current frame image has not changed from that of the first layer in the previous frame image, the parameter indicating the size change of the layer is 0. If the parameter indicating the size change of the layer obtained by the layer parameter collection submodule is 0, the size of the first layer in the current frame image is the same as that of the first layer in the previous frame image, and S304 is executed.

[0176] The layer synthesis method provided in the embodiment of the present application determines that both the current frame image and the previous frame image include a first layer, and when the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, then determines whether to adopt the first strategy for layer synthesis. If the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, it means that the current frame image is an image that stably displays the first layer, rather than the first time the first layer is displayed. This can avoid the situation where the display effect of the current frame image is poor due to the reuse of the first blurred layer of the previous frame image when the first layer is displayed for the first time.

[0177] S304: Determine whether the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images.

[0178] The N previous frames refer to the N images before the current frame. If the size of the first layer in the current frame is the same as the size of the first layer in the N previous images, it means that the current frame is a stable display of the first layer, rather than a period of fine-tuning the size of the first layer in the initial stage of displaying the first layer.

[0179] If the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images, then execute S305.

[0180] The layer synthesis method provided in the embodiment of the present application determines whether to adopt the first strategy to perform layer synthesis when it is determined that the size of the first layer in the current frame image is the same as the size of the first layer in N previous frame images. If the size of the first layer in the current frame image is the same as the first layer in the N previous images, it means that the current frame image is a stable display of the first layer, rather than a period of time when the size of the first layer image needs to be fine-tuned in the early stage of displaying the first layer. This can avoid the situation where the display effect of the current frame image is poor due to the reuse of the first blurred layer of the previous frame image when fine-tuning the first layer.

[0181] S305: Determine whether the number of the first layer in the current frame image is 1.

[0182] If the number of the first layer in the current frame image is 1, execute S306.

[0183] If the number of first layers in the current frame image is 0, or the number of first layers in the current frame image is greater than or equal to 2, then the first strategy for layer synthesis is exited, which is equivalent to the current frame image being a solution frame image for closing the solution.

[0184] S306: Determine whether the current frame image is the first image.

[0185] The first image may refer to an image having the corresponding first blurred layer. In a possible case, the first image may also be called a skipped frame image.

[0186] Optionally, the layer synthesis module also includes a decision submodule, which can be used to determine whether the current frame image is a normal frame image, a copy frame image, or a skip frame image based on the parameters obtained by the layer parameter acquisition submodule, and to close the first strategy. Wherein, closing the first strategy means that the GPU performs blur processing and layer synthesis processing on each frame image, that is, it does not distinguish what type of image the current frame image is, and does not perform corresponding layer synthesis processing based on the type of the current frame image.

[0187] For example, the layer parameter acquisition submodule obtains the identification information of the shutdown layer that exists in both the current frame image and the previous frame image, and the layer parameter acquisition submodule obtains N consecutive parameters indicating the size change of the shutdown layer, that is, the size of the shutdown layer in the current frame image and the shutdown layers in the N previous frames are the same. The decision submodule can determine that the current frame is converted from a normal frame to a copy frame based on the above parameters, and adopt the first strategy for layer synthesis, that is, reuse the first blurred layer of the previous frame image and the shutdown layer of the current frame image to merge the layers to obtain the image to be displayed.

[0188] For example, the layer parameter acquisition submodule obtains the identification information of the shutdown layer existing in both the current frame image and the previous frame image, and the N consecutive parameters indicating the size change of the shutdown layer obtained by the layer parameter acquisition submodule are 0, that is, the size of the shutdown layer in the current frame image and the shutdown layer in the N previous frames are the same, and the decision submodule can determine to adopt the first strategy for image synthesis based on the above parameters. At this time, the decision submodule can further determine whether the current frame image is a skipped frame or a copied frame, and the decision submodule can determine whether the current frame image has a corresponding first blurred layer based on the layer identification information obtained by the layer parameter acquisition submodule, that is, whether the previous frame image of the current frame image stores the first blurred layer.

[0189] If the identification information of the first blurred layer exists in the previous frame image acquired by the layer parameter acquisition submodule, the current frame image is a skipped frame image, that is, the first image.

[0190] If the identification information of the previous frame image acquired by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, then the current frame image is a skipped frame image, that is, the second image.

[0191] If the identification information of the first blurred layer does not exist in the previous frame image acquired by the layer parameter acquisition submodule, the current frame image is a copy frame image, that is, the second image.

[0192] If the identification information of the previous frame image acquired by the layer parameter acquisition submodule indicates that the previous frame image is a skipped frame, then the current frame image is a copy frame image, that is, the second image.

[0193] Alternatively, if the identification information of the N previous frame images acquired by the layer parameter acquisition submodule indicates that the N previous frame images are all normal frames, then the current frame image is a copy frame image, that is, the second image.

[0194] It can be understood that if the decision submodule has decided to adopt the first strategy for layer synthesis, that is, to start layer synthesis processing in the order of copying frames and skipping frames, if it is detected that the size of the first layer of the current frame image is different from the size of the first layer of the previous frame image, then the first strategy for layer synthesis can be exited, which is equivalent to the current frame being a normal frame; or, if it is detected that the current frame image includes at least 2 first layers, the first strategy for layer synthesis can be exited, which is equivalent to the current frame image being a solution off frame image.

[0195] Among the solution off frames, normal frames, copy frames and skip frames, the solution off frames have the highest priority, followed by normal frames, and finally the parallel copy frames and skip frames.

[0196] For example, when the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, the current frame is indicated as a skipped frame; at the same time, it is detected that the current frame image includes two first layers, indicating that the current frame is a solution off frame; since the solution off frame has the highest priority, the current frame image is a solution off frame, not a skipped frame.

[0197] For example, when the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, the current frame is indicated as a skipped frame; at the same time, it is detected that the size of the first layer in the current frame image is different from the size of the first layer in the first frame image in the previous frame image, indicating that the current frame is a normal frame; since the priority of the normal frame is higher than that of the skipped frame image, the current frame image is a normal frame, not a skipped frame image.

[0198] For example, when the identification information of the previous frame image obtained by the layer parameter acquisition submodule indicates that the previous frame image is a copy frame, the current frame is indicated as a skipped frame; and it is detected that the size of the first layer in the current frame image is different from the size of the first layer in the first frame image in the previous frame image, indicating that the current frame is a normal frame; at the same time, it is detected that the current frame image includes 2 first layers, indicating that the current frame is a solution off frame; since the priority of the solution off frame is higher than that of the normal frame and the skipped frame, the current frame image is a solution off frame, rather than a normal frame or a skipped frame image.

[0199] If the current frame image is the first image, execute S307.

[0200] If the current frame image is not the first image, the current frame image is the second image, and S308 is executed.

[0201] S307: Use the first blurred layer and the first layer to perform layer synthesis to generate an image to be displayed.

[0202] Among them, the first strategy means that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, and the first image refers to the image with the corresponding first blurred layer.

[0203] Optionally, the layer synthesis module may also include a frame skipping submodule, which may be used to control the current frame image to skip the GPU synthesis process and directly send the first blurred layer and the first layer to the HWC so that the HWC performs layer synthesis processing on the first blurred layer and the first layer.

[0204] S308: Blurring other layers in the current frame image to generate a first blurred layer.

[0205] S309: Store the first blurred layer.

[0206] S310: Perform layer synthesis processing on the first blurred layer and the first layer in the second image to generate an image to be displayed.

[0207] Optionally, the layer synthesis module also includes a blur cache submodule, which is used to control other layers and mask layers in the copied frame image to perform layer synthesis to obtain a first blurred layer, and then send the parameters of the first blurred layer to the cache framebuffer in the GPU for storage, so that the skipped frame image can call the first blurred layer generated by the copied frame image.

[0208] S311, display the image to be displayed.

[0209] It should be understood that, although each step in the flow chart in the above-described embodiment is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0210] It is understandable that in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0211] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. It should be noted that the names of the modules in the embodiment of the present application are schematic and are not limited to the names of the modules in actual implementation.

[0212] Fig.14 A schematic diagram of the structure of a layer synthesis device provided in an embodiment of the present application.

[0213] It should be understood that the layer synthesis device 600 can execute Figures 5 to 13 The layer synthesis method shown; the layer synthesis device 600 includes: an input unit 610 and a processing unit 620.

[0214] The processing unit 620 is used to determine whether the current frame image meets the preset conditions, and the preset conditions include that the current frame image includes the first layer, wherein when the current frame image includes the first layer, the current frame image is blurred; if the current frame image meets the preset conditions, the first strategy is adopted to perform layer synthesis, and the first strategy means that when performing layer synthesis, if the current frame image is the first image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, and the first image refers to the image with the corresponding first blurred layer; and the image to be displayed is displayed.

[0215] In one embodiment, the first strategy also includes: if the current frame image is the first image, and the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed.

[0216] In one embodiment, the first strategy further includes: if the current frame image is the first image, and the number of first layers included in the current frame image is 1, using the first blurred layer and the first layer to perform layer synthesis to generate an image to be displayed.

[0217] In one embodiment, the first strategy also includes: if the current frame image is the second image, blurring other layers in the current frame image to generate a first blurred layer, and storing the first blurred layer, the second image indicates that there is no image corresponding to the first blurred layer.

[0218] In one embodiment, the first strategy also includes: if the current frame image is the second image, blurring other layers in the current frame image to generate a first blurred layer, storing the first blurred layer, and performing layer synthesis processing on the first blurred layer and the first layer in the second image to generate an image to be displayed.

[0219] In one embodiment, the first strategy also includes: if the current frame image is the first image, the first blurred layer and the first layer are synthesized by the hardware synthesizer HWC to generate an image to be displayed.

[0220] In one embodiment, the first layer is a top layer in the current frame image.

[0221] In one embodiment, the preset condition further includes that the previous frame image includes the first layer.

[0222] In one embodiment, the preset condition also includes that the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

[0223] In one embodiment, the preset condition further includes: the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images.

[0224] In one embodiment, the image to be displayed is a dynamically displayed image.

[0225] The layer synthesis device provided in this embodiment is used to execute the layer synthesis method of the above-mentioned embodiment. The technical principle and technical effect are similar and will not be repeated here.

[0226] It should be noted that the above-mentioned layer synthesis device 600 is embodied in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0227] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0228] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0229] Fig.15 A structural schematic diagram of an electronic device provided by the present application is shown. Fig.15 The dotted line in indicates that the unit or the module is optional. The electronic device 700 can be used to implement the layer synthesis method described in the above method embodiment.

[0230] The electronic device 700 includes one or more processors 701, which can support the electronic device 700 to implement the layer synthesis method in the method embodiment. The processor 701 can be a general-purpose processor or a special-purpose processor. For example, the processor 701 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, such as discrete gates, transistor logic devices or discrete hardware components.

[0231] The processor 701 may be used to control the electronic device 700, execute software programs, and process data of the software programs. The electronic device 700 may also include a communication unit 705 to implement input (reception) and output (transmission) of signals.

[0232] For example, the electronic device 700 may be a chip, the communication unit 705 may be an input and / or output circuit of the chip, or the communication unit 705 may be a communication interface of the chip, and the chip may be a component of a terminal device or other electronic devices.

[0233] For another example, the electronic device 700 may be a terminal device, and the communication unit 705 may be a transceiver of the terminal device, or the communication unit 705 may be a transceiver circuit of the terminal device.

[0234] The electronic device 700 may include one or more memories 702 on which a program 704 is stored. The program 704 can be executed by the processor 701 to generate instructions 703, so that the processor 701 executes the impedance matching method described in the above method embodiment according to the instructions 703.

[0235] Optionally, data may be stored in the memory 702. Optionally, the processor 701 may read data stored in the memory 702. The data may be stored at the same storage address as the program 704, or may be stored at a different storage address than the program 704.

[0236] The processor 701 and the memory 702 may be provided separately or integrated together; for example, integrated on a system on chip (SOC) of a terminal device.

[0237] Exemplarily, the memory 702 can be used to store the related program 704 of the layer synthesis method provided in the embodiment of the present application, and the processor 701 can be used to call the related program 704 of the layer synthesis method stored in the memory 702 when performing layer synthesis, and execute the layer synthesis method of the embodiment of the present application; including: determining whether the current frame image meets the preset conditions, the preset conditions include that the current frame image includes the first layer, wherein when the current frame image includes the first layer, blurring the current frame image; if the current frame image meets the preset conditions, adopting the first strategy to perform layer synthesis, the first strategy means that when performing layer synthesis, if the current frame image is the first image, adopting the first blurred layer and the first layer to perform layer synthesis to generate an image to be displayed, the first blurred layer refers to the blurred layer corresponding to other layers in the previous frame image, the other layers refer to the layers other than the first layer in the previous frame image, and the first image refers to the image with the corresponding first blurred layer; displaying the image to be displayed.

[0238] The present application also provides a computer program product, which, when executed by the processor 701, implements the layer synthesis method described in any method embodiment of the present application.

[0239] The computer program product may be stored in the memory 702 , for example, a program 704 , which is converted into an executable target file that can be executed by the processor 701 after preprocessing, compiling, assembling, and linking.

[0240] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a computer, the layer synthesis method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0241] The computer-readable storage medium is, for example, a memory 702. The memory 702 may be a volatile memory or a nonvolatile memory, or the memory 702 may include both a volatile memory and a nonvolatile memory. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0242] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0243] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0244] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0245] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0246] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0247] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0248] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0249] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A layer synthesis method, characterized in that: include: Determining whether the current frame image meets a preset condition, wherein the preset condition includes that the current frame image includes a first layer, wherein when the current frame image includes the first layer, blurring the current frame image; If the current frame image meets the preset condition, a first strategy is used to perform layer synthesis, wherein the first strategy means that when performing layer synthesis, if the current frame image is a first image, a first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed, wherein the first blurred layer refers to a blurred layer corresponding to other layers in the previous frame image, and the other layers refer to layers in the previous frame image other than the first layer, and the first image refers to an image having the corresponding first blurred layer; The image to be displayed is displayed.

2. The method according to claim 1, characterized in that The first strategy also includes: if the current frame image is the first image, and the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image, the first blurred layer and the first layer are used to perform layer synthesis to generate an image to be displayed.

3. The method according to claim 1 or 2, characterized in that: The first strategy also includes: if the current frame image is the first image and the number of first layers included in the current frame image is 1, using the first blurred layer and the first layer to perform layer synthesis to generate an image to be displayed.

4. The method according to any one of claims 1 to 3, characterized in that: The first strategy also includes: if the current frame image is a second image, blurring the other layers in the current frame image to generate the first blurred layer, and storing the first blurred layer, and the second image indicates that there is no image corresponding to the first blurred layer.

5. The method according to claim 4, characterized in that The first strategy also includes: if the current frame image is a second image, blurring the other layers in the current frame image to generate the first blurred layer, storing the first blurred layer, and performing layer synthesis processing on the first blurred layer and the first layer in the second image to generate the image to be displayed.

6. The method according to any one of claims 1 to 5, characterized in that: The first strategy also includes: if the current frame image is the first image, the first blurred layer and the first layer are layer-synthesized by a hardware synthesizer HWC to generate an image to be displayed.

7. The method according to any one of claims 1 to 6, characterized in that: The first layer is the top layer in the current frame image.

8. The method according to any one of claims 1 to 7, characterized in that: The preset condition also includes that the previous frame image includes the first layer.

9. The method according to any one of claims 1 to 8, characterized in that: The preset condition also includes that the size of the first layer in the current frame image is the same as the size of the first layer in the previous frame image.

10. The method according to claim 9, characterized in that The preset condition also includes: the size of the first layer in the current frame image is the same as the size of the first layer in the N previous frame images.

11. The method according to any one of claims 1 to 10, characterized in that: The image to be displayed is a dynamically displayed image.

12. An electronic device, characterized in that: The electronic device comprises a module for executing the method according to any one of claims 1 to 11.

13. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the method according to any one of claims 1 to 11.

14. A chip system, characterized in that: The chip system includes a processor for calling and running a computer program from a memory, so that an electronic device equipped with the chip system executes the method according to any one of claims 1 to 11.

15. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 11.

Images