A layer synthesis method and related device

Abstract

This application provides a layer synthesis method and related apparatus. In this method, when a user draws an image containing multiple layers on an electronic device, the electronic device synthesizes only the dirty pixels in the layer the user is currently operating (i.e., the active layer) with the pixels in other layers at the same position as the dirty pixels to obtain and display the synthesis result. This reduces the computational complexity and time required for layer synthesis, thereby improving the performance of real-time layer synthesis and enhancing the user experience.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a layer synthesis method and related devices. Background Art

[0002] Currently, users can draw on electronic devices. During the drawing process, users can split their artwork into multiple layers. After the user draws the corresponding content in each layer, the electronic device can combine the multiple layers and display the final result on the display.

[0003] When electronic devices perform layer synthesis, the greater the number of layers to be synthesized, the greater the computational effort required. Excessive computational effort can lead to longer synthesis times and increased latency in the electronic device drawing and displaying the synthesized layers. This can cause lags in the real-time rendering of user-generated artwork, impacting the user experience. Summary of the Invention

[0004] This application provides a layer synthesis method and related apparatus. This method reduces the computational complexity of layer synthesis in electronic devices, improving layer synthesis performance. This allows electronic devices to more quickly present user-drawn artwork in real time, thereby enhancing the user experience.

[0005] In a first aspect, the present application provides a layer synthesis method, which may include: an electronic device displays a first user interface of a target application, wherein a first image is displayed in the first user interface, and the first image includes an activation layer and a background layer; the electronic device detects a first operation on the activation layer; in response to the first operation, the electronic device determines that multiple pixels in the activation layer are dirty pixels, and the dirty pixels include a first pixel; the electronic device synthesizes the first pixel with a second pixel in the background layer to obtain a second image, wherein the position of the second pixel in the background layer is the same as the position of the first pixel in the activation layer, the second image includes the background layer of the first image, and the content of the activation layer in the second image is different from the content of the activation layer in the first image.

[0006] The target applications may include painting applications and image processing applications.

[0007] With the layer synthesis method provided in the first aspect, the electronic device synthesizes only dirty pixels within a layer during layer synthesis. This reduces the computational complexity of layer synthesis and improves layer synthesis performance. The electronic device can more quickly render user-drawn artwork in real time, thereby enhancing the user experience.

[0008] In combination with the first aspect, in one possible implementation, in response to a first operation, the electronic device determines that multiple pixels in the activated layer are dirty pixels, which may specifically include: in response to the first operation, the electronic device saves the depth map of the activated layer in the current frame; based on the depth map, the electronic device determines that multiple pixels in the activated layer are dirty pixels.

[0009] A user operation on an active layer typically involves changes between multiple frames. The current frame may refer to the image frame currently being rendered by the electronic device. The current frame includes not only the content of the active layer but also the content of the background layer. The electronic device may render each frame according to the frame rate. The electronic device may also save the depth map corresponding to the image frame currently being rendered.

[0010] In this way, the electronic device can determine the dirty pixels in the activated layer based on the depth map.

[0011] In combination with the first aspect, in a possible implementation, the electronic device determines, based on the depth map, that multiple pixels in the activated layer are dirty pixels. Specifically, the following may be included: when the first operation is a brush drawing operation and the depth value of the first pixel in the depth map is greater than or equal to a first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value for depth testing in a brush drawing scenario; or, when the first operation is a transformation operation and the depth value of the first pixel in the depth map is greater than or equal to a second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is a threshold value for depth testing in a transformation scenario; or, when the first operation is a filter operation and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is a threshold value for depth testing in a filter scenario; or, when the first operation is a layer adjustment operation and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is a threshold value for depth testing in a layer adjustment scenario; or, when the first operation is a color fill operation and the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is a threshold value for depth testing in a layer adjustment scenario.

[0012] In this way, the electronic device can determine the dirty pixels in the activation layer under different scenarios.

[0013] In conjunction with the first aspect, in one possible implementation, the first value, the second value, and the fifth value may be values ​​that change with the current frame. For example, when the current frame is frame 1, the first value may be 1, the second value may be 1, and the fifth value may also be 1. When the current frame is frame 2, the first value may be 2, the second value may also be 2, and the fifth value may also be 2.

[0014] In conjunction with the first aspect, in one possible implementation, the third value and the fourth value may also be fixed values. That is, regardless of whether the current frame is the N-1th frame or the Nth frame, the third value and the fourth value do not change. For example, the third value may be 1, and the fourth value may be 1.

[0015] In conjunction with the first aspect, in one possible implementation, an electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves a depth map of the active layer in the current frame. Specifically, this may include: in response to the first operation, the CPU determines the depth value of the pixel of the active layer in the current frame and stores it in the CPU's first memory; the CPU instructs the GPU to copy the depth value in the first memory to the GPU's second memory; the GPU writes the depth value to a texture containing dirty pixel information of the active layer in the current frame to generate a depth map. In this way, the CPU in the electronic device can calculate the depth value of the pixel of the active layer in the current frame and can store the depth value in the depth map through the GPU.

[0016] Optionally, in a possible implementation, the GPU may write the depth value into a texture containing dirty pixel information of the activated layer in the current frame through a vertex shader to generate a depth map.

[0017] In conjunction with the first aspect, in one possible implementation, the electronic device determines, based on the depth map, that multiple pixels in the active layer are dirty pixels. This may specifically include: a GPU performing a depth test on the depth map to obtain a depth test result; and the GPU determining, based on the depth test result, that multiple pixels in the active layer are dirty pixels. In this way, the electronic device can determine the dirty pixels in the active layer through the depth test.

[0018] In combination with the first aspect, in a possible implementation method, the electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a second image, which may specifically include: the electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a first synthesis result, the color values ​​of the pixels at the same position as the first pixel in the first synthesis result have been updated, and the color values ​​of the pixels other than the dirty pixels in the first synthesis result have not been updated; the electronic device renders the first synthesis result to obtain the second image.

[0019] Optionally, not updating the color values ​​of pixels other than the dirty pixel in the first synthesis result may also refer to updating the same color value.

[0020] In this way, the electronic device obtains the second image by synthesizing the activated layer and the background layer.

[0021] In combination with the first aspect, in a possible implementation method, the GPU writes the depth value into the texture containing the dirty pixel information of the activated layer in the current frame to generate a depth map, which may specifically include: when the first operation is a brush drawing operation, the GPU writes the depth value of the pixel where the stable point of the activated layer in the current frame is located as a first depth value in the texture containing the dirty pixel information of the activated layer in the current frame, and writes the depth value of the pixel where the unstable point of the activated layer in the current frame is located as a second depth value to generate a depth map, and the second depth value is greater than the first depth value; or, when the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformed content of the activated layer in the current frame is located in the texture containing the dirty pixel information of the activated layer in the current frame The value is written as the third depth value to obtain a depth map; or, in the case where the first operation is a filter operation, the GPU writes the depth value of the pixel with content of the activated layer in the current frame as the fourth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is layer adjustment, the GPU writes the depth value of the pixel with content of the activated layer in the current frame as the fifth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is color filling, the GPU writes the depth value of the newly added fill pixel of the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map.

[0022] In this way, electronic devices can obtain depth maps corresponding to activated layers in different scenarios.

[0023] In combination with the first aspect, in a possible implementation, in response to the first operation, the electronic device determines that multiple pixels in the activated layer are dirty pixels, which may specifically include: when the first operation is a brush drawing operation, the electronic device determines that the pixels where the stable points and unstable points drawn by the user of the activated layer in the current frame are located, and the pixels where the unstable points drawn by the user of the activated layer in the previous frame are located are dirty pixels; or, when the first operation is a transformation operation, the electronic device determines that the pixels where the transformed content in the activated layer in the current frame is located and the pixels where the transformed content of the activated layer in the previous frame is located are dirty pixels; or, when the first operation is a filter operation, the electronic device determines that the pixels where the activated layer has content in the current frame are dirty pixels; or, when the first operation is a layer adjustment operation, the electronic device determines that the pixels where the activated layer has content in the current frame are dirty pixels; or, when the first operation is color filling, the electronic device determines that the pixels where the newly added fill area of ​​the activated layer in the current frame is located are dirty pixels.

[0024] In this way, in different scenarios, the electronic device can determine the dirty pixels in the activated layer.

[0025] In combination with the first aspect, in a possible implementation, after the electronic device determines that multiple pixels in the activated layer are dirty pixels, the method may also include: the electronic device marks the dirty pixels in the activated layer as a sixth value, and marks the pixels other than the dirty pixels in the activated layer as a seventh value to obtain a marking result; and the electronic device saves the marking result.

[0026] The sixth value is different from the seventh value. For example, the sixth value may be 1 and the seventh value may be 0. Alternatively, the sixth value may be 0 and the seventh value may be 1.

[0027] In the labeling result, the label value corresponding to each pixel needs to be stored using 1 bit.

[0028] In this way, the electronic device can mark the dirty pixels in the activation layer through the marking result.

[0029] In combination with the first aspect, in one possible implementation, in response to the first operation, the electronic device determines that multiple pixels in the activated layer are dirty pixels, which may specifically include: in response to the first operation, the electronic device saves the monochrome texture of the activated layer in the current frame, and the monochrome texture contains the depth values ​​of the pixels affected by the first operation in the current frame of the activated layer; based on the monochrome texture, the electronic device determines that multiple pixels in the activated layer are dirty pixels.

[0030] In this way, when the depth map in the electronic device cannot be used in layer synthesis, the electronic device can also determine the dirty pixels in the activated layer through the monochrome texture.

[0031] In combination with the first aspect, in a possible implementation, the electronic device may include a processor CPU and an image processor GPU. In response to the first operation, the electronic device saves the monochrome texture of the activated layer in the current frame. Specifically, it may include: in response to the first operation, the CPU determines the depth value of the pixel of the activated layer affected by the first operation in the current frame, and sends a first drawing instruction to the GPU; in response to the first drawing instruction, the GPU draws the monochrome texture of the activated layer in the current frame, and saves the depth value of the pixel affected by the first operation in the current frame of the activated layer to the monochrome texture.

[0032] In this way, the electronic device can calculate the depth value of the pixel affected by the first operation of the active layer in the current frame through the CPU, and then save the calculated depth value in a monochrome texture through the GPU.

[0033] In combination with the first aspect, in a possible implementation, the electronic device determines that multiple pixels in the activated layer are dirty pixels based on the monochrome texture, which may specifically include: the GPU determines that multiple pixels in the activated layer are dirty pixels based on the monochrome texture.

[0034] In a possible implementation, the GPU may determine, through a fragment shader, that a plurality of pixels in an active layer are dirty pixels.

[0035] In this way, the electronic device can determine the dirty pixels in the activated layer through the GPU.

[0036] In combination with the first aspect, in a possible implementation, when the first operation is a brush drawing operation, the pixels of the activated layer acted upon by the first operation in the current frame are the pixels where the stable points and unstable points drawn by the user are located, the depth value of the pixel where the stable point is located is the first depth value, and the depth value of the pixel where the unstable point is located is the second depth value; or, when the first operation is a transformation operation, the pixels of the activated layer acted upon by the first operation in the current frame are the pixels where the transformed content of the activated layer is located in the current frame, and the depth value of the pixel where the transformed content is located is the third depth value; or, when the first operation is a filter operation, the activated layer The pixels acted upon by the first operation on the layer in the current frame are the pixels on which the activated layer has content in the current frame, and the depth values ​​of the pixels on which the content exists are the fourth depth value; or, when the first operation is a layer adjustment operation, the pixels acted upon by the first operation on the activated layer in the current frame are the pixels on which the activated layer has content in the current frame, and the depth values ​​of the pixels on which the content exists are the fifth depth value; or, when the first operation is a color fill operation, the pixels acted upon by the first operation on the activated layer in the current frame are the pixels on which a newly added color fill area of ​​the activated layer is located in the current frame, and the depth values ​​of the pixels on which the newly added color fill area is located are the sixth depth value.

[0037] In this way, the electronic device can determine the depth value of the pixel on which the first operation is performed in different scenarios.

[0038] In combination with the first aspect, in a possible implementation, a format of a monochrome texture stored in the GPU is different from a format of a depth map stored in the GPU.

[0039] In a second aspect, a layer synthesis method is provided, which may include: an electronic device displays a first user interface of a target application, wherein a first image is displayed in the first user interface, and the first image includes an activation layer, a background layer, and a foreground layer; the electronic device detects a first operation on the activation layer; in response to the first operation, the electronic device saves a depth map of the activation layer in the current frame; based on the depth map, the electronic device determines that multiple pixels in the activation layer are dirty pixels, and the dirty pixels include a first pixel; the electronic device synthesizes the first pixel with a second pixel in the background layer and a third pixel in the foreground layer to obtain a second image, wherein the position of the second pixel in the background layer is the same as the position of the first pixel in the activation layer, and the position of the third pixel in the foreground layer is the same as the position of the first pixel in the activation layer, and the second image includes the background layer of the first image and the foreground layer of the first image, and the content of the activation layer in the second image is different from the content of the activation layer in the first image.

[0040] The target applications may include painting applications and image processing applications.

[0041] With the layer synthesis method provided in the second aspect, the electronic device synthesizes only dirty pixels within a layer during layer synthesis. This reduces the computational complexity of layer synthesis and improves layer synthesis performance. The electronic device can more quickly render user-drawn artwork in real time, thereby enhancing the user experience.

[0042] In combination with the second aspect, in a possible implementation, the electronic device determines, based on the depth map, that multiple pixels in the activated layer are dirty pixels. Specifically, the following may be included: when the first operation is a brush drawing operation and the depth value of the first pixel in the depth map is greater than or equal to a first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value for depth testing in a brush drawing scenario; or, when the first operation is a transformation operation and the depth value of the first pixel in the depth map is greater than or equal to a second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is a threshold value for depth testing in a transformation scenario; or, when the first operation is a filter operation and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is a threshold value for depth testing in a filter scenario; or, when the first operation is a layer adjustment operation and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is a threshold value for depth testing in a layer adjustment scenario; or, when the first operation is a color fill operation and the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is a threshold value for depth testing in a layer adjustment scenario.

[0043] In this way, the electronic device can determine the dirty pixels in the activation layer under different scenarios.

[0044] In conjunction with the second aspect, in one possible implementation, the first value, the second value, and the fifth value may be values ​​that change with the current frame. For example, when the current frame is frame 1, the first value may be 1, the second value may be 1, and the fifth value may also be 1. When the current frame is frame 2, the first value may be 2, the second value may also be 2, and the fifth value may also be 2.

[0045] In conjunction with the second aspect, in one possible implementation, the third value and the fourth value may also be fixed values. That is, regardless of whether the current frame is the N-1th frame or the Nth frame, the third value and the fourth value do not change. For example, the third value may be 1, and the fourth value may be 1.

[0046] In conjunction with the second aspect, in one possible implementation, the electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves a depth map of the active layer in the current frame. Specifically, the process may include: in response to the first operation, the CPU determines the depth value of the pixel of the active layer in the current frame and stores it in the CPU's first memory; the CPU instructs the GPU to copy the depth value in the first memory to the GPU's second memory; the GPU writes the depth value to a texture containing dirty pixel information of the active layer in the current frame to generate a depth map. In this way, the CPU in the electronic device can calculate the depth value of the pixel of the active layer in the current frame and store the depth value in the depth map through the GPU.

[0047] Optionally, in a possible implementation, the GPU may write the depth value into a texture containing dirty pixel information of the activated layer in the current frame through a vertex shader to generate a depth map.

[0048] In conjunction with the second aspect, in one possible implementation, the electronic device determines, based on the depth map, that multiple pixels in the active layer are dirty pixels. This may specifically include: a GPU performing a depth test on the depth map to obtain a depth test result; and the GPU determining, based on the depth test result, that multiple pixels in the active layer are dirty pixels. In this way, the electronic device can determine the dirty pixels in the active layer through the depth test.

[0049] In combination with the second aspect, in a possible implementation method, the electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a second image, which may specifically include: the electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a first synthesis result, the color values ​​of the pixels at the same position as the first pixel in the first synthesis result have been updated, and the color values ​​of the pixels other than the dirty pixels in the first synthesis result have not been updated; the electronic device renders the first synthesis result to obtain the second image.

[0050] Optionally, not updating the color values ​​of pixels other than the dirty pixel in the first synthesis result may also refer to updating the same color value.

[0051] In this way, the electronic device obtains the second image by synthesizing the activated layer and the background layer.

[0052] In combination with the second aspect, in a possible implementation method, the GPU writes the depth value into the texture containing the dirty pixel information of the activated layer in the current frame to generate a depth map, which may specifically include: when the first operation is a brush drawing operation, the GPU writes the depth value of the pixel where the stable point of the activated layer in the current frame is located as the first depth value, and writes the depth value of the pixel where the unstable point of the activated layer in the current frame is located as the second depth value in the texture containing the dirty pixel information of the activated layer in the current frame to generate a depth map, and the second depth value is greater than the first depth value; or, when the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformed content of the activated layer in the current frame is located in the texture containing the dirty pixel information of the activated layer in the current frame The value is written as the third depth value to obtain a depth map; or, in the case where the first operation is a filter operation, the GPU writes the depth value of the pixel with content of the activated layer in the current frame as the fourth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is layer adjustment, the GPU writes the depth value of the pixel with content of the activated layer in the current frame as the fifth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is color filling, the GPU writes the depth value of the newly added fill pixel of the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map.

[0053] In this way, electronic devices can obtain depth maps corresponding to activated layers in different scenarios.

[0054] According to a third aspect, an electronic device is provided, which may include a display, one or more processors and one or more memories; wherein the display, the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the method involved in any possible implementation of the first aspect.

[0055] In a fourth aspect, an electronic device is provided. The electronic device may include one or more functional modules, and the one or more functional modules are used for the method involved in any possible implementation manner of the first aspect.

[0056] In a fifth aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method involved in any possible implementation method of the first aspect.

[0057] In a sixth aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the method involved in any possible implementation of the first aspect.

[0058] In a seventh aspect, a computer program product is provided, which includes a computer program / instruction. When the computer program / instruction is run on an electronic device, the electronic device executes the method involved in any possible implementation of the first aspect.

[0059] It is understandable that the electronic device provided in the third aspect, the electronic device provided in the fourth aspect, the chip system provided in the fifth aspect, the computer-readable storage medium provided in the sixth aspect, and the computer program product provided in the seventh aspect are all used to execute the method provided in this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a schematic diagram of a drawing interface provided by an embodiment of the present application;

[0061] Figure 2 This is a schematic diagram of layer synthesis provided by an embodiment of the present application;

[0062] Figure 3 This is another layer synthesis schematic diagram provided by an embodiment of the present application;

[0063] Figure 4 This is another schematic diagram of layer synthesis provided by an embodiment of the present application;

[0064] Figure 5 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0065] Figure 6 Schematic diagram of the entire hardware architecture of the electronic device provided in the embodiment of the present application;

[0066] Figure 7A This is a schematic diagram of the N-1th image frame of the activated layer in the brush drawing scenario provided by an embodiment of the present application;

[0067] Figure 7B This is a schematic diagram of the Nth image frame of the activated layer in the brush drawing scenario provided by an embodiment of the present application;

[0068] Figure 7C This is a schematic diagram of dirty pixels of the Nth image frame of the activated layer in the brush drawing scenario provided by an embodiment of the present application;

[0069] Figure 8A This is a schematic diagram of the N-1th image frame of the activated layer in the transformation scene provided by an embodiment of the present application;

[0070] Figure 8B This is a schematic diagram of the Nth image frame of the activated layer in the transformation scene provided by an embodiment of the present application;

[0071] Figure 8C This is a schematic diagram of dirty pixels of the Nth image frame of the activated layer in the transformation scenario provided by an embodiment of the present application;

[0072] Figure 9A This is a schematic diagram of the N-1th image frame of the activated layer in the filter scene provided by an embodiment of the present application;

[0073] Figure 9B This is a schematic diagram of the Nth image frame of the activated layer in the filter scenario provided by an embodiment of the present application;

[0074] Figure 9C Schematic diagram of dirty pixels of the Nth image frame of the activated layer in the filter scenario provided by an embodiment of the present application;

[0075] Figure 10A This is a schematic diagram of the N-1th image frame of the activated layer in the layer adjustment scenario provided by an embodiment of the present application;

[0076] Figure 10B This is a schematic diagram of an Nth image frame of an activated layer in a layer adjustment scenario provided by an embodiment of the present application;

[0077] Figure 10C This is a schematic diagram of dirty pixels of the Nth image frame of the activated layer in the layer adjustment scenario provided by an embodiment of the present application;

[0078] Figure 11AThis is a schematic diagram of the N-1th image frame of the activated layer in the color quick fill scenario provided by an embodiment of the present application;

[0079] Figure 11B This is a schematic diagram of the Nth image frame of the activated layer in the color quick fill scenario provided by an embodiment of the present application;

[0080] Figure 11C This is a schematic diagram of dirty pixels of the Nth image frame of the activated layer in the color quick fill scenario provided by an embodiment of the present application;

[0081] Figure 12 This is a schematic diagram of a process for layer synthesis provided by an embodiment of the present application;

[0082] Figure 13A The brush drawing handwriting of the N-1th frame, the brush drawing handwriting of the Nth frame, the depth map corresponding to the N-1th frame, and the depth map corresponding to the Nth frame in the brush drawing scene activation layer provided by the embodiment of the present application;

[0083] Figure 13B The N-1th frame transformation content, the Nth frame transformation content, the N-1th frame transformation content, and the depth map corresponding to the N-1th frame and the depth map corresponding to the Nth frame in the transformation scene activation layer provided by the embodiment of the present application;

[0084] Figure 13C The display content of the N-1th frame before the filter operation, the display content of the Nth frame after the filter operation, and the depth map corresponding to the N-1th frame and the depth map corresponding to the Nth frame in the filter scene activation layer provided by the embodiment of the present application;

[0085] Figure 13D The layer adjustment scene activation layer provided in the embodiment of the present application includes the display content of the N-1th frame before the layer adjustment operation, the display content of the Nth frame after the layer adjustment operation, and the depth map corresponding to the N-1th frame and the depth map corresponding to the Nth frame;

[0086] Figure 13E The color filling area of ​​the N-1th frame, the color filling area of ​​the Nth frame, the depth map corresponding to the N-1th frame, and the depth map corresponding to the Nth frame in the color quick fill scene activation layer provided by the embodiment of the present application;

[0087] Figure 14 This is a schematic diagram of the implementation steps of layer synthesis provided in an embodiment of the present application;

[0088] Figure 15 This is a schematic diagram of a process for layer synthesis provided by an embodiment of the present application;

[0089] Figure 16 This is a schematic diagram of a process for layer synthesis provided by an embodiment of the present application;

[0090] Figure 17 It is a structural diagram of a device 1700 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0091] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0092] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. The terms "first" and "second" are used for descriptive purposes only and are not to be understood as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. "First" and "second" etc. are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first object and the second object are used to distinguish different objects, rather than to describe a specific order of objects.

[0093] In the description of the embodiments of this application, unless otherwise specified, "a plurality" means two or more. For example, a plurality of processing units refers to two or more processing units; a plurality of systems refers to two or more systems.

[0094] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0095] The term "and / or" in this application is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0096] The term "user interface (UI)" in the following embodiments of this application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0097] To better understand the technical solutions provided by this application, before describing the technical solutions of this application, we first describe, with reference to the accompanying drawings, an electronic device 100 with drawing and image processing functions to which this application is applicable. For example, the electronic device 100 includes drawing software or image processing software. In the embodiments of this application, the electronic device 100 may include, but is not limited to, devices with drawing and image processing functions, such as mobile phones, tablet computers, and smart watches. The embodiments of this application do not limit the specific form or type of the electronic device 100.

[0098] First, the technical terms involved in the embodiments of the present application are introduced:

[0099] 1) Pixel: The basic unit of image display, which cannot be divided any further. Each pixel is a small grid of a single color.

[0100] 2) Texture: A data object that contains image information. For example, this data object could be an array of pixels, where each pixel's data represents its color. The array of pixels could represent the entire image. For example, a texture could be the memory that stores a layer or image.

[0101] 3) Image layer: In image processing / painting applications, a layer is a component of an artwork (or image). An artwork can be composed of multiple layers, each containing different visual elements. Layers can be stacked on top of each other and blended to create the final effect. Each layer can be modified independently without affecting the content of other layers.

[0102] 4) Blending: Take multiple layers as input and output a resulting image.

[0103] 5) Synthesize: Take multiple layers as input and output a result image.

[0104] 6) Active layer: The layer that the user is currently modifying.

[0105] 7) Background layer: The layer below the active layer in the layer list.

[0106] 8) Foreground layer: The layer above the active layer in the layer list.

[0107] 9) Painting latency: The delay between the pen tip drawing on the screen and the ink appearing on the screen.

[0108] 10) Local blending: Blend a portion of the layer texture while leaving the rest of the texture unchanged.

[0109] 11) Vertex shader: A shader that performs a series of operations on vertices in the rendering pipeline.

[0110] 12) Fragment shader: The rendering pipeline is responsible for calculating and outputting the color of the shader.

[0111] 13) Dirty region: An area whose content has changed during the rendering process and needs to be redrawn.

[0112] 14) Dirty pixel: A pixel whose content has changed during the rendering process and needs to be redrawn.

[0113] Currently, the painting application or image processing application in the electronic device 100 can have the ability to mix (or synthesize) multiple layers in real time. Taking the painting application as an example, in a painting scene, the user can split the work to be drawn into multiple layers in the painting application of the electronic device 100. The user can draw and adjust the corresponding content on each layer separately. The electronic device 100 can synthesize the content drawn by the user in real time and display the synthesized content. For example, the electronic device 100 can display the following: Figure 1 The drawing interface 101 shown. Figure 1As shown, the drawing interface 101 may include a menu bar 102, which can be used to display layer information corresponding to the currently drawn image. Menu bar 102 may include settings for each layer, such as settings 103, 104, 105, 106, and 107. Settings 103 can be used to set the background layer and display background layer information. Settings 104 can be used to set layer 1 and display layer 1 information. Settings 105 can be used to set layer 2 and display layer 2 information. Settings 106 can be used to set layer 3 and display layer 3 information. Settings 107 can be used to set layer 4 and display layer 4 information. Layer 2 may contain the hat of a cartoon duck drawn by the user, Layer 3 may contain the body of the cartoon duck drawn by the user, and Layer 4 may contain the facial features of the cartoon duck drawn by the user. Layer 4 is the layer currently being drawn by the user. The user draws the facial features of a cartoon duck in layer 4, and the electronic device 100 can synthesize the content being drawn by the user with other layers in real time, and finally obtain a drawn image 109 displayed in the drawing interface 101. The drawn image 109 can be a cartoon duck.

[0114] Optionally, the menu bar 102 may include a control 102a and a control 102b, wherein the control 102a may be used to add a layer, and the control 102b may be used to save an image drawn by the user.

[0115] Alternatively, as Figure 1 As shown, the drawing interface 101 may further include a toolbar 108. The toolbar 108 may include a control 1081 for undoing drawing content, a control 1082 for enabling brush drawing, a control 1083 for enabling transform drawing, a control 1084 for enabling filter operations, a control 1085 for enabling layer adjustments, a control 1086 for enabling Color Draw, and a control 1087 for displaying more drawing tools.

[0116] Figure 1 The drawing interface 101 shown is only an example. In the embodiment of the present application, the drawing interface 101 may include more or fewer interface elements. The embodiment of the present application does not limit the interface elements included in the drawing interface 101.

[0117] When the electronic device 100 performs real-time synthesis of multiple layers of a user-drawn image, the electronic device 100 may synthesize the multiple layers sequentially from bottom to top according to the order in which the layers are arranged. Figure 2 As shown, Figure 1Taking the five layers (background layer, layer 1, layer 2, layer 3, and layer 4) included in the drawn image 109 shown in FIG as an example, layer 1 can be exemplified as the active layer. The electronic device 100 can first blend the background layer with layer 1 to obtain intermediate result 1. Then, the electronic device 100 blends intermediate result 1 with layer 2 to obtain intermediate result 2. Next, the electronic device 100 blends layer 3 with intermediate result 2 to obtain intermediate result 3. Finally, the electronic device 100 blends layer 4 with intermediate result 3 to obtain the final composite result.

[0118] The computational complexity of layer blending is positively correlated with the number of layers. The more layers there are, the greater the computational complexity. When a user draws a large number of layers, the computational complexity of layer blending by the electronic device 100 will be relatively high. This high computational complexity can lead to longer blending times and drawing delays, which can cause real-time rendering of the painting to become lags, affecting the user's painting experience.

[0119] In a possible implementation, in order to reduce drawing delay and improve drawing smoothness, the electronic device 100 may determine the dirty area in the layer and locally synthesize the dirty areas of multiple layers. Figure 3 As shown, the electronic device 100 can calculate the area (or bounding box) affected by the user operation in the active layer 301 based on the user's operation in the active layer, and set it as a dirty area (such as Figure 3 ). When performing layer compositing, electronic device 100 may combine pixels in dirty areas across multiple layers, while maintaining the previously combined result for pixels outside the dirty areas. For example, electronic device 100 may combine pixels within bounding box 3051 in layer 305, pixels within bounding box 3041 in layer 304, pixels within dirty bounding box 3011 in active layer 301, pixels within bounding box 3031 in layer 303, and pixels within bounding box 3021 in layer 302. The position of bounding box 3051 in layer 305 is the same as the position of dirty bounding box 3011 in active layer 301. The position of bounding box 3041 in layer 304 is the same as the position of dirty bounding box 3011 in active layer 301. The position of bounding box 3031 in layer 303 is the same as the position of dirty bounding box 3011 in active layer 301. The position of the bounding box 3021 in the layer 302 is the same as the position of the dirty area bounding box 3011 in the active layer 301 .

[0120] In this way, the electronic device 100 only synthesizes the pixels within the dirty area bounding box, thereby improving the real-time layer synthesis performance by reducing the number of pixels involved in the synthesis.

[0121] However, when the dirty bounding box contains pixels that do not require re-synthesis, the number of pixels in the dirty bounding box that actually need to be blended is far less than the number of dirty pixels. When the electronic device 100 only synthesizes the pixels in the dirty bounding box, the computational complexity is still high. As a result, when the electronic device 100 performs layer synthesis, layer blending may still take a long time, resulting in longer rendering delays, which in turn may cause the real-time rendering of the painting to be jittery, affecting the user experience.

[0122] In another possible implementation, the electronic device 100 may use a background layer pre-synthesis method to perform layer synthesis. When the user performs a drawing operation in the electronic device 100, he is currently only operating one operation, or several layers. The layer that the user is currently operating can be called an active layer. When the user is drawing in the active layer, the layer below the active layer, that is, the content in the background layer will not change. When the electronic device 100 is pre-synthesized based on the background layer, the electronic device 100 can pre-synthesize the background layer in advance and save the result in a temporary texture. When the electronic device 100 performs layer synthesis, it only needs to replace all background layers with temporary textures to mix with other layers. For example, as Figure 4 As shown, when a user draws an image on electronic device 100 that includes background layer 1, background layer 2, background layer 3, an active layer, layer 5, and layer 6, electronic device 100 can pre-synthesize background layer 1, background layer 2, and background layer 3 to obtain a temporary texture. Then, when the user draws on the active layer, electronic device 100 can synthesize the temporary texture with the active layer, layer 5, and layer 6 in real time to obtain the final result. In this way, when electronic device 100 performs real-time synthesis, the number of layers involved in the synthesis can be reduced, thereby reducing the amount of calculation and improving the performance of real-time layer synthesis.

[0123] However, when there are multiple layers above the active layer, the electronic device 100 uses a background pre-compositing method to perform layer compositing. Although this can reduce the computational effort required for the background layers involved in compositing, compositing the multiple layers above the active layer still requires significant computational effort. Consequently, when the electronic device 100 performs layer compositing, layer blending may still take a long time, resulting in increased drawing latency, which can cause lags in the real-time rendering of the painting, impacting the user experience.

[0124] To reduce the computational complexity of layer synthesis, thereby shortening the time required for layer synthesis and improving real-time layer synthesis performance, embodiments of the present application provide a layer synthesis method. In the layer synthesis method provided in embodiments of the present application, when electronic device 100 receives a user drawing operation on an activated layer, electronic device 100 identifies dirty pixels, synthesizes the dirty pixels from multiple layers, obtains a synthesis result, and displays the synthesis result. In this way, electronic device 100 only performs local synthesis on dirty pixels, which can reduce the number of pixels involved in layer synthesis, thereby improving layer synthesis performance.

[0125] The following introduces an exemplary electronic device 100 provided in an embodiment of the present application.

[0126] Figure 5 1 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application.

[0127] The following embodiments are described in detail using electronic device 100 as an example. It should be understood that electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0128] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0129] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0130] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0131] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0132] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0133] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0134] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus interfaces. The processor 110 can be coupled to the touch sensor 180K, charger, flash, camera 193, etc. through different I2C bus interfaces.

[0135] The I2S interface can be used for audio communication.

[0136] The PCM interface can also be used for audio communication to sample, quantize and encode analog signals.

[0137] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.

[0138] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0139] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0140] The SIM interface can be used to communicate with the SIM card interface 195 to implement the function of transmitting data to the SIM card or reading data in the SIM card.

[0141] The USB interface 130 is an interface that complies with USB standards, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transfer data between the electronic device 100 and peripheral devices.

[0142] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0143] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0144] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0145] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0146] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0147] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0148] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0149] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0150] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0151] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0152] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0153] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0154] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0155] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0156] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals.

[0157] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0158] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0159] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0160] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, applications required for at least one function (such as face recognition function, fingerprint recognition function, mobile payment function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as face information template data, fingerprint information template, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0161] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0162] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0163] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0164] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0165] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0166] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0167] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0168] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100 .

[0169] The air pressure sensor 180C is used to measure air pressure.

[0170] The magnetic sensor 180D includes a Hall sensor, and the electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case.

[0171] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes).

[0172] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance by infrared or laser.

[0173] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode.

[0174] The ambient light sensor 180L is used to sense the brightness of the ambient light.

[0175] The fingerprint sensor 180H is used to collect fingerprints.

[0176] The temperature sensor 180J is used to detect temperature.

[0177] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0178] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0179] Motor 191 can generate vibration prompts.

[0180] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0181] The SIM card interface 195 is used to connect a SIM card.

[0182] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0183] Figure 6A schematic diagram of the software and hardware architecture of a layer synthesis service on an electronic device 100 provided in an embodiment of the present application is shown.

[0184] like Figure 6 As shown, the hardware and software architecture of electronic device 100 includes a software system and a hardware layer that implements layer synthesis together with the software structure. The layered architecture of electronic device 100 divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: from top to bottom, the application layer, the application framework layer, the Android runtime and system library, the rendering hardware interface, and the kernel layer.

[0185] The application layer can include a series of application packages.

[0186] like Figure 6 As shown, an application package may include a target application. In some examples, the target application is an image processing application, a drawing application, or other application with image display and processing capabilities. In some examples, the target application is configured with functional logic such as a user interface, artwork management, layer management, and drawing functions. Drawing functions include, but are not limited to, brush drawing, filters, transformations, layer adjustments, and color quick fill.

[0187] Optionally, the application package may further include applications such as contacts, memos, calls, clipboard, gallery, map, camera, video, etc.

[0188] It is understandable that the application packages included in the electronic devices of different users may be different, and the embodiment of the present application does not limit the specific application packages included in the electronic device 100.

[0189] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0190] like Figure 6 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0191] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0192] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0193] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0194] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0195] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0196] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating electronic devices, or flashing indicator lights.

[0197] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0198] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0199] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0200] The system library may include a rendering engine. It may also include multiple other functional modules, such as a surface manager, media libraries, a 3D graphics library (e.g., OpenGL ES), and a 2D graphics engine (e.g., SGL).

[0201] In some embodiments, the drawing engine is used to implement the painting or image processing functions of the target application. The drawing engine includes, for example, modules such as layer synthesis, brush drawing, filter operation, transformation operation, and rendering pipeline. These modules can configure the rendering pipeline according to their own logical assembly, and the rendering pipeline is used to generate rendering instructions. For example, the layer synthesis module is used to perform a real-time layer synthesis process. During the real-time layer synthesis process, the layer synthesis module can be used to determine the dirty pixels in the activated layer, and locally synthesize the pixels in multiple layers that have the same position as the dirty pixels with the dirty pixels. In addition, before the layer synthesis, the layer synthesis module triggers the rendering pipeline to generate rendering instructions to instruct the electronic device 100 to synthesize the rendered layers.

[0202] The rendering pipeline can also include vertex shaders and fragment shaders. The vertex shader can be used to calculate the vertex color, texture coordinates, etc. in the rendered image. The fragment shader can be used to calculate the color of each pixel in the rendered image and color each pixel.

[0203] The surface manager is used to manage the display subsystem and provide the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0204] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0205] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0206] The two-dimensional graphics engine is a drawing engine for 2D drawing. In some examples, the drawing engine includes a three-dimensional graphics processing library and a two-dimensional graphics engine.

[0207] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0208] In some examples, the kernel layer and the system library include interfaces for the CPU to call GPU storage and computing functions, such as rendering hardware interfaces. In some examples, the rendering hardware interfaces include direct extension (DX) 12 interfaces, Metal interfaces, Vulkan interfaces, OpenGL interfaces, etc.

[0209] like Figure 6 As shown, the hardware layer may include a GPU, which may be used to perform dirty point calculations, image rendering, and the like.

[0210] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0211] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a still image or video through the camera 193.

[0212] The following describes how the electronic device 100 determines dirty pixels in the active layer with reference to the accompanying drawings.

[0213] Figures 7A-11C It demonstrates dirty pixels in scenarios such as brush painting, transformation operations, filter operations, layer adjustments, and color quick filling.

[0214] Figures 7A-7C Shows the dirty pixels in the brush-painted scene. Figure 7A As shown in Figure (a), the drawing interface 700 of the electronic device 100 can display the N-1 frame display content. The N-1 frame display content can include content contained in multiple layers (for example, background layer, layer 1, layer 2, layer 3 and layer 4). The user can draw with a brush in layer 4 and draw a drawing handwriting 701. The solid line part in the drawing handwriting 701 can be called a stable point, and the dotted line part in the drawing handwriting 701 is an unstable point. When the brush is drawn in each frame, the stable points and unstable points drawn by the user can be displayed in the current frame. A stable point can be a point that will not change once it falls on the canvas, and an unstable point can be a point that falls on the canvas and will be cleared in the next frame. The stable point will still be displayed in the next frame, and the unstable point will disappear in the next frame.

[0215] like Figure 7A As shown in FIG. 7 (a), the drawing interface 700 may also include a menu bar 702. The menu bar 702 includes setting items corresponding to multiple layers, such as a setting item 703 corresponding to layer 4, a setting item 704 corresponding to layer 3, a setting item 705 corresponding to layer 2, a setting item 706 corresponding to layer 1, and a setting item 707 corresponding to the background layer. The user can select setting item 703 to start brush drawing on layer 4.

[0216] like Figure 7AAs shown in FIG. 1 (a), the drawing interface 700 may further include a toolbar 108. A control 1082 for enabling brush drawing in the toolbar 108 may be highlighted to prompt the current user that brush drawing is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0217] exist Figure 7A In Figure (a), the user is operating layer 4, so layer 4 can be called the active layer. The layers below layer 4 can be called background layers.

[0218] Taking an image frame of the electronic device 100 containing 6*6 (* represents a multiplication sign) pixels as an example, the brush drawing content of the user in the N-1 frame can be as follows: Figure 7A As shown in Figure (b). Figure 7A In Figure (b), one grid represents one pixel. The brush drawing content of the N-1th frame can be displayed in 6*6 grids, that is, 6*6 pixels.

[0219] To facilitate readers’ understanding, 7A to 7C , Figures 8A to 8C The pixels shown in the figure and other figures are somewhat simplified, but for those skilled in the art, considering the pixel density of mobile phones or tablet electronic devices, these simplifications will not cause obstacles for technicians to understand the context. Figure 7A , the mobile phone or tablet will use a large number of pixels to present the user with N-1 frames of brushes corresponding to the user's operation.

[0220] The user can continue to paint with the brush in layer 4. For example, Figure 7B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 700. The Nth frame display content includes the stable point in the drawing handwriting 701 and the drawing handwriting 708. The drawing handwriting 708 can be the user's brush drawing content in the Nth frame. For example, the Nth frame brush drawing content can be as follows: Figure 7B As shown in Figure (b).

[0221] By comparing the brush drawing content of the Nth frame with the brush drawing content of the N-1th frame, the pixels that the user has changed in the Nth frame can be obtained. The pixels in the brush drawing content of the Nth frame that are different from the brush drawing content of the N-1th frame are the pixels that the user has changed in the Nth frame. The pixels that the user has changed in the Nth frame can be called dirty pixels. For example, the electronic device 100 can Figure 7C The N-1 frame brush drawing content shown in Figure (a) is the same as Figure 7CCompared with the brush drawing content of the Nth frame shown in Figure (b), the following is obtained: Figure 7C The brush in frame N shown in Figure (c) paints dirty pixels. Figure 7C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Figure 7C The content of the brush drawing in frame N-1 shown in (a) is Figure 7A Figure (b) shows the brush drawing content of frame N-1. Figure 7C The content of the brush drawing in frame N shown in (b) is Figure 7B Figure (b) shows the brush drawing content of the Nth frame.

[0222] Figures 8A-8C Shows the dirty pixels in the transformation operation scenario. Figure 8A As shown in FIG. 8 (a), the drawing interface 800 of the electronic device 100 may display the N-1 frame of display content. The N-1 frame of display content may include content contained in multiple layers (e.g., the background layer, layer 1, and layer 2). The user can perform a transformation operation in layer 2 and obtain a transformation result 805.

[0223] like Figure 8A As shown in FIG. 1 (a), the drawing interface 800 of the electronic device 100 may further include a menu bar 801. This menu bar 801 may include setting items corresponding to multiple layers, such as a setting item 802 corresponding to Layer 2, a setting item 803 corresponding to Layer 1, and a setting item 804 corresponding to the background layer. When a user needs to operate Layer 2, the user may select this setting item 802. After selecting this setting item 802, the user may perform transformation operations within Layer 2.

[0224] like Figure 8A As shown in FIG. 8( a ), the drawing interface 800 may further include a toolbar 108 , and a control 1083 for enabling a transformation operation in the toolbar 108 may be highlighted to prompt the current user that a transformation operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0225] The user performs a transformation operation in layer 2, and the resulting transformation result 805 can be displayed in an image frame containing 6*6 pixels. Figure 8A As shown in (b) of FIG. 5 , the transformation result of the N-1th frame, that is, the transformation result 805, can be displayed in 6*6 grids, where each grid represents a pixel.

[0226] The user can continue to perform transformation operations on layer 2. When transforming and drawing, all the drawing results of the transformation operation of the previous frame disappear, and only the transformation results of the current frame are displayed. Figure 8B As shown in FIG. 8( a ), the electronic device 100 can display the Nth frame display content in the drawing interface 800 . The Nth frame display content includes a transformation result 806 . The transformation result 806 can be the result obtained by the user through the transformation operation when the electronic device 100 displays the Nth frame. For example, the Nth frame image frame contains 6*6 pixels, and the Nth frame transformation result can be as follows: Figure 8B As shown in (b) in .

[0227] By comparing the Nth frame transformation result with the N-1th frame transformation result, the pixels that the user has changed in the Nth frame can be obtained. The pixels in the Nth frame transformation result that are different from the N-1th frame transformation result are the pixels that the user has changed in the Nth frame. The pixels that the user has changed in the Nth frame can be called dirty pixels. For example, the electronic device 100 can Figure 8C The N-1th transformation result shown in (a) of FIG, and Figure 8C Compared with the Nth frame transformation result shown in (b), we get Figure 8C The dirty pixels of the Nth frame are transformed as shown in Figure (c). Figure 8C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Figure 8C The transformation result of the N-1th frame shown in Figure (a) is Figure 8A Figure (b) shows the transformation result of the N-1th frame. Figure 8C The transformation result of the Nth frame shown in Figure (b) is Figure 8B Figure (b) shows the transformation result of the Nth frame.

[0228] Figures 9A-9C Shows the dirty pixels in the filter operation scenario. Figure 9A As shown in FIG. 1 (a), the drawing interface 900 of the electronic device 100 may display the N-1 frame of display content. The N-1 frame of display content may include content contained in multiple layers (e.g., the background layer, layer 1, and layer 2). Before the user performs a filter operation, the N-1 frame of display content may include the target content 905.

[0229] like Figure 9AAs shown in FIG. 1 (a), the drawing interface 900 of the electronic device 100 may further include a menu bar 901. The menu bar 901 may include setting items corresponding to multiple layers, for example, a setting item 902 corresponding to layer 2, a setting item 903 corresponding to layer 1, and a setting item 904 corresponding to the background layer. When a user needs to operate layer 2, the user may select the setting item 902. After the user selects the setting item 902, the electronic device 100 may display the content of layer 2, i.e., the target content 905.

[0230] like Figure 9A As shown in FIG. 1 (a), the drawing interface 900 may further include a toolbar 108. A control 1084 for enabling filter operation in the toolbar 108 may be highlighted to prompt the current user that the filter operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0231] The content displayed before the filter operation of the N-1 frame, that is, the target content 905 can be displayed in an image frame containing 6*6 pixels. Figure 9A As shown in (b), the display content of the N-1th frame before the filter operation, that is, the target content 905 can be displayed in 6*6 grids, where each grid represents a pixel.

[0232] Users can Figure 9A In response to the filter operation, the electronic device 100 can add a filter to the target content 905 in layer 2 to obtain the target content 906. For example, Figure 9B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 900, and the display content includes the target content 906. Taking the Nth frame image frame containing 6*6 pixels as an example, the display content after the Nth frame filter operation can be as follows: Figure 9B As shown in (b) in .

[0233] By comparing the displayed content after the filter operation on the Nth frame with the displayed content before the filter operation on the N-1th frame, the pixels that the user has changed in the Nth frame can be obtained. The pixels in the displayed content after the filter operation on the Nth frame that are different from the pixels in the displayed content before the filter operation on the N-1th frame are the pixels that the user has changed in the Nth frame. The pixels that the user has changed in the Nth frame can be called dirty pixels. In the filter operation scenario, the user's filter operation will change the color or style of all pixels with content. Therefore, the electronic device 100 can determine that the pixels whose color is changed by the filter operation are dirty pixels. Therefore, in the filter scenario, the dirty pixels can be all pixels with content in the current layer. Exemplarily, the electronic device 100 can Figure 9CThe content displayed before the filter operation of the N-1 frame shown in (a) is the same as Figure 9C Compared with the display content after the Nth frame filter operation shown in (b), the following is obtained: Figure 9C The Nth frame filter shown in Figure (c) operates on dirty pixels. Figure 9C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Figure 9C The content displayed before the filter operation of the N-1 frame shown in Figure (a) is Figure 9A Figure (b) shows the display content of the N-1th frame before the filter operation. Figure 9C The content displayed after the Nth frame filter operation shown in Figure (b) is Figure 9B Figure (b) shows the display content after the Nth frame filter operation.

[0234] Figures 10A-10C Shows the dirty pixels in the layer operation scene. Figure 10A As shown in FIG. 1 (a), the drawing interface 1000 of the electronic device 100 may display the N-1 frame of display content. The N-1 frame of display content may include content contained in multiple layers (e.g., the background layer, layer 1, and layer 2). Before the user adjusts the layer, the N-1 frame of display content may include the target content 1005.

[0235] like Figure 10A As shown in Figure (a) of FIG, the drawing interface 1000 of the electronic device 100 may further include a menu bar 1001. This menu bar 1001 may include setting items corresponding to multiple layers, for example, a setting item 1002 corresponding to layer 2, a setting item 1003 corresponding to layer 1, and a setting item 1004 corresponding to the background layer. When a user needs to operate layer 2, the user may select setting item 1002. After the user selects setting item 1002, the electronic device 100 may display the content of layer 2, namely, the target content 1005.

[0236] like Figure 10A As shown in FIG. 100 (a), the drawing interface 1000 may further include a toolbar 108. A control 1085 for enabling layer transformation operations in the toolbar 108 may be highlighted to prompt the current user that the layer transformation operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0237] The content displayed before the filter operation of the N-1 frame, that is, the target content 1005 can be displayed in an image frame containing 6*6 pixels. Figure 10A As shown in (b) of FIG. 3 , the display content of the N-1 frame before the filter operation, that is, the target content 1005 can be displayed in 6*6 grids, where each grid represents a pixel.

[0238] Users can Figure 10A In response to the layer adjustment operation, the electronic device 100 may perform layer adjustment (e.g., adjust visibility, adjust opacity) on the target content 1005 in layer 2. Figure 10B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 1000, and the display content includes the target content 1005 after the layer adjustment. For example, the Nth frame image frame includes 6*6 pixels, and the display content after the layer adjustment operation of the Nth frame can be as follows: Figure 10B As shown in (b) in .

[0239] By comparing the displayed content after the layer adjustment operation of the Nth frame with the displayed content before the layer adjustment operation of the N-1th frame, the pixels that the user has changed in the Nth frame can be obtained. The pixels in the displayed content after the layer adjustment operation of the Nth frame that are different from the pixels in the displayed content before the layer adjustment operation of the N-1th frame are the pixels that the user has changed in the Nth frame. The pixels that the user has changed in the Nth frame can be called dirty pixels. In the layer adjustment scenario, when the user performs a layer adjustment operation, the visibility and / or opacity of all pixels with content in the current layer will change. Therefore, in the layer adjustment scenario, the electronic device 100 can determine that all pixels with content are dirty pixels. Exemplarily, the electronic device 100 can Figure 10C The display content before the layer adjustment operation of the N-1 frame shown in (a) is the same as Figure 10C Compared with the display content after the N-th frame layer adjustment operation shown in (b), the following is obtained: Figure 10C The dirty pixels of the Nth frame layer are adjusted as shown in Figure (c). Figure 10C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Figure 10C The content displayed before the layer adjustment operation of the N-1 frame shown in Figure (a) is Figure 10A Figure (b) shows the display content of the N-1 frame before the layer adjustment operation. Figure 10C The content displayed after the Nth frame layer adjustment operation shown in Figure (b) is Figure 10B Figure (b) shows the display content after the Nth frame layer adjustment operation.

[0240] Figures 11A-11CShows the dirty pixels in the color quick fill scene. Figure 11A As shown in FIG. 1 (a), the drawing interface 1100 of the electronic device 100 may display the N-1 frame of display content. The N-1 frame of display content may include content contained in multiple layers (e.g., the background layer, layer 1, and layer 2). After the user performs a color quick fill (also referred to as a color fill) operation, the N-1 frame of display content may include a color fill area 1105.

[0241] like Figure 11A As shown in FIG. 1 (a), the drawing interface 1100 of the electronic device 100 may further include a menu bar 1101. The menu bar 1101 may include setting items corresponding to multiple layers, such as a setting item 1102 corresponding to layer 2, a setting item 1103 corresponding to layer 1, and a setting item 1104 corresponding to the background layer. When a user needs to operate layer 2, the user may select the setting item 1102. After the user selects the setting item 1102, the electronic device 100 may display layer 2, which may include a fill area 1105.

[0242] like Figure 11A As shown in FIG. 1 (a), the drawing interface 1100 may further include a toolbar 108. A control 1086 for enabling the color quick fill operation in the toolbar 108 may be highlighted to prompt the current user that the color quick fill operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0243] The N-1 frame display content can be displayed in an image frame containing 6*6 pixels. For example, Figure 11A As shown in FIG. 1( b ), the display content of the N-1th frame (including the color-filled area 1105 ) can be displayed in 6*6 grids, where each grid represents a pixel.

[0244] Users can Figure 11A In response to the color quick fill operation, the electronic device 100 may display a color fill area 1106 in layer 2. For example, as shown in FIG. Figure 11B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 1000, and the display content includes a fill area 1106. Taking the Nth frame image frame including 6*6 pixels as an example, the Nth frame display content can be as follows: Figure 11B As shown in (b) in .

[0245] By comparing the color-filled area of ​​the Nth frame with the color-filled area of ​​the N-1th frame, the pixels that the user has changed in the Nth frame can be obtained. The pixels in the color-filled area of ​​the Nth frame that are different in color from the color-filled area of ​​the N-1th frame are the pixels that the user has changed in the Nth frame. The pixels that the user has changed in the Nth frame can be called dirty pixels. In the color-filling scene, the color diffuses out from the color-filling starting point at a certain speed until the entire area to be filled is filled. For pixels that have been filled with color and mixed in the previous frame, the current frame does not need to repeat the mixing, and only needs to mix the newly added color-filled pixels. Therefore, in the color-filling scene, the dirty pixels are the newly added color-filled pixels in each frame. For example, the electronic device 100 can Figure 11C The N-1 frame color-filled area shown in (a) of FIG. Figure 11C Compared with the Nth frame color-filled area shown in (b), the following is obtained: Figure 11C The dirty pixel of the Nth frame is shown in Figure (c). Figure 11C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Figure 11C The color-filled area of ​​frame N-1 shown in (a) is Figure 11A Figure (b) shows the color-filled area of ​​frame N-1. Figure 11C The Nth frame filled area shown in Figure (b) is Figure 11B Figure (b) shows the color-filled area of ​​the Nth frame.

[0246] Thus, when the electronic device 100 determines dirty pixels in scenarios such as brush drawing, transformation operations, filter operations, layer adjustments, and color quick fill, the electronic device 100 can synthesize the dirty pixels in the active layer of the user-drawn image with pixels in other layers at the same location as the dirty pixels to obtain a synthesis result, which the electronic device 100 can display. In this way, when performing layer synthesis, the electronic device 100 only synthesizes dirty pixels, which can reduce the computational complexity of layer synthesis and shorten the time consumed by layer synthesis, thereby improving the performance of real-time layer synthesis and enhancing the user experience.

[0247] Figure 12 The following is a schematic diagram of a process flow of a layer synthesis method provided in this application example. Figure 12 As shown, a layer synthesis method provided in an embodiment of the present application may include the following steps:

[0248] S1201. The electronic device 100 displays a user interface 1. The user interface 1 includes an image 1. The image 1 includes multiple layers, including an active layer and a background layer.

[0249] The user interface 1 may be an image processing interface or an image drawing interface in an image processing application or a painting application. Figure 1 The drawing interface 101 is shown in FIG.

[0250] Taking the user interface 1 as an image drawing interface as an example, the user can draw an image in the user interface 1, and the electronic device 100 can also display the image drawn by the user in the user interface 1. The user draws a part of the content in the user interface 1, and the electronic device can display the part of the content drawn by the user in real time. In other words, the electronic device 100 can display the user's drawing handwriting or other drawing operations (for example, transformation operations, filter operations, etc.) in real time. For example, the user interface 1 can include an image 1, which can be Figure 1 The drawn image 109 shown in FIG. 1 may also be Figure 7A (a) includes an image of a triangle, a square, and a drawing handwriting 701. The embodiment of the present application does not specifically limit the image 1.

[0251] The image 1 may include multiple layers, which may include an active layer and a background layer. The embodiment of the present application does not limit the specific content displayed in the active layer and the background layer of the image 1.

[0252] Optionally, the multiple layers may include multiple background layers, and the embodiment of the present application does not limit the number of background layers.

[0253] Optionally, the image 1 may further include one or more foreground layers, which is not limited in this embodiment of the present application.

[0254] S1202: The electronic device 100 detects the user's operation 1 for activating a layer.

[0255] The electronic device 100 can detect the user's operation 1 on the activated layer. The operation 1 can be any one of brush drawing, transformation operation, filter operation, layer adjustment and color quick filling performed by the user in the activated layer. The embodiment of the present application does not specifically limit the operation 1.

[0256] S1203. In response to operation 1, the electronic device 100 records texture information 1, where the texture information 1 includes dirty pixel information in the active layer.

[0257] In response to operation 1, the electronic device 100 may record texture information 1, where the texture information 1 may include dirty pixel information in the active layer.

[0258] For example, in one possible implementation, a depth map may be stored in the memory of the electronic device 100. The depth map may be used to record the depth information of each frame of the rendered image. The texture information 1 may be the depth information recorded in the depth map.

[0259] Optionally, the depth map can be 16 bits, that is, the value of each pixel in the depth map can be a uint16 type ranging from 0 to 65535. uint16 is an unsigned integer type that occupies 16 bits (or 2 bytes) of storage space. The embodiment of the present application does not limit the storage format of the depth map. The depth map involved in the embodiment of the present application can be saved in any achievable format.

[0260] For different user operation scenarios, the method for the electronic device 100 to generate depth texture may be as shown in Table 1 below.

[0261] Table 1

[0262] Scenario The depth value of the depth texture of the Nth frame Brush Painting The pixel depth value of the stable point in the Nth frame is N; the pixel depth value of the unstable point in the Nth frame is N+1 Transform The converted pixel depth value of frame N is N+1 Filters All pixels with content have a depth value of 1 Layer Adjustment All pixels with content have a depth value of 1 Color Fast The depth value of the newly added fill pixel in the Nth frame is N

[0263] As shown in Table 1, in the brush drawing scene, the pixel depth value of the stable point drawn by the user in the Nth frame is N, and the pixel depth value of the unstable point drawn by the user in the Nth frame is N+1. N is a natural number greater than or equal to 1, that is, N=1, 2, 3, .... For example, Figure 13A As shown, Figure 13A Figure (a) shows the brush drawing content in frame N-1. Figure 13A Figure (b) shows the brush drawing content in frame N. Figure 13A The depth map corresponding to the brush drawing content in frame N-1 in (a) is Figure 13A (c) shows the depth map of the brush-drawn scene in frame N-1. Figure 13A In the depth map of the brush-drawn scene in frame (c) shown in FIG, the pixel depth value of the stable point drawn by the brush in frame (N-1) is N-1; the pixel depth value of the unstable point drawn by the brush in frame (N-1) is N. Figure 13A The depth map corresponding to the brush drawing content in frame N in (b) is Figure 13A The brush drawing scene (d) shows the depth map of the Nth frame. Figure 13A In the depth map of frame N of the brush-drawn scene shown in Figure (d), the depth value of the pixel at the stable point of the brush-drawn scene in frame N is N; the depth value of the pixel at the unstable point of the brush-drawn scene in frame N is N+1. Optionally, the depth value corresponding to pixels without brush-drawn traces in the depth map may be 0, which is not limited in this embodiment of the application.

[0264] It is understood that for an image drawn by a user on electronic device 100, there is only one corresponding depth map stored in the memory of electronic device 100. Upon detecting the N-1th frame of brushstrokes, electronic device 100 will adjust the depth values ​​of the pixels containing the brushstrokes based on the user's brushstrokes, while the depth values ​​of the remaining pixels remain unchanged. Upon detecting the Nth frame of brushstrokes, electronic device 100 will adjust the depth values ​​of the pixels containing the Nth frame of brushstrokes in the depth map, while the depth values ​​of the remaining pixels remain unchanged.

[0265] As shown in Table 1, in the transformation scene, the transformation pixel depth value of the Nth frame is N+1. For example, Figure 13B As shown, Figure 13B Figure (a) is the transformation result of the N-1 frame. Figure 13B Figure (b) shows the transformation result of the Nth frame. Figure 13B The depth map corresponding to the transformation result of the N-1th frame in (a) is Figure 13B (c) shows the depth map of the N-1th frame of the transformed scene. Figure 13B In the depth map of the transformed scene frame (C) shown in FIG, the pixel where the transformed content is located in the frame (N-1) has a depth value of N, that is, a depth value of the transformed pixel. Figure 13B The depth map corresponding to the brush drawing content in frame N in (b) is Figure 13B The depth map of the Nth frame of the transformed scene is shown in (d). Figure 13B In the depth map of the Nth frame of the transformed scene shown in (d), the depth value of the transformed pixel in the Nth frame is N+1. Similarly, the memory of the electronic device 100 only stores one depth map. After the electronic device 100 records the depth value of the transformed pixel corresponding to the transformation result of the N-1th frame in the depth map, it will continue to record the depth value of the transformed pixel of the Nth frame in the depth map. If the transformed pixel of the Nth frame is the same as the transformed pixel of the N-1th frame, then in the Nth frame, the electronic device 100 will adjust the depth value corresponding to the transformed pixel. For example. The depth value N corresponding to the transformed pixel in the N-1th frame is adjusted to the depth value N+1 corresponding to the Nth frame.

[0266] As shown in Table 1, in the filter scene, the depth value of all pixels with content (also referred to as drawn content) in the Nth frame is 1. For example, Figure 13C As shown, Figure 13C Figure (a) shows the display content before the filter operation of frame N-1. Figure 13C Figure (b) shows the display content after the filter operation on the Nth frame. Figure 13C In (a) of the figure, the depth map corresponding to the display content of the N-1 frame before the filter operation is Figure 13C (c) shows the depth map of the N-1th frame of the filter scene. Figure 13CIn the depth map of the N-1th frame of the filter scene shown in (c), the depth value of all pixels with content in the N-1th frame is 1. Figure 13C The depth map corresponding to the content displayed after the Nth frame filter operation in Figure (b) is Figure 13C The depth map of the Nth frame of the filtered scene is shown in (d). Figure 13C In the depth map of the Nth frame of the filter scene shown in (d), the depth value of all pixels with content in the Nth frame is 1. Similarly, a depth map is stored in the memory of the electronic device 100. After the electronic device 100 records the depth value of the pixels with content in the N-1th frame in the depth map, it will continue to record the depth value of the content in the Nth frame in the depth map. In the filter scene, as long as the user performs a filter operation, the color value of all pixels with content in the N-1th frame will change. In this scenario, the depth maps corresponding to the N-1th frame and the Nth frame are the same.

[0267] As shown in Table 1, in the layer adjustment scenario, the depth value of all pixels with content in the Nth frame is 1. For example, Figure 13D As shown, Figure 13D Figure (a) shows the display content before the N-1 frame layer adjustment operation. Figure 13D Figure (b) shows the displayed content after the layer adjustment operation of the Nth frame. Figure 13D The depth map corresponding to the display content before the N-1 layer adjustment operation in Figure (a) is Figure 13D The depth map of the N-1 frame of the layer adjustment scene is shown in (c). Figure 13D In the depth map of the N-1 frame of the layer adjustment scene shown in (c), the depth value of all pixels with content in the N-1 frame is 1. Figure 13D The depth map corresponding to the displayed content after the Nth frame layer adjustment operation in Figure (b) is Figure 13D The layer adjustment scene (d) shows the depth map of the Nth frame. Figure 13D In the depth map of the Nth frame of the layer adjustment scenario shown in (d), the depth value of all pixels with content in the Nth frame is 1. Similarly, the memory of the electronic device 100 only stores one depth map. After the electronic device 100 records the depth value of the pixels with content in the N-1th frame in the depth map, it will continue to record the depth value of the content in the Nth frame in the depth map. In the layer adjustment scenario, as long as the user performs a layer adjustment operation, the visibility and / or opacity of all pixels with content in the N-1th frame will change. The depth maps corresponding to the N-1th frame and the Nth frame are the same.

[0268] As shown in Table 1, in the color quick fill scene, the depth value of the newly added color-filled pixel in the Nth frame is N. For example, Figure 13E As shown, Figure 13E Figure (a) is the color-filled area of ​​frame N-1. Figure 13E Figure (b) shows the color-filled area of ​​the Nth frame. Figure 13E The depth map corresponding to the color-filled area of ​​the N-1 frame in (a) is Figure 13E (c) shows the depth map of the N-1 frame of the color quick fill scene. Figure 13E In the depth map of the N-1 frame of the color quick fill scene shown in (c), the depth value of the newly added color-filled pixel in the N-1 frame is N-1. Figure 13E The depth map corresponding to the colored area of ​​the Nth frame in (b) is Figure 13E The color quick fill scene (d) shows the depth map of the Nth frame. Figure 13E In the depth map of the Nth frame of the color quick fill scene shown in Figure (d), the depth value of the newly added fill pixel in the Nth frame is N. Similarly, the electronic device 100 only stores one depth map in its memory. After recording the depth value of the newly added fill pixel in the N-1th frame in the depth map, the electronic device 100 continues to record the depth value of the newly added fill pixel in the Nth frame in the depth map.

[0269] An operation performed by the user on an activated layer generally involves changes between multiple frames.

[0270] S1204. The electronic device 100 determines dirty pixels in the active layer based on the texture information 1.

[0271] The electronic device 100 may determine dirty pixels in the active layer based on the texture information 1, for example, the depth map, and calculate whether each pixel in the depth map is a dirty pixel.

[0272] In one possible implementation, the CPU of the electronic device 100 can calculate the depth value of the pixels that need to be updated in the Nth image frame of the activated layer. In the brush drawing scene, the pixels that need to be updated in the Nth image frame are the pixels where the stable points and unstable points drawn by the user are located. In the transformation scene, the pixels that need to be updated in the Nth image frame are the pixels where the user's transformation operation content is located. In the filter scene, the pixels that need to be updated in the Nth image frame are the pixels where content exists. In the layer adjustment scene, the pixels that need to be updated in the Nth image frame are the pixels where content exists. In the color quick fill scene, the pixels that need to be updated in the Nth image frame are the newly added color fill pixels. The depth values ​​calculated by the CPU in different scenes can be found in Table 1 above. The CPU can send the calculated depth value to the GPU.

[0273] In one possible implementation, the CPU may store the calculated depth value in the CPU's memory, and then the CPU may instruct the GPU to copy the calculated depth value to the GPU's memory. The GPU then calls the vertex shader's drawing instructions to write the depth value into texture information 1 to generate a depth map. The GPU may call the drawing instructions of the rendering pipeline for synthesis to perform a depth test and determine the dirty pixels in the depth map. For example, for different user operation scenarios, the method for calculating the dirty pixels in the depth map based on the depth test may be shown in the following table.

[0274] Table 2

[0275] Scenario Dirty pixels in frame N based on depth test Brush Painting Pixels with depth values ​​greater than or equal to N Transform Pixels with depth values ​​greater than or equal to N Filters Pixels with a depth value of 1 Layer Adjustment Pixels with a depth value of 1 Color Fast Pixels with depth values ​​greater than or equal to N

[0276] As shown in Table 2, in a brush drawing scenario, the electronic device 100 may determine that pixels with depth values ​​greater than or equal to N in the depth map are dirty pixels. Figure 13A As shown in (d), in the depth map of the Nth frame of the scene drawn by the brush, pixels with depth values ​​of N and depth values ​​of N+1 are both dirty pixels.

[0277] As shown in Table 2, in the transformation scene, the electronic device 100 may determine that the pixels with depth values ​​greater than or equal to N in the depth map are dirty pixels. Figure 13B As shown in (d), in the depth map of the Nth frame of the transformed scene, pixels with a depth value of N and a depth value of N+1 are both dirty pixels.

[0278] As shown in Table 2, in the filter scene, the electronic device 100 can determine that the pixels with a depth value of 1 in the depth map are dirty pixels. Figure 13C As shown in (d), in the depth map of the Nth frame of the filter scene, all pixels with a depth value of 1 are dirty pixels.

[0279] As shown in Table 2, in the layer adjustment scenario, the electronic device 100 may determine that the pixels with a depth value of 1 in the depth map are dirty pixels. Figure 13D As shown in (d), in the depth map of the Nth frame of the layer-adjusted scene, all pixels with a depth value of 1 are dirty pixels.

[0280] As shown in Table 2, in a color quick fill scenario, the electronic device 100 may determine that pixels with depth values ​​greater than or equal to N in the depth map are dirty pixels. Figure 13E As shown in (d), in the depth map of the Nth frame of the color quick fill scene, the pixels with a depth value of N are all dirty pixels.

[0281] Optionally, in one possible implementation, after the electronic device 100 determines the dirty pixels in the depth map according to the method in Table 2, it can store the dirty pixel marking result in the cache. The dirty pixel marking result can be used to mark whether the i-th pixel in the depth map is a dirty pixel.

[0282] S1205. The electronic device 100 synthesizes the dirty pixels in the activated layer with the pixels in the background layer at the same position as the dirty pixels, and generates image 2. Image 2 includes multiple layers included in image 1, and the content of the activated layer in image 2 is different from the content of the activated layer in image 1.

[0283] After the electronic device 100 determines the dirty pixels in the Nth frame of the active layer, the electronic device 100 may synthesize the dirty pixels. For the Nth image frame, the electronic device 100 may synthesize the dirty pixels in the active layer with the pixels in the background layer at the same location as the dirty pixels. For other pixels in the active layer that are not dirty pixels, the electronic device 100 does not update their color values. Finally, the electronic device 100 may fuse the synthesized results of the dirty pixels with the synthesized results of the non-dirty pixels to obtain Image 2. Image 2 may be the Nth frame image, and Image 1 may be the N-1th frame image.

[0284] For example, in a brush drawing scenario, the image 1 may be as follows: Figure 7A The N-1th frame is shown in FIG. (a) in FIG. Image 2 can be Figure 7B The Nth image frame is shown in FIG. (a) in FIG. In the transformation scene, the image 1 can be as follows Figure 8A The N-1th frame is shown in FIG. (a) in FIG. Image 2 can be Figure 8B The Nth image frame is shown in (a) of FIG. In the filter scene, image 1 can be as follows Figure 9A The N-1th frame is shown in FIG. (a) in FIG. Image 2 can be Figure 9B The Nth image frame is shown in FIG. (a) in FIG. In the layer adjustment scene, image 1 can be as follows Figure 10A The N-1th frame is shown in FIG. (a) in FIG. Image 2 can be Figure 10B The Nth image frame is shown in FIG. (a) in FIG. In the color fast filling scene, the image 1 can be as follows Figure 11A The N-1th frame is shown in FIG. (a) in FIG. Image 2 can be Figure 11B The Nth image frame is shown in FIG. 1 (a) of FIG. 1 . The embodiment of the present application does not specifically limit image 1 and image 2.

[0285] The specific steps of the electronic device 100 for performing layer synthesis can be found in Figure 14 .like Figure 14As shown, the electronic device 100 may perform layer synthesis including the following steps:

[0286] S1401: The CPU of the electronic device 100 calculates a depth value required to perform a depth test and transmits the depth value to the GPU.

[0287] After the CPU sets the depth value for depth testing to the vertex shader in the rendering pipeline, it can instruct the GPU to perform depth testing through the rendering pipeline. The CPU can calculate the depth value for depth testing in different scenarios, as described in Table 2 above.

[0288] S1402: The GPU of the electronic device 100 performs a depth test on the depth map to determine whether the pixels in the depth map pass the depth test; if so, execute step S1403.

[0289] The GPU can call the rendering pipeline for compositing and perform a depth test based on the depth value set for performing the depth test and the depth map stored in the GPU. When performing the depth test, the GPU can determine that the pixels in the depth map that meet the depth value for performing the depth test are dirty pixels.

[0290] The GPU can perform a depth test on the depth map. During the depth test, the GPU can identify dirty pixels in the depth map according to the method for determining dirty pixels under different user operation scenarios in Table 2 above. For example, when the GPU determines that the current scene is a brush drawing scene or a transformation scene, or a color quick fill scene, the GPU can determine that the pixels with depth values ​​greater than or equal to N in the depth map corresponding to the Nth image frame can pass the depth test; the GPU can determine that the pixels with depth values ​​less than N in the depth map corresponding to the Nth image frame cannot pass (or are called failed) the depth test. For another example, when the GPU determines that the current scene is a filter scene or a layer adjustment scene, the GPU can determine that the pixels with depth values ​​of 1 in the depth map corresponding to the Nth image frame can pass the depth test; the GPU can determine that the pixels with depth values ​​other than 1 (for example, 0) in the depth map corresponding to the Nth image frame cannot pass (or are called failed) the depth test.

[0291] After the user selects an operating mode in user interface 1, the target application corresponding to that operating mode can send the corresponding operating scenario to the rendering engine. The rendering engine, on the CPU side, generates drawing instructions corresponding to different operating scenarios by assembling the rendering pipeline. These drawing instructions can carry the user's operating scenario. The rendering engine can then send the drawing instructions to the GPU. The GPU can use the user operating scenario carried in the drawing instructions to determine the current user operating scenario and the corresponding method for calculating dirty pixels in the depth map.

[0292] S1403 . The GPU of the electronic device 100 synthesizes the pixels that have passed the depth test to obtain a synthesis result 1 .

[0293] The electronic device 100 can synthesize pixels that pass the depth test, that is, dirty pixels. That is, the dirty pixels of the active layer are synthesized with the pixels in the background layer that have the same position as the dirty pixels. For example, if the dirty pixel in the active layer is the pixel in the i-th row and j-th column, then the electronic device 100 can synthesize the dirty pixel with the pixel in the i-th row and j-th column in the background layer and update the color value of the pixel. The GPU of the electronic device 100 does not update the color value of pixels that fail the depth test.

[0294] After the GPU in the electronic device 100 synthesizes all layers, a synthesis result 1 may be obtained. The synthesis result 1 may include the color values ​​of all dirty pixels of the active layer in the current frame and the color values ​​of all non-dirty pixels.

[0295] S1404 . The GPU of the electronic device 100 renders the synthesis result 1 to obtain image 2 .

[0296] The GPU of electronic device 100 can render synthesis result 1, which includes the color values ​​of all pixels in the Nth image frame, into image 2. Electronic device 100 can then display image 2. Image 2 is the image that the user can see in user interface 1 after performing operation 1. Image 2 can also be called the Nth image frame.

[0297] It is understood that before the user performs operation 1, electronic device 100 displays image 1 in user interface 1, and image 1 can be stored in memory 1 of electronic device 100. After the user performs operation 2, electronic device 100 displays image 2 in user interface 1, and electronic device 100 can store image 2 in memory 1, overwriting image 1. In other words, image 1 and image 2 will not be stored in memory 1 at the same time. Image 2 will reuse memory 1 that stores image 1. Memory 1 can be runtime memory that the GPU requests from internal memory to store images displayed by the target application.

[0298] In this way, the electronic device 100 can fully utilize the depth test capability of the rendering pipeline to determine dirty pixels when performing layer synthesis, and only synthesize the dirty pixels, while skipping synthesis of pixels that are not dirty pixels. In this way, the performance of layer synthesis can be improved.

[0299] Optionally, in the layer synthesis method provided in the embodiment of the present application, the electronic device 100 may not save the texture containing the dirty pixel information of each image frame, but directly use the difference between each image frame to calculate the dirty pixel, mark the dirty pixel, and save the marking result. In this way, the memory of the electronic device 100 can be saved. For example, Figure 15 As shown, a layer synthesis method provided in an embodiment of the present application may include the following steps:

[0300] S1501. The electronic device 100 displays a user interface 1. The user interface 1 includes an image 1. The image 1 includes multiple layers, including an active layer and a background layer.

[0301] For step S1501, please refer to the description of step S1201 above, which will not be repeated here.

[0302] S1502: The electronic device 100 detects the user's operation 1 for activating a layer.

[0303] For step S1502, please refer to the description of step S1202 above, which will not be repeated here.

[0304] S1503. In response to operation 1, the electronic device 100 calculates the dirty pixels in the active layer.

[0305] In response to operation 1, the electronic device 100 can calculate the dirty pixel in the Nth image frame in the active layer based on the difference between the Nth image frame and the N-1th image frame in the active layer. User operations in different scenarios in the electronic device 100 can correspond to different dirty pixel identification methods.

[0306] Specifically, in response to operation 1, the CPU may generate a drawing instruction and send the drawing instruction to the GPU. Based on the drawing instruction, the GPU calls the rendering pipeline to calculate the dirty pixels in the active layer. The GPU may calculate the dirty pixels of the active layer in different scenes according to the identification method of dirty pixels in different scenes shown in Table 3 below.

[0307] For example, dirty pixel identification methods for different scenarios in the electronic device 100 may be as shown in Table 3.

[0308] Table 3

[0309]

[0310] As shown in Table 3, in the brush drawing scenario, the electronic device 100 can determine that the dirty pixels of the Nth image frame in the active layer are the pixels where the unstable points are located in the N-1th image frame, the pixels where the stable points are located in the Nth image frame, and the pixels where the unstable points are located in the Nth image frame. For example, in the brush drawing scenario, the dirty pixels of the Nth image frame can be as follows: Figure 7C As shown in Figure (c).

[0311] As shown in Table 3, in a transformation scenario, the electronic device 100 may determine that the dirty pixels of the Nth image frame in the active layer are the pixels where the transformation content is located in the N-1th image frame and the pixels where the transformation content is located in the Nth image frame. For example, in a transformation scenario, the dirty pixels of the Nth image frame may be as follows: Figure 8C As shown in Figure (c).

[0312] As shown in Table 3, in the filter scenario, the electronic device 100 can determine that the dirty pixels of the Nth image frame in the active layer are all pixels with content in the Nth image frame. For example, in the filter scenario, the dirty pixels of the Nth image frame can be as follows: Figure 9C As shown in Figure (c).

[0313] As shown in Table 3, in the layer adjustment scenario, the electronic device 100 can determine that the dirty pixels of the Nth image frame in the active layer are all pixels with content in the Nth image frame. For example, in the layer adjustment scenario, the dirty pixels of the Nth image frame can be as follows: Figure 10C As shown in Figure (c).

[0314] As shown in Table 3, in the color quick fill scenario, the electronic device 100 can determine that the dirty pixels of the Nth image frame in the active layer are all the newly added color-filled pixels in the Nth image frame. For example, in the color quick fill scenario, the dirty pixels of the Nth image frame can be as follows: Figure 11C As shown in Figure (c).

[0315] For example, in a possible implementation, when the user selects a control for enabling the brush drawing function in the user interface 1 (eg, Figure 1 When the user selects the control for enabling the transformation function in the user interface 1 (for example, Figure 1 When the user selects a control for enabling the filter function in the user interface 1 (e.g., Figure 1When the user selects the control for enabling the layer adjustment function in the user interface 1 (for example, Figure 1 When the user selects the control 1805 shown in the figure, the electronic device 100 can determine that the current scene is a layer adjustment scene. Figure 1 When the control 1086 shown in FIG is displayed, the electronic device 100 can determine that the current scene is a color quick fill scene.

[0316] The electronic device 100 can calculate the dirty pixel of each pixel in the Nth image frame in the manner shown in Table 3. Then, the electronic device 100 can mark each pixel in the Nth image frame. For example, the electronic device 100 can mark the dirty pixel of the Nth image frame as a value L1, and mark the pixel of the Nth image frame that is not a dirty pixel as a value L2. The value L1 is different from the value L2. For example, the value L1 can be 1, and the value L2 can be 0. The embodiment of the present application does not limit the values ​​of the value L1 and the value L2.

[0317] The electronic device 100 may store the dirty pixel calculation result of the Nth frame in the memory. A memory space may exist in the memory of the electronic device 100 for storing the dirty pixel storage result. When the electronic device 100 displays the N-1th image frame of the activated layer, the dirty pixel storage result stored in the memory space is the dirty pixel calculation result of the N-1th image frame. When the electronic device 100 displays the Nth frame of the activated layer, the electronic device 100 may first clear the memory space used to store the dirty pixel calculation result, and then store the dirty pixel calculation result of the Nth image frame. In the dirty pixel calculation result, each pixel only needs to use 1 bit to be saved.

[0318] S1504. The electronic device 100 synthesizes the dirty pixels in the activated layer with the pixels in the background layer at the same position as the dirty pixels, and generates image 2. Image 2 includes multiple layers included in image 1, and the content of the activated layer in image 2 is different from the content of the activated layer in image 1.

[0319] After electronic device 100 determines the dirty pixels in the Nth frame of the active layer, it may synthesize the dirty pixels. For the Nth image frame, electronic device 100 may synthesize the dirty pixels in the active layer with pixels in the background layer at the same location as the dirty pixels. For other pixels in the active layer that are not dirty pixels, electronic device 100 does not change their color values. Finally, electronic device 100 may render the result of synthesizing multiple layers to obtain Image 2. Image 2 may be the Nth frame image, and Image 1 may be the N-1th frame image.

[0320] Step S1504 may refer to the description of the above step S1204 and will not be repeated here.

[0321] In this way, when the electronic device 100 performs layer synthesis, it only synthesizes dirty pixels, which can improve synthesis performance. In addition, each pixel in the dirty pixel result stored by the electronic device 100 only requires one bit of memory, which can save memory of the electronic device 100.

[0322] Optionally, in the layer synthesis method provided in the embodiment of the present application, the electronic device 100 may record the dirty pixel information in a single-channel monochrome texture, such as an r channel, a g channel, or a b channel.

[0323] When the electronic device 100 performs layer synthesis using a fragment shader, it checks each pixel in the monochrome texture to see if it is a dirty pixel. If the fragment shader determines that a pixel in the monochrome texture is a dirty pixel, the electronic device 100 synthesizes the pixel using the fragment shader. If the fragment shader determines that a pixel in the monochrome texture is not a dirty pixel, synthesis is skipped for that pixel.

[0324] For example, Figure 16 As shown, a layer synthesis method provided in an embodiment of the present application may include the following steps:

[0325] S1601. The electronic device 100 displays a user interface 1. The user interface 1 includes an image 1. The image 1 includes multiple layers, including an active layer and a background layer.

[0326] Step S1601 may refer to the description of step S1201 above and will not be repeated here.

[0327] S1602: The electronic device 100 detects the user's operation 1 for activating a layer.

[0328] Step S1602 may refer to the description of step S1202 above and will not be repeated here.

[0329] S1603. In response to operation 1, the electronic device 100 records texture information 2, where the texture information 2 includes dirty pixel information in the active layer.

[0330] In response to operation 1, the electronic device 100 may save texture information 2 for each image frame in the activated layer, where the texture information 2 includes the depth value of the dirty pixel. The texture information 2 may be saved in a single-channel uint16 format. It is understood that the texture information 2 may be saved in other formats, not limited to a single-channel uint16 format, and this embodiment of the present application does not limit the format of the texture information 2.

[0331] S1604. When the GPU of the electronic device 100 performs synthesis through the fragment shader, the fragment shader determines whether each pixel in the active layer is a dirty pixel based on the texture information 2. When it is determined that pixel 1 in the active layer is a dirty pixel, pixel 1 in the active layer is synthesized with the pixel in the background layer at the same position as pixel 1, and image 2 is generated. Image 2 includes multiple layers included in image 1, and the content in the active layer in image 2 is different from the content in the active layer in image 1.

[0332] The electronic device 100 does not need to first calculate the dirty pixels in the Nth image frame in the active layer, but directly inputs the monochrome texture containing dirty information of the Nth image frame (i.e., texture information 2) into the fragment shader for synthesis. When the GPU synthesizes through the fragment shader, the fragment shader can first determine whether each pixel in the Nth image frame is a dirty pixel. If so, synthesis is performed, and if not, synthesis is skipped. That is, when synthesizing, the fragment shader needs to determine whether each pixel is a dirty pixel. The Nth image frame in the active layer contains pixel 1 and pixel 2. When it is determined that pixel 1 in the active layer is a dirty pixel, pixel 1 in the active layer is synthesized with the pixel in the background layer at the same position as pixel 1 to obtain synthesis result 1; when it is determined that pixel 2 in the active layer is not a dirty pixel, the color value of pixel 2 is not changed.

[0333] For example, the fragment shader can determine whether a pixel is a dirty pixel based on the depth value of pixel 1 in the Nth image frame in the active layer. The color texture containing dirty information stores the depth value of pixel 1. The fragment shader determines the dirty pixels in the active layer according to the dirty pixel identification methods in different scenarios shown in Table 2.

[0334] After electronic device 100 determines the dirty pixels in the Nth frame of the active layer, it may synthesize the dirty pixels. For the Nth image frame, electronic device 100 may synthesize the dirty pixels in the active layer with pixels in the background layer at the same location as the dirty pixels. For other pixels in the active layer that are not dirty pixels, electronic device 100 does not change their color values. Finally, electronic device 100 may render the result of synthesizing multiple layers to obtain Image 2. Image 2 may be the Nth frame image, and Image 1 may be the N-1th frame image.

[0335] Thus, when the electronic device 100 determines dirty pixels in scenarios such as brush drawing, transformation operations, filter operations, layer adjustments, and color quick fill, the electronic device 100 can synthesize the dirty pixels in the active layer of the user-drawn image with pixels in other layers at the same location as the dirty pixels to obtain a synthesis result, which the electronic device 100 can display. In this way, when performing layer synthesis, the electronic device 100 only synthesizes dirty pixels, which can reduce the computational complexity of layer synthesis and shorten the time consumed by layer synthesis, thereby improving the performance of real-time layer synthesis and enhancing the user experience.

[0336] In the embodiment of the present application, the first image may be image 1, and the second image may be image 2. The first value may be N, the second value may be N, the third value may be 1, the fourth value may be 1, the fifth value may be N, the sixth value may be value L1, and the seventh value may be value L2. The first depth value may be N, the second depth value may be N+1, the third depth value may be N+1, the fourth depth value may be 1, the fifth depth value may be 1, and the sixth depth value may be N. The embodiment of the present application does not limit the specific values ​​of the first value, the second value, the third value, the fourth value, the fifth value, the sixth value, the seventh value, the first depth value, the second depth value, the third depth value, the fourth depth value, the fifth depth value, and the sixth depth value.

[0337] For example, a comparison of the frame rate of the layer synthesis method according to the embodiment of the present application and the frame rate of the layer synthesis method based on the dirty region bounding box can be shown in Table 4 below.

[0338] Table 4

[0339]

[0340]

[0341] As shown in Table 4, in brush painting, transformation, filter, and layer adjustment scenarios, regardless of whether the number of layers to be synthesized is 50, 100, or 150, the frame rate of the layer synthesis method provided by the embodiment of the present application is higher than that of the layer synthesis method based on dirty bounding boxes. This application can improve the performance of real-time layer synthesis in multiple scenarios of painting applications.

[0342] In one embodiment, the present application also provides a device 1700. Figure 17As shown, the apparatus 1700 may include a processor 1701 (e.g., a CPU) and a memory 1702. The memory 1702 may include a high-speed random-access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one disk storage. The memory 1702 may store various instructions for performing various processing functions and implementing the method steps of the present application.

[0343] Optionally, the device 1700 involved in this application may further include: a power supply 1703, a communication bus 1704, and a communication port 1705. The communication port 1705 is used to enable communication between the electronic device and other peripheral devices. In the embodiment of the present application, the memory 1702 is used to store computer-executable program code, which includes instructions; when the processor 1701 executes the instructions, the instructions cause the processor 1701 of the device 1700 to perform the actions performed by the processor of the electronic device 100 in the above-mentioned method embodiment. The implementation principles and technical effects are similar and will not be repeated here.

[0344] Optionally, the apparatus 1700 involved in the present application may further include: a display screen 1706. The display screen 1706 is configured to display an interface of the electronic device.

[0345] It should be noted that the modules or components described in the above embodiments may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code, such as a controller. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0346] Optionally, the device 1700 may further include a GPU, which may be an independent chip, or may be integrated into the CPU 1701 .

[0347] The device 1700 can be used to execute a layer synthesis method executed by the above-mentioned electronic device 100 and any possible implementation of the layer synthesis method.

[0348] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0349] In some examples, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor that is implemented by reading software code stored in a memory.

[0350] In some examples, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0351] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0352] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0353] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the above-mentioned related steps to implement the layer synthesis method in the above-mentioned embodiment.

[0354] An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the layer synthesis method in the above-mentioned embodiment.

[0355] In addition, embodiments of the present application also provide a device. This device can be a component or module, and can include one or more processors and a memory connected together. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the layer composition method described in each of the above method embodiments.

[0356] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0357] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner or can be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0358] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0359] As used in the above embodiments, the term “when…” may be interpreted to mean “if…” or “after…” or “in response to determining…” or “in response to detecting…”, depending on the context. Similarly, the phrases “upon determining…” or “if (stated condition or event) is detected” may be interpreted to mean “if determining…” or “in response to determining…” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0360] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0361] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A layer synthesis method, characterized in that: The method comprises: The electronic device displays a first user interface of a target application, wherein the first user interface displays a first image, the first image including an active layer and a background layer; wherein the target application is a painting application; The electronic device detects a first operation on the activation layer; In response to the first operation, the electronic device determines that a plurality of pixels in the active layer are dirty pixels, the dirty pixels including a first pixel; The electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a second image, the position of the second pixel in the background layer is the same as the position of the first pixel in the activation layer, the second image includes the background layer of the first image, and the content of the activation layer in the second image is different from the content of the activation layer in the first image.

2. The method according to claim 1, characterized in that In response to the first operation, the electronic device determines that a plurality of pixels in the active layer are dirty pixels, specifically comprising: In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame; The electronic device determines, based on the depth map, that a plurality of pixels in the activation layer are dirty pixels.

3. The method according to claim 2, characterized in that The electronic device determines, based on the depth map, that a plurality of pixels in the active layer are dirty pixels, specifically comprising: When the first operation is a brush drawing operation and the depth value of the first pixel in the depth map is greater than or equal to a first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value of a depth test in a brush drawing scenario; or When the first operation is a transformation operation and the depth value of the first pixel in the depth map is greater than or equal to a second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is a threshold value for depth testing in a transformation scenario; or When the first operation is a filter operation and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is a threshold value for depth testing in a filter scenario; or When the first operation is a layer adjustment operation and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is a threshold value for depth testing in a layer adjustment scenario; or When the first operation is a color filling operation, and when the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is a threshold value for depth testing in a layer adjustment scenario.

4. The method according to claim 2, characterized in that The electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame, specifically including: In response to the first operation, the CPU determines a depth value of a pixel of the activated layer in a current frame and stores the depth value in a first memory of the CPU; The CPU instructs the GPU to copy the depth value in the first memory to the second memory of the GPU; The GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map; The electronic device determines, based on the depth map, that a plurality of pixels in the active layer are dirty pixels, specifically comprising: The GPU performs a depth test on the depth map to obtain a depth test result; The GPU determines that multiple pixels in the active layer are dirty pixels based on the depth test result.

5. The method according to any one of claims 1 to 4, characterized in that The electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a second image, specifically comprising: The electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a first synthesis result, wherein the color values ​​of pixels in the first synthesis result that are located at the same position as the first pixel are updated, and the color values ​​of pixels other than the dirty pixel in the first synthesis result are not updated; The electronic device renders the first synthesis result to obtain a second image.

6. The method according to claim 4, characterized in that The GPU writes the depth value into a texture containing dirty pixel information of the active layer in the current frame to generate a depth map, specifically including: When the first operation is a brush drawing operation, the GPU writes, in a texture containing dirty pixel information of the active layer in the current frame, depth values ​​of pixels at which stable points of the active layer are located in the current frame as first depth values, and writes depth values ​​of pixels at which unstable points of the active layer are located in the current frame as second depth values, to generate the depth map, wherein the second depth value is greater than the first depth value; or When the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformed content of the active layer in the current frame is located in the texture containing the dirty pixel information of the active layer in the current frame as the third depth value to obtain the depth map; or When the first operation is a filter operation, the GPU writes the depth values ​​of pixels of the active layer having content in the current frame as fourth depth values ​​in a texture containing dirty pixel information of the active layer in the current frame to obtain the depth map; or When the first operation is layer adjustment, the GPU writes the depth values ​​of pixels of the active layer having content in the current frame as fifth depth values ​​in a texture containing dirty pixel information of the active layer in the current frame to obtain the depth map; or When the first operation is color filling, the GPU writes the depth value of the newly added filled pixels of the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map.

7. The method according to claim 1, characterized in that In response to the first operation, the electronic device determines that a plurality of pixels in the active layer are dirty pixels, specifically comprising: In a case where the first operation is a brush drawing operation, the electronic device determines pixels at which stable points and unstable points drawn by the user of the activated layer in the current frame are located, and pixels at which unstable points drawn by the user of the activated layer in the previous frame are located as dirty pixels; or In the case where the first operation is a transformation operation, the electronic device determines that pixels in the active layer where the transformation content is located in the current frame and pixels in the active layer where the transformation content is located in the previous frame are dirty pixels; or In the case where the first operation is a filter operation, the electronic device determines that pixels of the activated layer having content in the current frame are dirty pixels; or In the case where the first operation is a layer adjustment operation, the electronic device determines that a pixel of the activated layer having content in the current frame is a dirty pixel; or In the case where the first operation is color filling, the electronic device determines that the pixels in the newly added color filling area of ​​the activated layer in the current frame are dirty pixels.

8. The method according to claim 7, characterized in that After the electronic device determines that the plurality of pixels in the active layer are dirty pixels, the method further includes: The electronic device marks the dirty pixels in the active layer as a sixth value, and marks the pixels other than the dirty pixels in the active layer as a seventh value, to obtain a marking result; The electronic device stores the marking result.

9. The method according to claim 1, characterized in that In response to the first operation, the electronic device determines that a plurality of pixels in the active layer are dirty pixels, specifically including: In response to the first operation, the electronic device saves a monochrome texture of the active layer in the current frame, where the monochrome texture includes depth values ​​of pixels of the active layer affected by the first operation in the current frame; The electronic device determines that a plurality of pixels in the active layer are dirty pixels based on the monochrome texture.

10. The method according to claim 9, wherein the electronic device comprises a processor (CPU) and a graphics processor (GPU), and in response to the first operation, the electronic device saves the monochrome texture of the activated layer in the current frame, specifically comprising: In response to the first operation, the CPU determines a depth value of a pixel of the active layer on which the first operation is performed in the current frame, and sends a first drawing instruction to the GPU; In response to the first drawing instruction, the GPU draws a monochrome texture of the active layer in the current frame, and saves depth values ​​of pixels affected by the first operation in the current frame of the active layer into the monochrome texture; The electronic device determines, based on the monochrome texture, that a plurality of pixels in the active layer are dirty pixels, specifically comprising: The GPU determines, based on the monochrome texture, that a plurality of pixels in the active layer are dirty pixels.

11. The method according to claim 10, characterized in that In the case where the first operation is a brush drawing operation, the pixels of the activated layer affected by the first operation in the current frame are the pixels where the stable points and unstable points drawn by the user are located, the depth value of the pixel where the stable point is located is the first depth value, and the depth value of the pixel where the unstable point is located is the second depth value; or In the case where the first operation is a transformation operation, the pixel of the active layer in the current frame on which the first operation is performed is the pixel where the transformation content is located in the current frame of the active layer, and the depth value of the pixel where the transformation content is located is the third depth value; or, In a case where the first operation is a filter operation, the pixels of the activated layer in the current frame that are affected by the first operation are pixels of the activated layer that have content in the current frame, and the depth value of the pixels that have content is a fourth depth value; or, In a case where the first operation is a layer adjustment operation, the pixels of the activated layer in the current frame that are affected by the first operation are pixels of the activated layer that have content in the current frame, and the depth value of the pixels that have content is the fifth depth value; or, When the first operation is a color quick fill operation, the pixels of the activated layer affected by the first operation in the current frame are the pixels of the newly added color fill area of ​​the activated layer in the current frame, and the depth value of the pixels of the newly added color fill area is the sixth depth value.

12. The method according to claim 11, characterized in that A format of the monochrome texture stored in the GPU is different from a format of a depth map stored in the GPU.

13. A layer synthesis method, characterized in that: The method comprises: The electronic device displays a first user interface of a target application, wherein the first user interface displays a first image, the first image including an active layer, a background layer, and a foreground layer; wherein the target application is a painting application; The electronic device detects a first operation on the activation layer; In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame; The electronic device determines, based on the depth map, that a plurality of pixels in the active layer are dirty pixels, where the dirty pixels include a first pixel; The electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a second image, wherein the position of the second pixel in the background layer is the same as the position of the first pixel in the activation layer, and the position of the third pixel in the foreground layer is the same as the position of the first pixel in the activation layer. The second image includes the background layer of the first image and the foreground layer of the first image, and the content of the activation layer in the second image is different from the content of the activation layer in the first image.

14. The method according to claim 13, wherein: The electronic device determines, based on the depth map, that a plurality of pixels in the active layer are dirty pixels, specifically comprising: When the first operation is a brush drawing operation and the depth value of the first pixel in the depth map is greater than or equal to a first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value of a depth test in a brush drawing scenario; or When the first operation is a transformation operation and the depth value of the first pixel in the depth map is greater than or equal to a second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is a threshold value for depth testing in a transformation scenario; or When the first operation is a filter operation and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is a threshold value for depth testing in a filter scenario; or When the first operation is a layer adjustment operation and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is a threshold value for depth testing in a layer adjustment scenario; or When the first operation is a color filling operation, and when the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is a threshold value for depth testing in a layer adjustment scenario.

15. The method according to claim 13, characterized in that The electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame, specifically including: In response to the first operation, the CPU determines a depth value of a pixel of the activated layer in a current frame and stores the depth value in a first memory of the CPU; The CPU instructs the GPU to copy the depth value in the first memory to the second memory of the GPU; The GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map; The electronic device determines, based on the depth map, that a plurality of pixels in the active layer are dirty pixels, specifically comprising: The GPU performs a depth test on the depth map to obtain a depth test result; The GPU determines that multiple pixels in the active layer are dirty pixels based on the depth test result.

16. The method according to any one of claims 13 to 15, characterized in that: The electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a second image, specifically comprising: The electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a first synthesis result, wherein the color values ​​of pixels in the first synthesis result that are located at the same position as the first pixel are updated, and the color values ​​of pixels other than the dirty pixel in the first synthesis result are not updated; The electronic device renders the first synthesis result to obtain a second image.

17. The method according to claim 15, characterized in that The GPU writes the depth value into a texture containing dirty pixel information of the active layer in the current frame to generate a depth map, specifically including: When the first operation is a brush drawing operation, the GPU writes, in a texture containing dirty pixel information of the active layer in the current frame, depth values ​​of pixels at which stable points of the active layer are located in the current frame as first depth values, and writes depth values ​​of pixels at which unstable points of the active layer are located in the current frame as second depth values, to generate the depth map, wherein the second depth value is greater than the first depth value; or When the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformed content of the active layer in the current frame is located in the texture containing the dirty pixel information of the active layer in the current frame as the third depth value to obtain the depth map; or When the first operation is a filter operation, the GPU writes the depth values ​​of pixels of the active layer having content in the current frame as fourth depth values ​​in a texture containing dirty pixel information of the active layer in the current frame to obtain the depth map; or When the first operation is layer adjustment, the GPU writes the depth values ​​of pixels of the active layer having content in the current frame as fifth depth values ​​in a texture containing dirty pixel information of the active layer in the current frame to obtain the depth map; or When the first operation is color filling, the GPU writes the depth value of the newly added filled pixels of the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map.

18. An electronic device, characterized in that: include: A processor and a memory, the memory being coupled to the processor, the memory being used to store computer program code, the computer program code comprising computer instructions, and when the processor reads the computer instructions from the memory, the electronic device executes the method according to any one of claims 1 to 17.

19. A chip system, applied to electronic equipment, comprising one or more processors, characterized in that: The processor is configured to call computer instructions so as to execute the method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a computer program, and when the computer program is run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 17.

21. A computer program product, characterized in that When the computer program product is run on a computer, the electronic device is enabled to perform the method according to any one of claims 1 to 17.

Images