Layer synthesis method and related device

Abstract

The invention provides a layer synthesis method and a related device, in the method, when a user draws a drawn image containing a plurality of layers in an electronic device, the electronic device only synthesizes the layers operated by the user, namely dirty point pixels in the activated layers, with pixels in the same positions as the dirty point pixels in other layers, and then the image is synthesized. And obtaining and displaying a synthesis result. Therefore, the calculation amount of layer synthesis can be reduced, and the time consumption of layer synthesis can be shortened, so that the performance of real-time layer synthesis can be improved, and the user experience can be improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a layer synthesis method and related devices. Background Art

[0002] Currently, users can draw in electronic devices. During the drawing process, users can split the work to be drawn into multiple layers. After the user draws the corresponding content in each layer, the electronic device can synthesize the multiple layers and display the final synthesis result on the display screen.

[0003] When electronic devices perform layer synthesis, the more layers to be synthesized, the greater the amount of calculation required for layer synthesis. When the amount of calculation required for the layer synthesis process is too large, the layer synthesis will take a long time, and the delay in the electronic device drawing and displaying the synthesized layer will increase. This will cause the real-time effect of the user's work to be jittery, affecting the user experience. Summary of the invention

[0004] The present application provides a layer synthesis method and related devices, through which the electronic device can reduce the amount of calculation for layer synthesis when performing layer synthesis, and can improve the layer synthesis performance. The electronic device can present the user's drawn works in real time faster, thereby improving the user experience.

[0005] In a first aspect, the present application provides a layer synthesis method, which may include: an electronic device displays a first user interface of a target application, a first image is displayed in the first user interface, and the first image includes an activation layer and a background layer; the electronic device detects a first operation on the activation layer; in response to the first operation, the electronic device determines that multiple pixels in the activation layer are dirty pixels, and the dirty pixels include a first pixel; the electronic device synthesizes the first pixel with a second pixel in the background layer to obtain a second image, the position of the second pixel in the background layer is the same as the position of the first pixel in the activation layer, the second image includes the background layer of the first image, and the content of the activation layer in the second image is different from the content of the activation layer in the first image.

[0006] The target applications may include painting applications and image processing applications.

[0007] By using the layer synthesis method provided in the first aspect, when performing layer synthesis, the electronic device only synthesizes the dirty pixels in the layer. In this way, the amount of calculation for layer synthesis can be reduced, and the layer synthesis performance can be improved. The electronic device can present the user's drawn works in real time faster, thereby improving the user experience.

[0008] In combination with the first aspect, in a possible implementation method, in response to a first operation, the electronic device determines that multiple pixels in the activated layer are dirty pixels, which may specifically include: in response to the first operation, the electronic device saves the depth map of the activated layer in the current frame; based on the depth map, the electronic device determines that multiple pixels in the activated layer are dirty pixels.

[0009] An operation of the user in the activated layer generally involves changes between multiple frames. The current frame may refer to the image frame currently rendered by the electronic device. The current frame contains not only the content contained in the activated layer, but also the content contained in the background layer. The electronic device may render each frame of the image according to the frame rate. The electronic device may save the depth map corresponding to the frame of the image being rendered in the current frame.

[0010] In this way, the electronic device can determine the dirty pixels in the activated layer based on the depth map.

[0011] In combination with the first aspect, in a possible implementation, the electronic device determines that multiple pixels in the activated layer are dirty pixels based on the depth map, which may specifically include: when the first operation is a brush drawing operation, and the depth value of the first pixel in the depth map is greater than or equal to the first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value of the depth test in the brush drawing scenario; or, when the first operation is a transformation operation, and the depth value of the first pixel in the depth map is greater than or equal to the second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is the threshold value of the depth test in the transformation scenario; or, when the first operation is a filter operation, and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is the threshold value of the depth test in the filter scenario; or, when the first operation is a layer adjustment operation, and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is the threshold value of the depth test in the layer adjustment scenario; or, when the first operation is a color fill operation, and the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is the threshold value of the depth test in the layer adjustment scenario.

[0012] In this way, the electronic device can determine the dirty pixels in the activation layer under different scenarios.

[0013] In conjunction with the first aspect, in a possible implementation, the first value, the second value, and the fifth value may be values ​​that change as the current frame changes. For example, when the current frame is the first frame, the first value may be 1, the second value may be 1, and the fifth value may also be 1. When the current frame is the second frame, the first value may be 2, the second value may also be 2, and the fifth value may also be 2.

[0014] In combination with the first aspect, in a possible implementation, the third value and the fourth value may also be fixed values. That is, no matter whether the current frame is the N-1th frame or the Nth frame, the third value and the fourth value will not change. For example, the third value may be 1, and the fourth value may be 1.

[0015] In combination with the first aspect, in a possible implementation, the electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame. Specifically, the following may be included: in response to the first operation, the CPU determines the depth value of the pixel of the activated layer in the current frame and stores it in the first memory of the CPU; the CPU instructs the GPU to copy the depth value in the first memory to the second memory of the GPU; the GPU writes the depth value to the texture containing the dirty pixel information of the activated layer in the current frame to generate a depth map. In this way, the CPU in the electronic device can calculate the depth value of the pixel of the activated layer in the current frame, and can store the depth value in the depth map through the GPU.

[0016] Optionally, in a possible implementation, the GPU may write the depth value into a texture containing dirty pixel information of the activated layer in the current frame through a vertex shader to generate a depth map.

[0017] In conjunction with the first aspect, in a possible implementation, the electronic device determines that multiple pixels in the activated layer are dirty pixels based on the depth map, which may specifically include: the GPU performs a depth test on the depth map to obtain a depth test result; the GPU determines that multiple pixels in the activated layer are dirty pixels based on the depth test result. In this way, the electronic device can determine the dirty pixels in the activated layer through the depth test.

[0018] In combination with the first aspect, in a possible implementation, the electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a second image, which may specifically include: the electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a first synthesis result, the color values ​​of the pixels in the first synthesis result that are at the same position as the first pixel have been updated, and the color values ​​of the pixels other than the dirty pixels in the first synthesis result have not been updated; the electronic device renders the first synthesis result to obtain the second image.

[0019] Optionally, the color values ​​of pixels other than the dirty pixel in the first synthesis result not being updated may also refer to updating the same color value.

[0020] In this way, the electronic device obtains the second image by synthesizing the activated layer and the background layer.

[0021] In combination with the first aspect, in a possible implementation method, the GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map, which may specifically include: when the first operation is a brush drawing operation, the GPU writes the depth value of the pixel where the stable point of the activated layer in the current frame is located as a first depth value, and writes the depth value of the pixel where the unstable point of the activated layer in the current frame is located as a second depth value in the texture containing dirty pixel information of the activated layer in the current frame to generate a depth map, and the second depth value is greater than the first depth value; or, when the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformed content of the activated layer in the current frame is located in the texture containing dirty pixel information of the activated layer in the current frame The GPU writes the depth value of the pixel with content in the activated layer in the current frame as the third depth value to obtain a depth map; or, in the case where the first operation is a filter operation, the GPU writes the depth value of the pixel with content in the activated layer in the current frame as the fourth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is a layer adjustment, the GPU writes the depth value of the pixel with content in the activated layer in the current frame as the fifth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is color filling, the GPU writes the depth value of the newly added fill pixel in the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map.

[0022] In this way, the electronic device can obtain the depth map corresponding to the activated layer in different scenarios.

[0023] In combination with the first aspect, in a possible implementation, in response to the first operation, the electronic device determines that multiple pixels in the activated layer are dirty pixels, which may specifically include: when the first operation is a brush drawing operation, the electronic device determines that the pixels at which stable points and unstable points drawn by the user of the activated layer in the current frame are located, and the pixels at which unstable points drawn by the user of the activated layer in the previous frame are located are dirty pixels; or, when the first operation is a transformation operation, the electronic device determines that the pixels at which the transformed content of the activated layer is located in the current frame and the pixels at which the transformed content of the activated layer is located in the previous frame are dirty pixels; or, when the first operation is a filter operation, the electronic device determines that the pixels at which the activated layer has content in the current frame are dirty pixels; or, when the first operation is a layer adjustment operation, the electronic device determines that the pixels at which the activated layer has content in the current frame are dirty pixels; or, when the first operation is a color fill operation, the electronic device determines that the pixels at which the newly added fill area of ​​the activated layer is located in the current frame are dirty pixels.

[0024] In this way, in different scenarios, the electronic device can determine the dirty pixels in the activated layer.

[0025] In combination with the first aspect, in a possible implementation, after the electronic device determines that multiple pixels in the activated layer are dirty pixels, the method may also include: the electronic device marks the dirty pixels in the activated layer as a sixth value, and marks the pixels other than the dirty pixels in the activated layer as a seventh value, to obtain a marking result; and the electronic device saves the marking result.

[0026] The sixth value is different from the seventh value. For example, the sixth value may be 1 and the seventh value may be 0. Alternatively, the sixth value may be 0 and the seventh value may be 1.

[0027] In the labeling result, the label value corresponding to each pixel needs to be stored using 1 bit.

[0028] In this way, the electronic device can mark the dirty pixels in the activation layer through the marking result.

[0029] In combination with the first aspect, in a possible implementation method, in response to a first operation, the electronic device determines that multiple pixels in an activated layer are dirty pixels, which may specifically include: in response to the first operation, the electronic device saves the monochrome texture of the activated layer in the current frame, the monochrome texture containing the depth values ​​of the pixels acted upon by the first operation in the current frame of the activated layer; based on the monochrome texture, the electronic device determines that multiple pixels in the activated layer are dirty pixels.

[0030] In this way, when the depth map in the electronic device cannot be used in layer synthesis, the electronic device can also determine the dirty pixels in the activated layer through the monochrome texture.

[0031] In combination with the first aspect, in a possible implementation, the electronic device may include a processor CPU and an image processor GPU. In response to the first operation, the electronic device saves the monochrome texture of the activated layer in the current frame. Specifically, it may include: in response to the first operation, the CPU determines the depth value of the pixel of the activated layer affected by the first operation in the current frame, and sends a first drawing instruction to the GPU; in response to the first drawing instruction, the GPU draws the monochrome texture of the activated layer in the current frame, and saves the depth value of the pixel affected by the first operation in the current frame of the activated layer to the monochrome texture.

[0032] In this way, the electronic device can calculate the depth value of the pixel on which the first operation of the activated layer in the current frame is applied through the CPU, and then save the calculated depth value in the monochrome texture through the GPU.

[0033] In combination with the first aspect, in a possible implementation, the electronic device determines that multiple pixels in the activated layer are dirty pixels based on the monochrome texture, which may specifically include: the GPU determines that multiple pixels in the activated layer are dirty pixels based on the monochrome texture.

[0034] In a possible implementation, the GPU may determine, through a fragment shader, that a plurality of pixels in an activated layer are dirty pixels.

[0035] In this way, the electronic device can determine the dirty pixels in the activated layer through the GPU.

[0036] In combination with the first aspect, in a possible implementation, when the first operation is a brush drawing operation, the pixels of the activated layer acted upon by the first operation in the current frame are the pixels where the stable points and unstable points drawn by the user are located, the depth value of the pixel where the stable point is located is the first depth value, and the depth value of the pixel where the unstable point is located is the second depth value; or, when the first operation is a transformation operation, the pixels of the activated layer acted upon by the first operation in the current frame are the pixels where the transformed content of the activated layer is located in the current frame, and the depth value of the pixel where the transformed content is located is the third depth value; or, when the first operation is a filter operation, the activated layer The pixels acted upon by the first operation on the layer in the current frame are the pixels where content exists in the activated layer in the current frame, and the depth value of the pixels where content exists is the fourth depth value; or, when the first operation is a layer adjustment operation, the pixels acted upon by the first operation on the activated layer in the current frame are the pixels where content exists in the activated layer in the current frame, and the depth value of the pixels where content exists is the fifth depth value; or, when the first operation is a color fill operation, the pixels acted upon by the first operation on the activated layer in the current frame are the pixels where a newly added color fill area is located in the activated layer in the current frame, and the depth value of the pixels where the newly added color fill area is located is the sixth depth value.

[0037] In this way, the electronic device can determine the depth value of the pixel on which the first operation is performed in different scenarios.

[0038] In combination with the first aspect, in a possible implementation manner, a format of a monochrome texture stored in the GPU is different from a format of a depth map stored in the GPU.

[0039] In a second aspect, a layer synthesis method is provided, which may include: an electronic device displays a first user interface of a target application, wherein a first image is displayed in the first user interface, and the first image includes an activation layer, a background layer and a foreground layer; the electronic device detects a first operation on the activation layer; in response to the first operation, the electronic device saves a depth map of the activation layer in the current frame; based on the depth map, the electronic device determines that multiple pixels in the activation layer are dirty pixels, and the dirty pixels include a first pixel; the electronic device synthesizes the first pixel with a second pixel in the background layer and a third pixel in the foreground layer to obtain a second image, wherein a position of the second pixel in the background layer is the same as a position of the first pixel in the activation layer, and a position of the third pixel in the foreground layer is the same as a position of the first pixel in the activation layer, and the second image includes the background layer of the first image and the foreground layer of the first image, and content of the activation layer in the second image is different from content of the activation layer in the first image.

[0040] The target applications may include painting applications and image processing applications.

[0041] By using the layer synthesis method provided in the second aspect, when performing layer synthesis, the electronic device only synthesizes the dirty pixels in the layer. In this way, the amount of calculation for layer synthesis can be reduced, and the layer synthesis performance can be improved. The electronic device can present the user's drawn works in real time faster, thereby improving the user experience.

[0042] In combination with the second aspect, in a possible implementation, the electronic device determines that multiple pixels in the activated layer are dirty pixels based on the depth map, which may specifically include: when the first operation is a brush drawing operation, and the depth value of the first pixel in the depth map is greater than or equal to the first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value of the depth test in the brush drawing scenario; or, when the first operation is a transformation operation, and the depth value of the first pixel in the depth map is greater than or equal to the second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is the threshold value of the depth test in the transformation scenario; or, when the first operation is a filter operation, and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is the threshold value of the depth test in the filter scenario; or, when the first operation is a layer adjustment operation, and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is the threshold value of the depth test in the layer adjustment scenario; or, when the first operation is a color fill operation, and the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is the threshold value of the depth test in the layer adjustment scenario.

[0043] In this way, the electronic device can determine the dirty pixels in the activation layer under different scenarios.

[0044] In conjunction with the second aspect, in a possible implementation, the first value, the second value, and the fifth value may be values ​​that change as the current frame changes. For example, when the current frame is the first frame, the first value may be 1, the second value may be 1, and the fifth value may also be 1. When the current frame is the second frame, the first value may be 2, the second value may also be 2, and the fifth value may also be 2.

[0045] In conjunction with the second aspect, in a possible implementation, the third value and the fourth value may also be fixed values. That is, no matter whether the current frame is the N-1th frame or the Nth frame, the third value and the fourth value will not change. For example, the third value may be 1, and the fourth value may be 1.

[0046] In conjunction with the second aspect, in a possible implementation, the electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame. Specifically, the following may be included: in response to the first operation, the CPU determines the depth value of the pixel of the activated layer in the current frame and stores it in the first memory of the CPU; the CPU instructs the GPU to copy the depth value in the first memory to the second memory of the GPU; the GPU writes the depth value to the texture containing the dirty pixel information of the activated layer in the current frame to generate a depth map. In this way, the CPU in the electronic device can calculate the depth value of the pixel of the activated layer in the current frame, and can store the depth value in the depth map through the GPU.

[0047] Optionally, in a possible implementation, the GPU may write the depth value into a texture containing dirty pixel information of the activated layer in the current frame through a vertex shader to generate a depth map.

[0048] In conjunction with the second aspect, in a possible implementation, the electronic device determines that multiple pixels in the activated layer are dirty pixels based on the depth map, which may specifically include: the GPU performs a depth test on the depth map to obtain a depth test result; the GPU determines that multiple pixels in the activated layer are dirty pixels based on the depth test result. In this way, the electronic device can determine the dirty pixels in the activated layer through the depth test.

[0049] In combination with the second aspect, in a possible implementation, the electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a second image, which may specifically include: the electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a first synthesis result, the color values ​​of the pixels in the first synthesis result that are at the same position as the first pixel have been updated, and the color values ​​of the pixels other than the dirty pixels in the first synthesis result have not been updated; the electronic device renders the first synthesis result to obtain the second image.

[0050] Optionally, the color values ​​of pixels other than the dirty pixel in the first synthesis result not being updated may also refer to updating the same color value.

[0051] In this way, the electronic device obtains the second image by synthesizing the activated layer and the background layer.

[0052] In combination with the second aspect, in a possible implementation method, the GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map, which may specifically include: when the first operation is a brush drawing operation, the GPU writes the depth value of the pixel where the stable point of the activated layer in the current frame is located as a first depth value, and writes the depth value of the pixel where the unstable point of the activated layer in the current frame is located as a second depth value in the texture containing dirty pixel information of the activated layer in the current frame to generate a depth map, and the second depth value is greater than the first depth value; or, when the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformed content of the activated layer in the current frame is located in the texture containing dirty pixel information of the activated layer in the current frame The GPU writes the depth value of the pixel with content in the activated layer in the current frame as the third depth value to obtain a depth map; or, in the case where the first operation is a filter operation, the GPU writes the depth value of the pixel with content in the activated layer in the current frame as the fourth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is a layer adjustment, the GPU writes the depth value of the pixel with content in the activated layer in the current frame as the fifth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map; or, in the case where the first operation is color filling, the GPU writes the depth value of the newly added fill pixel in the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain a depth map.

[0053] In this way, the electronic device can obtain the depth map corresponding to the activated layer in different scenarios.

[0054] According to a third aspect, an electronic device is provided, which may include a display, one or more processors and one or more memories; wherein the display, the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method involved in any possible implementation of the first aspect.

[0055] In a fourth aspect, an electronic device is provided. The electronic device may include one or more functional modules, and the one or more functional modules are used for the method involved in any possible implementation manner of the first aspect.

[0056] In a fifth aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method involved in any possible implementation manner in the first aspect.

[0057] In a sixth aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute a method as involved in any possible implementation of the first aspect.

[0058] In a seventh aspect, a computer program product is provided, which includes a computer program / instructions. When the computer program / instructions are executed on an electronic device, the electronic device executes the method involved in any possible implementation of the first aspect.

[0059] It can be understood that the electronic device provided in the third aspect, the electronic device provided in the fourth aspect, the chip system provided in the fifth aspect, the computer-readable storage medium provided in the sixth aspect, and the computer program product provided in the seventh aspect are all used to execute the method provided in the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of a drawing interface provided in an embodiment of the present application;

[0061] Figure 2 It is a schematic diagram of layer synthesis provided in an embodiment of the present application;

[0062] Figure 3 This is another schematic diagram of layer synthesis provided in an embodiment of the present application;

[0063] Figure 4 This is another schematic diagram of layer synthesis provided in an embodiment of the present application;

[0064] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0065] Figure 6 is a schematic diagram of the full hardware architecture of an electronic device provided in an embodiment of the present application;

[0066] Fig. 7A This is a schematic diagram of the N-1th image frame of the activated layer in the brush drawing scenario provided by an embodiment of the present application;

[0067] Figure 7B This is a schematic diagram of an Nth image frame of an activated layer in a brush drawing scenario provided by an embodiment of the present application;

[0068] Figure 7C It is a schematic diagram of dirty pixel of the Nth image frame of the activated layer in the brush drawing scenario provided by an embodiment of the present application;

[0069] Fig. 8A This is a schematic diagram of the N-1th image frame of the activated layer in the transformation scene provided by an embodiment of the present application;

[0070] Figure 8B This is a schematic diagram of an Nth image frame of an activated layer in a transformation scene provided by an embodiment of the present application;

[0071] Figure 8C This is a schematic diagram of dirty pixel of the Nth image frame of the activated layer in the transformation scene provided by an embodiment of the present application;

[0072] Fig. 9A This is a schematic diagram of the N-1th image frame of the activated layer in the filter scene provided in an embodiment of the present application;

[0073] Fig. 9B It is a schematic diagram of the Nth image frame of the activated layer in the filter scene provided in an embodiment of the present application;

[0074] Fig. 9C It is a schematic diagram of dirty pixel of the Nth image frame of the activated layer in the filter scene provided by the embodiment of the present application;

[0075] Fig. 10A This is a schematic diagram of the N-1th image frame of the activated layer in the layer adjustment scenario provided by an embodiment of the present application;

[0076] Fig. 10B This is a schematic diagram of an Nth image frame of an activated layer in a layer adjustment scenario provided by an embodiment of the present application;

[0077] Fig. 10C It is a schematic diagram of dirty pixel of the Nth image frame of the activated layer in the layer adjustment scenario provided by an embodiment of the present application;

[0078] Fig.11AThis is a schematic diagram of the N-1th image frame of the activated layer in the color quick fill scenario provided by an embodiment of the present application;

[0079] Fig. 11B This is a schematic diagram of an Nth image frame of an activated layer in a color quick fill scenario provided by an embodiment of the present application;

[0080] Fig. 11C This is a schematic diagram of dirty pixel of the Nth image frame of the activated layer in the color quick fill scenario provided by an embodiment of the present application;

[0081] Fig.12 It is a schematic diagram of a process flow of a layer synthesis method provided in an embodiment of the present application;

[0082] Fig.13A It is the N-1th frame drawing handwriting, the Nth frame drawing handwriting, the N-1th frame drawing handwriting, the depth map corresponding to the N-1th frame, and the depth map corresponding to the Nth frame in the brush drawing scene activation layer provided by the embodiment of the present application;

[0083] Fig. 13B It is the N-1th frame transformation content, the Nth frame transformation content, the N-1th frame corresponding to the depth map and the Nth frame corresponding to the depth map in the transformation scene activation layer provided by the embodiment of the present application;

[0084] Fig. 13C It is the display content of the N-1th frame before the filter operation, the display content of the Nth frame after the filter operation, and the depth map corresponding to the N-1th frame and the depth map corresponding to the Nth frame in the filter scene activation layer provided by the embodiment of the present application;

[0085] Fig.13D It is the display content of the N-1th frame before the layer adjustment operation, the display content of the Nth frame after the layer adjustment operation, and the depth map corresponding to the N-1th frame and the depth map corresponding to the Nth frame in the layer adjustment scene activation layer provided by the embodiment of the present application;

[0086] Fig.13E The color filling area of ​​the N-1th frame, the color filling area of ​​the Nth frame, the depth map corresponding to the N-1th frame, and the depth map corresponding to the Nth frame in the color quick fill scene activation layer provided by the embodiment of the present application;

[0087] Fig.14 It is a schematic diagram of a layer synthesis implementation step provided in an embodiment of the present application;

[0088] Fig.15 It is a schematic diagram of a process flow of a layer synthesis method provided in an embodiment of the present application;

[0089] Fig.16 It is a schematic diagram of a process flow of a layer synthesis method provided in an embodiment of the present application;

[0090] Fig.17 It is a structural schematic diagram of a device 1700 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0091] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0092] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear contrary indication in the context. The terms "first" and "second" are only used for descriptive purposes, and cannot be understood as implying or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features. "First" and "second" etc. are used to distinguish different objects, rather than for describing a specific order of objects. For example, the first object and the second object are used to distinguish different objects, rather than for describing a specific order of objects.

[0093] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0094] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or related scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0095] The term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0096] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0097] In order to better understand the technical solution provided by the present application, before describing the technical solution of the present application, firstly, in conjunction with the accompanying drawings, an electronic device 100 with drawing and image processing functions to which the present application is applicable is described. For example, the electronic device 100 has drawing software or image processing software. In the embodiment of the present application, the electronic device 100 may include but is not limited to: mobile phones, tablet computers, smart watches and other devices with drawing and image processing functions. The embodiment of the present application does not limit the specific form and type of the electronic device 100.

[0098] First, the technical terms involved in the embodiments of the present application are introduced:

[0099] 1) Pixel: The basic unit of image display, which cannot be divided any further. Each pixel is a small grid of a single color.

[0100] 2) Texture: A data object that contains image information. For example, this data object can be an array of pixels, where the data of each pixel represents the color of the pixel, and the array of pixels can represent the entire image. For example, a texture can be a memory that stores a layer or image.

[0101] 3) Layer: In image processing applications / painting applications, a layer is a component of a work (or described as an image). A work can be composed of multiple layers, each containing different visual elements. Layers can be stacked on top of each other and blended into the final effect. Each layer can be modified independently without affecting the content of other layers.

[0102] 4) Blending: Take multiple layers as input and output a result image.

[0103] 5) Synthesize: Take multiple layers as input and output a result image.

[0104] 6) Active layer: The layer that the user is currently modifying.

[0105] 7) Background layer: The layer below the active layer in the layer list.

[0106] 8) Foreground layer: The layer above the active layer in the layer list.

[0107] 9) Painting latency: The delay when the pen tip draws on the screen.

[0108] 10) Local blending: Blend a portion of the layer texture, while the content of the remaining area remains unchanged.

[0109] 11) Vertex shader: A shader that performs a series of operations on vertices in the rendering pipeline.

[0110] 12) Fragment shader: The shader in the rendering pipeline is responsible for calculating and outputting colors.

[0111] 13) Dirty region: An area whose content has changed during the rendering process and needs to be redrawn.

[0112] 14) Dirty pixel: A pixel whose content has changed during the rendering process and needs to be redrawn.

[0113] Currently, the painting application or image processing application in the electronic device 100 may have the ability to mix (or synthesize) multiple layers in real time. Taking the painting application as an example, in a painting scene, the user can split the work to be drawn into multiple layers in the painting application of the electronic device 100. The user can draw and adjust the corresponding content on each layer. The electronic device 100 can synthesize the content drawn by the user in real time and display the synthesized content. For example, the electronic device 100 can display the following: Figure 1 The drawing interface 101 shown. Figure 1As shown, the drawing interface 101 may include a menu bar 102, and the menu bar 102 may be used to display the layer information corresponding to the current drawing image. The menu bar 102 may include setting items for setting each layer separately, for example, setting item 103, setting item 104, setting item 105, setting item 106 and setting item 107. Setting item 103 may be used to set the background layer and display the background layer information. Setting item 104 may be used to set layer 1 and display layer 1 information. Setting item 105 may be used to set layer 2 and display layer 2 information. Setting item 106 may be used to set layer 3 and display layer 3 information. Setting item 107 may be used to set layer 4 and display layer 4 information. Layer 2 may be the hat of the cartoon duck drawn by the user, layer 3 may be the body of the cartoon duck drawn by the user, and layer 4 may be the facial features of the cartoon duck drawn by the user. Among them, layer 4 is the layer that the user is drawing. The user draws the facial features of a cartoon duck in layer 4, and the electronic device 100 can synthesize the content being drawn by the user with other layers in real time, and finally obtain a drawing image 109 displayed in the drawing interface 101. The drawing image 109 can be a cartoon duck.

[0114] Optionally, the menu bar 102 may include a control 102a and a control 102b, wherein the control 102a may be used to add a layer, and the control 102b may be used to save an image drawn by a user.

[0115] Alternatively, if Figure 1 As shown, the drawing interface 101 may further include a toolbar 108. The toolbar 108 may include a control 1081 for retracting drawing content, a control 1082 for enabling brush drawing, a control 1083 for enabling transform drawing, a control 1084 for enabling filter operation, a control 1085 for enabling layer adjustment, a control 1086 for enabling color quick fill, and a control 1087 for displaying more drawing tools.

[0116] Figure 1 The drawing interface 101 shown is only an example. In the embodiment of the present application, the drawing interface 101 may include more or fewer interface elements. The embodiment of the present application does not limit the interface elements included in the drawing interface 101.

[0117] When the electronic device 100 performs real-time synthesis of multiple layers of a user-drawn image, the electronic device 100 may synthesize the multiple layers in order from bottom to top according to the arrangement order of the layers. Figure 2 As shown, Figure 1Taking the five layers (background layer, layer 1, layer 2, layer 3 and layer 4) included in the drawn image 109 shown in FIG as an example, layer 1 can be exemplified as an activated layer. The electronic device 100 can first mix the background layer and layer 1 to obtain an intermediate result 1. Then, the electronic device 100 mixes the intermediate result 1 and layer 2 to obtain an intermediate result 2. Next, the electronic device 100 mixes layer 3 and intermediate result 2 to obtain an intermediate result 3. Finally, the electronic device 100 mixes layer 4 and intermediate result 3 to obtain a final synthesis result.

[0118] The amount of calculation for layer blending is positively correlated with the number of layers. The more layers there are, the greater the amount of calculation for layer blending. When the user draws more layers, the amount of calculation for layer blending by the electronic device 100 will be relatively large. When the amount of calculation for layer blending is large, it will cause the layer blending to take a long time, the drawing delay will be longer, and then the real-time effect of the painting creation will be jammed, affecting the user's painting creation.

[0119] In a possible implementation, in order to reduce drawing delay and improve drawing smoothness, the electronic device 100 may determine dirty areas in a layer and locally synthesize the dirty areas of multiple layers. Figure 3 As shown, the electronic device 100 can calculate the area (or bounding box) affected by the user operation in the active layer 301 based on the user's operation in the active layer, and set it as a dirty area (such as Figure 3 ). When performing layer synthesis, the electronic device 100 may synthesize the pixels in the dirty areas of multiple layers, and keep the pixels outside the dirty areas unchanged. For example, the electronic device 100 may synthesize the pixels in the bounding box 3051 in layer 305, the pixels in the bounding box 3041 in layer 304, the pixels in the dirty area bounding box 3011 in the activated layer 301, the pixels in the bounding box 3031 in layer 303, and the pixels in the bounding box 3021 in layer 302. Among them, the position of the bounding box 3051 in layer 305 is the same as the position of the dirty area bounding box 3011 in the activated layer 301. The position of the bounding box 3041 in layer 304 is the same as the position of the dirty area bounding box 3011 in the activated layer 301. The position of the bounding box 3031 in layer 303 is the same as the position of the dirty area bounding box 3011 in the activated layer 301. The position of the bounding box 3021 in the layer 302 is the same as the position of the dirty area bounding box 3011 in the active layer 301 .

[0120] In this way, the electronic device 100 only synthesizes the pixels within the dirty area bounding box, thereby improving the real-time synthesis performance of the layer by reducing the number of pixels involved in the synthesis.

[0121] However, when the dirty area bounding box contains pixels that do not need to be re-synthesized, the number of pixels in the dirty area bounding box that actually need to be blended is far less than the number of dirty point pixels. When the electronic device 100 only synthesizes the pixels in the dirty area bounding box, the amount of calculation is still large. In this way, when the electronic device 100 performs layer synthesis, it is still possible that the layer blending takes a long time, the drawing delay becomes longer, and the real-time effect presentation of the painting creation is jammed, affecting the user experience.

[0122] In another possible implementation, the electronic device 100 may use a background layer pre-synthesis method to perform layer synthesis. When a user performs a drawing operation in the electronic device 100, he is currently only operating one operation, or several layers. The layer that the user is currently operating may be called an active layer. When the user is drawing in the active layer, the layer below the active layer, that is, the content in the background layer will not change. When the electronic device 100 is based on the background layer pre-synthesis, the electronic device 100 may pre-synthesize the background layer in advance and save the result in a temporary texture. When the electronic device 100 performs layer synthesis, it only needs to replace all background layers with temporary textures for mixing with other layers. For example, as Figure 4 As shown, when the user draws an image including background layer 1, background layer 2, background layer 3, activation layer, layer 5 and layer 6 in the electronic device 100, the electronic device 100 can pre-synthesize the background layer 1, background layer 2 and background layer 3 in advance to obtain a temporary texture. Then, when the user draws in the activation layer, the electronic device 100 can synthesize the temporary texture with the activation layer, layer 5 and layer 6 in real time to obtain the final result. In this way, when the electronic device 100 performs real-time synthesis, the number of layers involved in the synthesis can be reduced, thereby reducing the amount of calculation and improving the real-time synthesis performance of the layers.

[0123] However, when there are many layers above the active layer, the electronic device 100 uses the background pre-synthesis method to perform layer synthesis, which can reduce the amount of calculation for the background layer to participate in the synthesis. However, synthesizing many layers above the active layer still requires a lot of calculation. In this way, when the electronic device 100 performs layer synthesis, it is still possible that the layer blending takes a long time and the drawing delay becomes longer, which in turn causes the real-time effect of the painting creation to be stuck, affecting the user experience.

[0124] In order to reduce the amount of calculation for layer synthesis, thereby reducing the time required for layer synthesis and improving the performance of real-time layer synthesis, an embodiment of the present application provides a layer synthesis method. In the layer synthesis method provided in the embodiment of the present application, when the electronic device 100 receives a drawing operation of a user in an activated layer, the electronic device 100 determines dirty pixels, synthesizes the dirty pixels in multiple layers, obtains a synthesis result, and displays the synthesis result. In this way, the electronic device 100 only performs local synthesis on the dirty pixels, which can reduce the number of pixels involved in layer synthesis, thereby improving the performance of layer synthesis.

[0125] An exemplary electronic device 100 provided in an embodiment of the present application is introduced below.

[0126] Figure 5 It is a schematic diagram of the structure of the electronic device 100 provided in an embodiment of the present application.

[0127] The embodiment is described in detail below by taking the electronic device 100 as an example. It should be understood that the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0128] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194 and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0129] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0130] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0131] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0132] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0133] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0134] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces.

[0135] The I2S interface can be used for audio communication.

[0136] The PCM interface can also be used for audio communication to sample, quantize and encode analog signals.

[0137] The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.

[0138] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. The processor 110 and the display screen 194 communicate through the DSI interface to realize the display function of the electronic device 100.

[0139] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0140] The SIM interface can be used to communicate with the SIM card interface 195 to implement the function of transmitting data to the SIM card or reading data in the SIM card.

[0141] The USB interface 130 is an interface that complies with USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transmit data between the electronic device 100 and a peripheral device.

[0142] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0143] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0144] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.

[0145] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0146] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0147] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0148] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0149] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0150] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0151] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0152] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0153] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0154] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0155] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0156] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals.

[0157] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0158] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0159] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0160] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, applications required for at least one function (such as face recognition function, fingerprint recognition function, mobile payment function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as face information template data, fingerprint information template, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0161] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0162] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0163] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0164] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0165] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0166] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0167] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0168] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100 .

[0169] The air pressure sensor 180C is used to measure air pressure.

[0170] The magnetic sensor 180D includes a Hall sensor, and the electronic device 100 can detect the opening and closing of the flip leather case by using the magnetic sensor 180D.

[0171] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes).

[0172] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser.

[0173] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode.

[0174] The ambient light sensor 180L is used to sense the brightness of ambient light.

[0175] The fingerprint sensor 180H is used to collect fingerprints.

[0176] The temperature sensor 180J is used to detect the temperature.

[0177] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0178] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0179] Motor 191 can generate vibration prompts.

[0180] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power changes, messages, missed calls, notifications, etc.

[0181] The SIM card interface 195 is used to connect a SIM card.

[0182] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0183] Figure 6A schematic diagram of the software and hardware architecture of a layer synthesis service on an electronic device 100 provided in an embodiment of the present application is shown.

[0184] like Figure 6 As shown, the hardware and software architecture of the electronic device 100 includes a software system and a hardware layer that realizes layer synthesis together with the software structure. The layered architecture of the electronic device 100 divides the software into several layers, each of which has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, the rendering hardware interface and the kernel layer.

[0185] The application layer can include a series of application packages.

[0186] like Figure 6 As shown, the application package may include a target application. In some examples, the target application is an image processing application, a drawing application, or other application with image display and processing functions. In some examples, the target application is configured with functional logic such as a user interface, work management, layer management, and drawing functions. The drawing functions include but are not limited to brush drawing, filters, transformations, layer adjustments, and color quick fill.

[0187] Optionally, the application package may also include applications such as contacts, memos, calls, clipboard, gallery, maps, camera, video, etc.

[0188] It is understandable that the application packages included in the electronic devices of different users may be different, and the embodiment of the present application does not limit the specific application packages included in the electronic device 100.

[0189] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0190] like Figure 6 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0191] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0192] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0193] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0194] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0195] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0196] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0197] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0198] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0199] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0200] The system library may include a drawing engine. The system library may also include multiple other functional modules, such as a surface manager, a media library, a 3D graphics processing library (such as OpenGL ES), a 2D graphics engine (such as SGL), etc.

[0201] In some embodiments, the drawing engine is used to implement the painting or image processing functions of the target application. The drawing engine includes, for example, modules such as layer synthesis, brush drawing, filter operation, transformation operation, and rendering pipeline. These modules can configure the rendering pipeline according to their own logic assembly, and the rendering pipeline is used to generate rendering instructions. For example, the layer synthesis module is used to perform the real-time layer synthesis process. During the real-time layer synthesis process, the layer synthesis module can be used to determine the dirty pixels in the activated layer, and locally synthesize the pixels in multiple layers with the same position as the dirty pixels with the dirty pixels. In addition, before the layer synthesis, the layer synthesis module triggers the rendering pipeline to generate rendering instructions to instruct the electronic device 100 to synthesize the rendered layers.

[0202] The rendering pipeline may also include a vertex shader and a fragment shader. The vertex shader may be used to calculate vertex colors, texture coordinates, etc. in the image to be rendered. The fragment shader may be used to calculate the color of each pixel of the image to be rendered and to color each pixel.

[0203] The surface manager is used to manage the display subsystem and provide the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0204] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0205] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0206] The two-dimensional graphics engine is a drawing engine for 2D drawing. In some examples, the drawing engine includes a three-dimensional graphics processing library and a two-dimensional graphics engine.

[0207] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0208] In some examples, the kernel layer and the system library include an interface for the CPU to call GPU storage and computing functions, such as a rendering hardware interface. In some examples, the rendering hardware interface includes, for example, a direct extension (DX) 12 interface, a metal interface, a vulkan interface, an OpenGL interface, etc.

[0209] like Figure 6 As shown, the hardware layer may include a GPU, which may be used to perform dirty point calculations, image rendering, and the like.

[0210] The following is an illustrative description of the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0211] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

[0212] The following describes how the electronic device 100 determines dirty pixels in the active layer in conjunction with the accompanying drawings.

[0213] Figures 7A-11C Dirty pixels in scenarios such as brush painting, transformation operations, filter operations, layer adjustments, and color quick filling are demonstrated respectively.

[0214] Figure 7A-7C This shows the dirty pixels in the brush-painted scene. Fig. 7A As shown in Figure (a), the drawing interface 700 of the electronic device 100 can display the display content of the N-1th frame. The display content of the N-1th frame may include content contained in multiple layers (for example, background layer, layer 1, layer 2, layer 3 and layer 4). The user can draw with a brush in layer 4 and draw a drawing handwriting 701. The solid line portion in the drawing handwriting 701 can be called a stable point, and the dotted line portion in the drawing handwriting 701 is an unstable point. When each frame of brush drawing is performed, the stable points and unstable points drawn by the user can be displayed in the current frame. A stable point can be a point that will not change if it falls on the canvas, and an unstable point can be a point that falls on the canvas and will be cleared in the next frame. The stable point will still be displayed in the next frame, and the unstable point will disappear in the next frame.

[0215] like Fig. 7A As shown in FIG. 7( a ), the drawing interface 700 may further include a menu bar 702. The menu bar 702 includes setting items corresponding to multiple layers, for example, a setting item 703 corresponding to layer 4, a setting item 704 corresponding to layer 3, a setting item 705 corresponding to layer 2, a setting item 706 corresponding to layer 1, and a setting item 707 corresponding to the background layer. The user may select the setting item 703 to perform brush drawing on layer 4.

[0216] like Fig. 7AAs shown in FIG. 100 (a), the drawing interface 700 may further include a toolbar 108. A control 1082 for enabling brush drawing in the toolbar 108 may be highlighted to prompt the current user that brush drawing is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0217] exist Fig. 7A In the figure (a), the user is operating layer 4, so layer 4 can be called the active layer. The layers below layer 4 can be called background layers.

[0218] Taking an image frame of the electronic device 100 containing 6*6 (* represents a multiplication sign) pixels as an example, the brush drawing content of the user in the N-1th frame can be as follows: Fig. 7A As shown in Figure (b). Fig. 7A In the figure (b), one grid represents one pixel. The brush drawing content of the N-1th frame can be displayed in 6*6 grids, that is, 6*6 pixels.

[0219] To facilitate readers’ understanding, 7A to 7C , FIG. 8A to FIG. 8C In the other figures, the pixels shown are somewhat simplified, but for those skilled in the art, considering the pixel density of mobile phones or tablet electronic devices, these simplifications will not cause obstacles for technicians to understand the context. Fig. 7A , the mobile phone or tablet will use a large number of pixels to present N-1 frames of brushes corresponding to the user's operation to the user.

[0220] The user can continue to paint with the brush in layer 4. For example, Figure 7B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 700. The Nth frame display content includes the stable point in the drawing trace 701 and the drawing trace 708. The drawing trace 708 can be the user's brush drawing content in the Nth frame. For example, the brush drawing content of the Nth frame can be as follows: Figure 7B As shown in Figure (b).

[0221] By comparing the brush drawing content of the Nth frame with the brush drawing content of the N-1th frame, the pixels changed by the user in the Nth frame can be obtained. The pixels in the brush drawing content of the Nth frame that are different from the brush drawing content of the N-1th frame are the pixels changed by the user in the Nth frame. The pixels changed by the user in the Nth frame can be called dirty pixels. Figure 7C The brush drawing content of the N-1 frame shown in Figure (a) is the same as Figure 7CCompared with the brush drawing content of the Nth frame shown in Figure (b) in FIG. Figure 7C The brush in frame (c) depicts dirty pixels. Figure 7C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Figure 7C The brush drawing content of the N-1 frame shown in Figure (a) is Fig. 7A Figure (b) shows the brush drawing content of the N-1th frame. Figure 7C The brush drawing content of the Nth frame shown in Figure (b) is Figure 7B Figure (b) shows the brush drawing content of the Nth frame.

[0222] Figure 8A-8C The dirty pixels in the transformation operation scenario are shown. Fig. 8A As shown in FIG. 8( a ), the drawing interface 800 of the electronic device 100 may display the N-1 frame display content. The N-1 frame display content may include content included in multiple layers (e.g., background layer, layer 1, and layer 2). The user may perform a transformation operation in layer 2 and obtain a transformation result 805.

[0223] like Fig. 8A As shown in FIG. 8( a ), the drawing interface 800 of the electronic device 100 may further include a menu bar 801. The menu bar 801 may include setting items corresponding to multiple layers, for example, a setting item 802 corresponding to layer 2, a setting item 803 corresponding to layer 1, and a setting item 804 corresponding to the background layer. When the user needs to operate layer 2, the user may select the setting item 802. After the user selects the setting item 802, the user may perform a transformation operation in layer 2.

[0224] like Fig. 8A As shown in FIG. 8( a ), the drawing interface 800 may further include a toolbar 108 , and a control 1083 for enabling a transformation operation in the toolbar 108 may be highlighted to prompt the current user that a transformation operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0225] The user performs a transformation operation in layer 2, and the obtained transformation result 805 can be displayed in an image frame containing 6*6 pixels. Fig. 8A As shown in (b) of FIG. 8 , the transformation result of the N-1th frame, that is, the transformation result 805 can be displayed in 6*6 grids, where each grid represents a pixel.

[0226] The user can continue to perform transformation operations on layer 2. When transforming and drawing, all the drawing results of the transformation operation of the previous frame disappear, and only the transformation results of the current frame are displayed. Figure 8B As shown in FIG. 8( a ), the electronic device 100 can display the Nth frame display content in the drawing interface 800 . The Nth frame display content includes a transformation result 806 . The transformation result 806 can be the result obtained by the user through the transformation operation when the electronic device 100 displays the Nth frame. For example, the Nth frame image frame contains 6*6 pixels, and the Nth frame transformation result can be as follows: Figure 8B As shown in (b) in .

[0227] By comparing the transformation result of the Nth frame with the transformation result of the N-1th frame, the pixels changed by the user in the Nth frame can be obtained. The pixels in the transformation result of the Nth frame that are different from the transformation result of the N-1th frame are the pixels changed by the user in the Nth frame. The pixels changed by the user in the Nth frame can be called dirty pixels. Figure 8C The N-1th transformation result shown in (a) of FIG. Figure 8C Compared with the Nth frame transformation result shown in (b) in FIG. Figure 8C The Nth frame shown in Figure (c) transforms the dirty pixels. Figure 8C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Figure 8C The transformation result of the N-1th frame shown in Figure (a) is Fig. 8A Figure (b) shows the transformation result of the N-1th frame. Figure 8C The transformation result of the Nth frame shown in Figure (b) is Figure 8B Figure (b) shows the transformation result of the Nth frame.

[0228] Figure 9A-9C This shows the dirty pixels in the filter operation scenario. Fig. 9A As shown in FIG. 1 (a), the N-1 frame display content may be displayed in the drawing interface 900 of the electronic device 100. The N-1 frame display content may include content included in multiple layers (e.g., background layer, layer 1, and layer 2). Before the user filter operation, the N-1 frame display content may include the target content 905.

[0229] like Fig. 9AAs shown in FIG. 1 (a), the drawing interface 900 of the electronic device 100 may further include a menu bar 901. The menu bar 901 may include setting items corresponding to multiple layers, for example, a setting item 902 corresponding to layer 2, a setting item 903 corresponding to layer 1, and a setting item 904 corresponding to the background layer. When the user needs to operate layer 2, the user may select the setting item 902. After the user selects the setting item 902, the electronic device 100 may display the content in layer 2, that is, the target content 905.

[0230] like Fig. 9A As shown in FIG. 1 (a), the drawing interface 900 may further include a toolbar 108. A control 1084 for enabling filter operation in the toolbar 108 may be highlighted to prompt the current user that the filter operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0231] The display content before the filter operation of the N-1th frame, that is, the target content 905, can be displayed in an image frame containing 6*6 pixels. Fig. 9A As shown in (b) of FIG. 1 , the display content before the filter operation of the N-1th frame, that is, the target content 905 can be displayed in 6*6 grids, where each grid represents a pixel.

[0232] Users can Fig. 9A In response to the filter operation, the electronic device 100 may add a filter to the target content 905 in layer 2 to obtain the target content 906. For example, Fig. 9B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 900, and the display content includes the target content 906. Taking the Nth frame image frame containing 6*6 pixels as an example, the display content of the Nth frame after the filter operation can be as follows: Fig. 9B As shown in (b) in .

[0233] By comparing the displayed content after the Nth frame filter operation with the displayed content before the N-1th frame filter operation, the pixels changed by the user in the Nth frame can be obtained. The pixels in the displayed content after the Nth frame filter operation that are different from the displayed content before the N-1th frame filter operation are the pixels changed by the user in the Nth frame. The pixels changed by the user in the Nth frame can be called dirty pixels. In the filter operation scenario, the user's filter operation will change the color or style of all pixels with content. Therefore, the electronic device 100 can determine that the pixels whose color is changed by the filter operation are dirty pixels. Therefore, in the filter scenario, the dirty pixels can be all pixels with content in the current layer. Exemplarily, the electronic device 100 can Fig. 9CThe display content of the N-1 frame before the filter operation is shown in Figure (a) Fig. 9C Compared with the display content after the filter operation of the Nth frame shown in (b) in FIG. Fig. 9C The Nth frame filter shown in Figure (c) operates on dirty pixels. Fig. 9C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Fig. 9C The display content before the filter operation of the N-1th frame shown in Figure (a) is Fig. 9A Figure (b) shows the display content of the N-1th frame before the filter operation. Fig. 9C The content displayed after the filter operation of the Nth frame shown in Figure (b) is Fig. 9B Figure (b) shows the display content after the Nth frame filter operation.

[0234] Figure 10A-10C Shows the dirty pixels in the layer operation scenario. Fig. 10A As shown in FIG. 1 (a), the drawing interface 1000 of the electronic device 100 may display the N-1 frame display content. The N-1 frame display content may include content included in multiple layers (e.g., background layer, layer 1, and layer 2). Before the user adjusts the layer, the N-1 frame display content may include the target content 1005.

[0235] like Fig. 10A As shown in FIG. 1 (a), the drawing interface 1000 of the electronic device 100 may further include a menu bar 1001. The menu bar 1001 may include setting items corresponding to multiple layers, for example, a setting item 1002 corresponding to layer 2, a setting item 1003 corresponding to layer 1, and a setting item 1004 corresponding to the background layer. When the user needs to operate layer 2, the user may select the setting item 1002. After the user selects the setting item 1002, the electronic device 100 may display the content in layer 2, that is, the target content 1005.

[0236] like Fig. 10A As shown in FIG. 100 (a), the drawing interface 1000 may further include a toolbar 108. A control 1085 for enabling layer transformation operation in the toolbar 108 may be highlighted to prompt the current user that the layer transformation operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0237] The content displayed before the filter operation of the N-1th frame, that is, the target content 1005 can be displayed in an image frame containing 6*6 pixels. Fig. 10A As shown in (b), the display content before the filter operation of the N-1th frame, that is, the target content 1005 can be displayed in 6*6 grids, where each grid represents a pixel.

[0238] Users can Fig. 10A In response to the layer adjustment operation, the electronic device 100 may perform layer adjustment (for example, adjust visibility, adjust opacity) on the target content 1005 in layer 2. Fig. 10B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 1000, and the display content includes the target content 1005 after the layer adjustment. Taking the Nth frame image frame containing 6*6 pixels as an example, the display content after the layer adjustment operation of the Nth frame can be as follows: Fig. 10B As shown in (b) in .

[0239] By comparing the displayed content after the layer adjustment operation on the Nth frame with the displayed content before the layer adjustment operation on the N-1th frame, the pixels changed by the user in the Nth frame can be obtained. The pixels in the displayed content after the layer adjustment operation on the Nth frame that are different from the displayed content before the layer adjustment operation on the N-1th frame are the pixels changed by the user in the Nth frame. The pixels changed by the user in the Nth frame can be called dirty pixels. In the layer adjustment scenario, when the user performs a layer adjustment operation, the visibility and / or opacity of all pixels with content in the current layer will change. Therefore, in the layer adjustment scenario, the electronic device 100 can determine that all pixels with content are dirty pixels. Exemplarily, the electronic device 100 can Fig. 10C The display content of the N-1 frame before the layer adjustment operation is shown in (a) of FIG. Fig. 10C Compared with the display content after the layer adjustment operation of the Nth frame shown in (b), the following is obtained: Fig. 10C The Nth frame layer shown in Figure (c) adjusts the dirty pixels. Fig. 10C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Fig. 10C The display content before the layer adjustment operation of the N-1 frame shown in Figure (a) is Fig. 10A Figure (b) shows the display content of the N-1 frame before the layer adjustment operation. Fig. 10C The content displayed after the Nth frame layer adjustment operation shown in Figure (b) is Fig. 10B Figure (b) shows the display content after the Nth frame layer adjustment operation.

[0240] Figure 11A-11CShows dirty pixels in a color fill scene. Fig.11A As shown in FIG. 1 (a), the drawing interface 1100 of the electronic device 100 may display the N-1 frame display content. The N-1 frame display content may include content included in multiple layers (e.g., background layer, layer 1, and layer 2). After the user performs a color quick fill (which may be referred to as color fill) operation, the N-1 frame display content may include a color fill area 1105.

[0241] like Fig.11A As shown in FIG. 1 (a), the drawing interface 1100 of the electronic device 100 may further include a menu bar 1101. The menu bar 1101 may include setting items corresponding to multiple layers, for example, a setting item 1102 corresponding to layer 2, a setting item 1103 corresponding to layer 1, and a setting item 1104 corresponding to the background layer. When the user needs to operate layer 2, the user may select the setting item 1102. After the user selects the setting item 1102, the electronic device 100 may display the layer 2, and the layer 2 may include a fill area 1105.

[0242] like Fig.11A As shown in FIG. 100 (a), the drawing interface 1100 may further include a toolbar 108. A control 1086 for enabling the color quick fill operation in the toolbar 108 may be highlighted to prompt the current user that the color quick fill operation is being enabled. Figure 1 The description of the toolbar 108 is omitted here.

[0243] The N-1th frame display content may be displayed in an image frame containing 6*6 pixels. For example, Fig.11A As shown in FIG. 1( b ), the display content of the N-1th frame (including the color-filled area 1105 ) can be displayed in 6*6 grids, where each grid represents a pixel.

[0244] Users can Fig.11A The color quick fill operation is continued in layer 2 shown in (a) of FIG. In response to the color quick fill operation, the electronic device 100 may display a color fill area 1106 in layer 2. For example, Fig. 11B As shown in FIG. 1 (a), the electronic device 100 can display the Nth frame display content in the drawing interface 1000, and the display content includes the color-filled area 1106. Taking the Nth frame image frame including 6*6 pixels as an example, the Nth frame display content can be as follows: Fig. 11B As shown in (b) in .

[0245] By comparing the color-filled area of ​​the Nth frame with the color-filled area of ​​the N-1th frame, the pixels that the user has changed in the Nth frame can be obtained. The pixels in the color-filled area of ​​the Nth frame and the color-filled area of ​​the N-1th frame that have different colors are the pixels that the user has changed in the Nth frame. The pixels that the user has changed in the Nth frame can be called dirty pixels. In the color-filling scenario, the color diffuses out from the color-filling starting point at a certain speed until the entire area to be filled is filled. For pixels that have been filled and mixed in the previous frame, the current frame does not need to repeat the mixing, and only the newly added color-filled pixels need to be mixed. Therefore, in the color-filling scenario, the dirty pixels are the newly added color-filled pixels in each frame. Exemplarily, the electronic device 100 can Fig. 11C The N-1 frame coloring area shown in (a) and Fig. 11C Compared with the Nth frame coloring area shown in (b) in FIG. Fig. 11C The dirty pixel of the Nth frame is shown in Figure (c). Fig. 11C In Figure (c), each grid represents a pixel, where the grid without color filling (or called white grid) represents a pixel that is not a dirty pixel, and the grid filled with color (for example, dark gray) represents a dirty pixel. Fig. 11C The color-filled area of ​​the N-1th frame shown in Figure (a) is Fig.11A Figure (b) shows the color-filled area of ​​the N-1th frame. Fig. 11C The color-filled area of ​​the Nth frame shown in Figure (b) is Fig. 11B Figure (b) shows the color-filled area of ​​the Nth frame.

[0246] In this way, when the electronic device 100 determines the dirty pixels in the scenes of brush drawing, transformation operation, filter operation, layer adjustment and color quick filling, the electronic device 100 can synthesize the dirty pixels in the active layer of the user-drawn image with the pixels in other layers with the same position as the dirty pixels to obtain a synthesis result, and the electronic device 100 can display the synthesis result. In this way, when the electronic device 100 performs layer synthesis, it only synthesizes the dirty pixels, which can reduce the amount of calculation of layer synthesis and shorten the time consumption of layer synthesis, thereby improving the performance of real-time layer synthesis and improving user experience.

[0247] Fig.12 The following is a schematic diagram of a layer synthesis method provided in the present application example. Fig.12 As shown, a layer synthesis method provided by an embodiment of the present application may include the following steps:

[0248] S1201. The electronic device 100 displays a user interface 1. The user interface 1 includes an image 1. The image 1 includes multiple layers, including an active layer and a background layer.

[0249] The user interface 1 may be an image processing interface or an image drawing interface in an image processing application or a painting application. Figure 1 The drawing interface 101 is shown in FIG.

[0250] Taking the user interface 1 as an image drawing interface as an example, the user can draw an image in the user interface 1, and the electronic device 100 can also display the image drawn by the user in the user interface 1. The user draws a part of the content in the user interface 1, and the electronic device can display the part of the content drawn by the user in real time. In other words, the electronic device 100 can display the drawing content obtained by the user's drawing handwriting or other drawing operations (for example, transformation operations, filter operations, etc.) in real time. Exemplarily, the user interface 1 can include an image 1, which can be Figure 1 The drawn image 109 shown in Fig. 7A (a) includes an image of a triangle, a square, and a drawing handwriting 701. The present embodiment of the application does not specifically limit the image 1.

[0251] The image 1 may include multiple layers, and the multiple layers may include an active layer and a background layer. The embodiment of the present application does not limit the specific contents displayed in the active layer and the background layer of the image 1.

[0252] Optionally, the multiple layers may include multiple background layers, and the embodiment of the present application does not limit the number of background layers.

[0253] Optionally, the image 1 may also include one or more foreground layers, which is not limited in the embodiment of the present application.

[0254] S1202: The electronic device 100 detects the user's operation 1 for activating a layer.

[0255] The electronic device 100 can detect the user's operation 1 on the activated layer. The operation 1 can be any one of brush drawing, transformation operation, filter operation, layer adjustment and color quick filling performed by the user in the activated layer, and the embodiment of the present application does not specifically limit the operation 1.

[0256] S1203. In response to operation 1, the electronic device 100 records texture information 1, where texture information 1 includes dirty pixel information in the activated layer.

[0257] In response to operation 1, the electronic device 100 may record texture information 1, which may include dirty pixel information in the activated layer.

[0258] For example, in a possible implementation, a depth map may be stored in the memory of the electronic device 100. The depth map may be used to record the depth information of each frame of the drawn image. The texture information 1 may be the depth information recorded in the depth map.

[0259] Optionally, the depth map can be 16 bits, that is, the value of each pixel in the depth map can be a uint16 type ranging from 0 to 65535. uint16 is an unsigned integer type that occupies 16 bits (or 2 bytes) of storage space. The present embodiment of the application does not limit the storage format of the depth map. The depth map involved in the embodiment of the present application can be saved in any achievable format.

[0260] For different user operation scenarios, the method for the electronic device 100 to generate depth texture may be as shown in Table 1 below.

[0261] Table 1

[0262] Scenario The depth value of the depth texture of the Nth frame Brush Painting The pixel depth value of the stable point in the Nth frame is N; the pixel depth value of the unstable point in the Nth frame is N+1 Transform The converted pixel depth value of frame N is N+1 Filters All pixels with content have a depth value of 1 Layer Adjustment All pixels with content have a depth value of 1 Color Fast Fill The depth value of the newly added color pixel in the Nth frame is N

[0263] As shown in Table 1, in the brush drawing scene, the pixel depth value of the stable point drawn by the user in the Nth frame is N, and the pixel depth value of the unstable point drawn by the user in the Nth frame is N+1. N is a natural number greater than or equal to 1, that is, N=1, 2, 3, ... For example, Fig.13A As shown, Fig.13A Figure (a) shows the brush drawing content in frame N-1. Fig.13A Figure (b) shows the brush drawing content in frame N. Fig.13A The depth map corresponding to the brush drawing content in frame N-1 in (a) is Fig.13A (c) shows the depth map of the brush-drawn scene in frame N-1. Fig.13A In the depth map of the brush drawing scene in frame (C) shown in FIG. 1 , the pixel depth value of the stable point drawn by the brush in frame (N-1) is N-1; the pixel depth value of the unstable point drawn by the brush in frame (N-1) is N. Fig.13A The depth map corresponding to the brush drawing content of the Nth frame in (b) is Fig.13A The depth map of the brush-drawn scene in frame N is shown in (d). Fig.13A In the depth map of the brush drawing scene in the Nth frame shown in (d), the depth value of the pixel where the stable point of the brush drawing in the Nth frame is located is N; the depth value of the pixel where the unstable point of the brush drawing in the Nth frame is located is N+1. Optionally, the depth value corresponding to the pixel without the brush drawing trace in the depth map may be 0, which is not limited in the embodiment of the present application.

[0264] It is understandable that for the image drawn by the user on the electronic device 100, there is only one corresponding depth map in the memory of the electronic device 100. When the N-1th frame of brush drawing traces is detected, the electronic device 100 will change the depth value of the pixel where the brush drawing traces are located based on the user's brush drawing traces, and the depth values ​​of the other pixels will remain unchanged. When the Nth frame of brush drawing traces is detected, the electronic device 100 will adjust the depth value of the pixel where the Nth frame of brush drawing traces are located in the depth map, and the depth values ​​of the other pixels will remain unchanged.

[0265] As shown in Table 1, in the transformation scene, the transformation pixel depth value of the Nth frame is N+1. Fig. 13B As shown, Fig. 13B Figure (a) shows the transformation result of the N-1th frame. Fig. 13B Figure (b) shows the transformation result of the Nth frame. Fig. 13B The depth map corresponding to the transformation result of the N-1th frame in (a) is Fig. 13B (c) shows the depth map of the N-1th frame of the transformed scene. Fig. 13B In the depth map of the transformed scene in frame (C) shown in FIG. 1 , the pixel where the transformed content is located in the frame (N-1) has a depth value of N, that is, the transformed pixel. Fig. 13B The depth map corresponding to the brush drawing content of the Nth frame in (b) is Fig. 13B The depth map of the Nth frame of the transformed scene is shown in (d). Fig. 13B In the depth map of the Nth frame of the transformed scene shown in (d), the depth value of the transformed pixel in the Nth frame is N+1. Similarly, only one depth map is stored in the memory of the electronic device 100. After the electronic device 100 records the depth value of the transformed pixel corresponding to the transformation result of the N-1th frame in the depth map, it will continue to record the depth value of the transformed pixel of the Nth frame in the depth map. If the transformed pixel of the Nth frame is the same as the transformed pixel of the N-1th frame, then in the Nth frame, the electronic device 100 will adjust the depth value corresponding to the transformed pixel. For example. The depth value N corresponding to the transformed pixel in the N-1 frame is adjusted to the depth value N+1 corresponding to the Nth frame.

[0266] As shown in Table 1, in the filter scene, the depth value of all pixels with content (also referred to as drawn content) in the Nth frame is 1. For example, Fig. 13C As shown, Fig. 13C Figure (a) shows the display content before the filter operation of frame N-1. Fig. 13C Figure (b) shows the display content after the filter operation on the Nth frame. Fig. 13C In (a) of the figure, the depth map corresponding to the display content of the N-1th frame before the filter operation is Fig. 13C The depth map of the filtered scene in frame N-1 is shown in (c). Fig. 13CIn the depth map of the N-1th frame of the filter scene shown in (c), the depth value of all pixels with content in the N-1th frame is 1. Fig. 13C The depth map corresponding to the content displayed after the Nth frame filter operation in (b) is Fig. 13C The depth map of the Nth frame of the filtered scene is shown in (d). Fig. 13C In the depth map of the Nth frame of the filter scene shown in (d), the depth value of all pixels with content in the Nth frame is 1. Similarly, a depth map is stored in the memory of the electronic device 100. After the electronic device 100 records the depth values ​​of the pixels with content in the N-1th frame in the depth map, it will continue to record the depth values ​​of the content in the Nth frame in the depth map. In the filter scene, as long as the user performs a filter operation, the color values ​​of all pixels with content in the N-1th frame will change. In this scenario, the depth maps corresponding to the N-1th frame and the Nth frame are the same.

[0267] As shown in Table 1, in the layer adjustment scenario, the depth value of all pixels with content in the Nth frame is 1. For example, Fig.13D As shown, Fig.13D Figure (a) shows the display content before the layer adjustment operation of frame N-1. Fig.13D Figure (b) shows the display content after the layer adjustment operation of the Nth frame. Fig.13D The depth map corresponding to the displayed content before the adjustment operation of the N-1th layer in (a) is Fig.13D (c) shows the depth map of the N-1 frame of the layer adjustment scene. Fig.13D In the depth map of the N-1 frame of the layer adjustment scene shown in (c), the depth value of all pixels with content in the N-1 frame is 1. Fig.13D The depth map corresponding to the displayed content after the layer adjustment operation of the Nth frame in (b) is Fig.13D The layer adjustment scene (d) shows the depth map of the Nth frame. Fig.13D In the depth map of the Nth frame of the layer adjustment scenario shown in (d), the depth values ​​of all pixels with content in the Nth frame are 1. Similarly, the memory of the electronic device 100 only stores one depth map. After the electronic device 100 records the depth values ​​of the pixels with content in the N-1th frame in the depth map, it will continue to record the depth values ​​of the content in the Nth frame in the depth map. In the layer adjustment scenario, as long as the user performs a layer adjustment operation, the visibility and / or opacity of all pixels with content in the N-1th frame will change. The depth maps corresponding to the N-1th frame and the Nth frame are the same.

[0268] As shown in Table 1, in the color quick fill scene, the depth value of the newly added color fill pixel in the Nth frame is N. For example, Fig.13E As shown, Fig.13E Figure (a) shows the color-filled area of ​​frame N-1. Fig.13E Figure (b) shows the color-filled area of ​​the Nth frame. Fig.13E The depth map corresponding to the colored area of ​​the N-1th frame in (a) is Fig.13E (c) shows the depth map of the N-1th frame of the color quick fill scene. Fig.13E In the depth map of the N-1 frame of the color quick fill scene shown in (c), the depth value of the newly added color filling pixel in the N-1 frame is N-1. Fig.13E The depth map corresponding to the colored area of ​​the Nth frame in (b) is Fig.13E The color quick fill scene (d) shows the depth map of the Nth frame. Fig.13E In the depth map of the Nth frame of the color quick fill scene shown in (d), the depth value of the newly added color-filled pixel in the Nth frame is N. Similarly, the electronic device 100 only stores one depth map in its memory, and after the electronic device 100 records the depth value of the newly added color-filled pixel in the N-1th frame in the depth map, it will continue to record the depth value of the newly added color-filled pixel in the Nth frame in the depth map.

[0269] An operation performed by the user on an activated layer generally involves changes between multiple frames.

[0270] S1204. The electronic device 100 determines the dirty pixels in the activated layer based on the texture information 1.

[0271] The electronic device 100 may determine dirty pixels in the activated layer based on the texture information 1, for example, the depth map. The electronic device 100 may calculate whether each pixel in the depth map is a dirty pixel.

[0272] In one possible implementation, the CPU of the electronic device 100 can calculate the depth value of the pixels that need to be updated in the Nth image frame of the activated layer. In the brush drawing scene, the pixels that need to be updated in the Nth image frame are the pixels where the stable points and unstable points drawn by the user are located. In the transformation scene, the pixels that need to be updated in the Nth image frame are the pixels where the user's transformation operation content is located. In the filter scene, the pixels that need to be updated in the Nth image frame are the pixels where content exists. In the layer adjustment scene, the pixels that need to be updated in the Nth image frame are the pixels where content exists. In the color quick fill scene, the pixels that need to be updated in the Nth image frame are the newly added color fill pixels. The depth values ​​calculated by the CPU in different scenes can be found in Table 1 above. The CPU can send the calculated depth value to the GPU.

[0273] In one possible implementation, the CPU may store the calculated depth value in the CPU's memory, and then the CPU may instruct the GPU to copy the calculated depth value to the memory in the GPU. Then, the GPU calls the drawing instruction of the vertex shader to write the depth value into the texture information 1 to generate a depth map. The GPU may call the drawing instruction of the rendering pipeline for synthesis to perform a depth test to determine the dirty pixels in the depth map. Exemplarily, for different user operation scenarios, the method for calculating the dirty pixels in the depth map based on the depth test may be as shown in the following table.

[0274] Table 2

[0275] Scenario Dirty pixels in frame N based on depth test Brush Painting Pixels with depth values ​​greater than or equal to N Transform Pixels with depth values ​​greater than or equal to N Filters Pixels with a depth value of 1 Layer Adjustment Pixels with a depth value of 1 Color Fast Fill Pixels with depth values ​​greater than or equal to N

[0276] As shown in Table 2, in a brush drawing scenario, the electronic device 100 may determine that pixels in the depth map whose depth values ​​are greater than or equal to N are dirty pixels. Fig.13A As shown in (d), in the depth map of the Nth frame of the scene drawn by the brush, the pixels with depth values ​​of N and depth values ​​of N+1 are all dirty pixels.

[0277] As shown in Table 2, in the transformation scene, the electronic device 100 may determine that the pixels in the depth map whose depth values ​​are greater than or equal to N are dirty pixels. Fig. 13B As shown in (d), in the depth map of the Nth frame of the transformed scene, pixels with depth values ​​of N and N+1 are both dirty pixels.

[0278] As shown in Table 2, in the filter scene, the electronic device 100 may determine that the pixel with a depth value of 1 in the depth map is a dirty pixel. Fig. 13C As shown in (d) in the depth map of the Nth frame of the filter scene, all pixels with a depth value of 1 are dirty pixels.

[0279] As shown in Table 2, in the layer adjustment scenario, the electronic device 100 may determine that a pixel with a depth value of 1 in the depth map is a dirty pixel. Fig.13D As shown in (d) in the figure, in the depth map of the Nth frame of the layer adjustment scene, all pixels with a depth value of 1 are dirty pixels.

[0280] As shown in Table 2, in the color quick fill scenario, the electronic device 100 may determine that pixels in the depth map whose depth values ​​are greater than or equal to N are dirty pixels. Fig.13E As shown in (d), in the depth map of the Nth frame of the color quick fill scene, the pixels with a depth value of N are all dirty pixels.

[0281] Optionally, in a possible implementation, the electronic device 100 may store the dirty pixel marking result in a cache after determining the dirty pixel in the depth map according to the method in Table 2. The dirty pixel marking result may be used to mark whether the i-th pixel in the depth map is a dirty pixel.

[0282] S1205, the electronic device 100 synthesizes the dirty pixels in the activated layer with the pixels in the background layer at the same position as the dirty pixels, and generates image 2, where image 2 includes multiple layers included in image 1, and the content in the activated layer in image 2 is different from the content in the activated layer in image 1.

[0283] After the electronic device 100 determines the dirty pixels of the Nth frame in the activated layer, the electronic device 100 can synthesize the dirty pixels. For the Nth image frame, the electronic device 100 can synthesize the dirty pixels in the activated layer with the pixels in the background layer at the same position as the dirty pixels. For other pixels in the activated layer that are not dirty pixels, the electronic device 100 does not update their color values. Finally, the electronic device 100 can merge the synthesis results of the dirty pixels and the synthesis results of the non-dirty pixels to obtain Image 2. Image 2 can be the Nth frame image, and Image 1 can be the N-1th frame image.

[0284] For example, in a brush drawing scenario, the image 1 may be as follows: Fig. 7A The N-1th frame of the image is shown in (a) of FIG. Image 2 can be Figure 7B The Nth image frame is shown in FIG. (a) of FIG. In the transformation scene, the image 1 can be as follows Fig. 8A The N-1th frame of the image is shown in (a) of FIG. Image 2 can be Figure 8B The Nth image frame is shown in (a) of FIG. In the filter scene, image 1 can be as follows Fig. 9A The N-1th frame of the image is shown in (a) of FIG. Image 2 can be Fig. 9B The Nth image frame is shown in FIG. (a) in FIG. In the layer adjustment scene, image 1 can be as follows Fig. 10A The N-1th frame of the image is shown in (a) of FIG. Image 2 can be Fig. 10B The Nth image frame is shown in FIG. (a) of FIG. In the color quick fill scene, image 1 can be as follows Fig.11A The N-1th frame of the image is shown in (a) of FIG. Image 2 can be Fig. 11B The Nth image frame is shown in FIG. (a) of FIG. The present application embodiment does not specifically limit image 1 and image 2.

[0285] The specific steps of the electronic device 100 for performing layer synthesis can be found in Fig.14 .like Fig.14As shown, the electronic device 100 may perform layer synthesis including the following steps:

[0286] S1401. The CPU of the electronic device 100 calculates a depth value required to perform a depth test and transmits it to the GPU.

[0287] After the CPU sets the depth value for performing the depth test to the vertex shader in the rendering pipeline, the GPU can be instructed to perform the depth test through the rendering pipeline. The CPU can calculate the depth value for performing the depth test in different scenes, as described in Table 2 above.

[0288] S1402: The GPU of the electronic device 100 performs a depth test on the depth map to determine whether the pixels in the depth map pass the depth test; if so, execute step S1403.

[0289] The GPU may call a rendering pipeline for synthesis, and perform a depth test according to a depth value set for performing a depth test and a depth map stored in the GPU. When performing a depth test, the GPU may determine that a pixel in the depth map that meets the depth value for performing a depth test is a dirty pixel.

[0290] The GPU can perform a depth test on the depth map. During the depth test, the GPU can identify dirty pixels in the depth map according to the method for determining dirty pixels under different user operation scenarios in Table 2 above. For example, when the GPU determines that the current scene is a brush drawing scene or a transformation scene, or a color quick fill scene, the GPU can determine that the pixels with depth values ​​greater than or equal to N in the depth map corresponding to the Nth image frame can pass the depth test; the GPU can determine that the pixels with depth values ​​less than N in the depth map corresponding to the Nth image frame cannot pass (or are called failing) the depth test. For another example, when the GPU determines that the current scene is a filter scene or a layer adjustment scene, the GPU can determine that the pixels with depth values ​​of 1 in the depth map corresponding to the Nth image frame can pass the depth test; the GPU can determine that the pixels with depth values ​​other than 1 (for example, 0) in the depth map corresponding to the Nth image frame cannot pass (or are called failing) the depth test.

[0291] When the user selects an operation mode in the user interface 1, the target application corresponding to the user interface 1 can send the operation scene corresponding to the operation mode to the drawing engine. The drawing engine generates drawing instructions corresponding to different operation scenes by assembling the rendering pipeline on the CPU side. The drawing instructions can carry the user's operation scene. The drawing engine can send the drawing instructions to the GPU. The GPU can determine the current user operation scene and the corresponding method for calculating dirty pixels in the depth map through the user operation scene carried in the drawing instructions.

[0292] S1403 . The GPU of the electronic device 100 synthesizes the pixels that have passed the depth test to obtain a synthesis result 1 .

[0293] The electronic device 100 can synthesize the pixels that pass the depth test, that is, the dirty pixels. That is, the dirty pixels of the activated layer are synthesized with the pixels in the background layer with the same position as the dirty pixels. For example, if the dirty pixel in the activated layer is the pixel in the i-th row and j-th column, then the electronic device 100 can synthesize the dirty pixel with the pixel in the i-th row and j-th column in the background layer, and update the color value of the pixel. The GPU of the electronic device 100 does not update the color value of the pixel that fails the depth test.

[0294] After the GPU in the electronic device 100 synthesizes all the layers, a synthesis result 1 may be obtained. The synthesis result 1 may include the color values ​​of all dirty pixels of the active layer in the current frame and the color values ​​of all pixels that are not dirty pixels.

[0295] S1404 . The GPU of the electronic device 100 renders the synthesis result 1 to obtain an image 2 .

[0296] The synthesis result 1 includes the color values ​​of all pixels of the Nth image frame. The GPU of the electronic device 100 can render the synthesis result 1 into an image 2. Then, the electronic device 100 can display the image 2. The image 2 is the image that the user can see in the user interface 1 after the user performs the operation 1. The image 2 can also be called the Nth image frame.

[0297] It is understandable that before the user performs operation 1, the electronic device 100 displays image 1 in the user interface 1, and the image 1 can be stored in the memory 1 of the electronic device 100. After the user performs operation 2, the electronic device 100 displays image 2 in the user interface 1, and the electronic device 100 can store the image 2 in the memory 1 and overwrite the image 1. That is, the memory 1 will not store image 1 and image 2 at the same time. Image 2 will reuse the memory 1 storing image 1. The memory 1 can be the runtime memory that the GPU applies to the internal memory for storing the image displayed by the target application.

[0298] In this way, the electronic device 100 can fully utilize the depth test capability of the rendering pipeline to determine dirty pixels when performing layer synthesis, and only synthesize the dirty pixels, and skip synthesizing pixels that are not dirty pixels. In this way, the performance of layer synthesis can be improved.

[0299] Optionally, in the layer synthesis method provided in the embodiment of the present application, the electronic device 100 may not save the texture containing the dirty pixel information of each image frame, but directly use the difference between each image frame to calculate the dirty pixel, mark the dirty pixel, and save the marking result. In this way, the memory of the electronic device 100 can be saved. For example, Fig.15 As shown, a layer synthesis method provided by an embodiment of the present application may include the following steps:

[0300] S1501. The electronic device 100 displays a user interface 1. The user interface 1 includes an image 1. The image 1 includes multiple layers, including an active layer and a background layer.

[0301] For step S1501, please refer to the description of step S1201 above, which will not be repeated here.

[0302] S1502: The electronic device 100 detects the user's operation 1 for activating a layer.

[0303] For step S1502, please refer to the description of step S1202 above, which will not be repeated here.

[0304] S1503. In response to operation 1, the electronic device 100 calculates the dirty pixels in the activated layer.

[0305] The electronic device 100 may calculate the dirty pixel in the Nth image frame in the activated layer based on the difference between the Nth image frame and the N-1th image frame in the activated layer in response to operation 1. User operations in different scenarios in the electronic device 100 may correspond to different dirty pixel identification methods.

[0306] Specifically, in response to operation 1, the CPU may generate a drawing instruction and send the drawing instruction to the GPU. Based on the drawing instruction, the GPU calls the rendering pipeline to calculate the dirty pixels in the active layer. The GPU may calculate the dirty pixels of the active layer in different scenes according to the identification method of the dirty pixels in different scenes shown in Table 3 below.

[0307] Illustratively, dirty pixel identification methods for different scenarios in the electronic device 100 may be as shown in Table 3.

[0308] Table 3

[0309]

[0310] As shown in Table 3, in the brush drawing scenario, the electronic device 100 may determine that the dirty pixel of the Nth image frame in the active layer is the pixel where the unstable point is located in the N-1th image frame, the pixel where the stable point is located in the Nth image frame, and the pixel where the unstable point is located in the Nth image frame. Exemplarily, in the brush drawing scenario, the dirty pixel of the Nth image frame may be as follows: Figure 7C As shown in Figure (c).

[0311] As shown in Table 3, in the transformation scene, the electronic device 100 can determine that the dirty pixel of the Nth image frame in the active layer is the pixel where the transformation content is located in the N-1th image frame and the pixel where the transformation content is located in the Nth image frame. Exemplarily, in the transformation scene, the dirty pixel of the Nth image frame can be as follows: Figure 8C As shown in Figure (c).

[0312] As shown in Table 3, in the filter scenario, the electronic device 100 may determine that the dirty pixels of the Nth image frame in the activated layer are all pixels with content in the Nth image frame. Exemplarily, in the filter scenario, the dirty pixels of the Nth image frame may be as follows: Fig. 9C As shown in Figure (c).

[0313] As shown in Table 3, in the layer adjustment scenario, the electronic device 100 may determine that the dirty pixels of the Nth image frame in the activated layer are all pixels with content in the Nth image frame. Exemplarily, in the layer adjustment scenario, the dirty pixels of the Nth image frame may be as follows: Fig. 10C As shown in Figure (c).

[0314] As shown in Table 3, in the color quick fill scenario, the electronic device 100 can determine that the dirty pixels of the Nth image frame in the activated layer are all the newly added color-filled pixels in the Nth image frame. Fig. 11C As shown in Figure (c).

[0315] For example, in a possible implementation, when the user selects a control for enabling the brush drawing function in the user interface 1 (for example, Figure 1 When the user selects a control for enabling the transformation function in the user interface 1 (for example, Figure 1 1803 shown in FIG), the electronic device 100 may determine that the current scene is a transformation scene. When the user selects a control for enabling the filter function in the user interface 1 (eg, Figure 1When the user selects a control for enabling the layer adjustment function in the user interface 1 (for example, Figure 1 When the user selects a control for enabling the color quick fill function in the user interface 1 (for example, Figure 1 When the control 1086 shown in FIG. 1 is pressed, the electronic device 100 may determine that the current scene is a color quick fill scene.

[0316] The electronic device 100 can calculate the dirty pixel of each pixel in the Nth image frame in the manner shown in Table 3. Then, the electronic device 100 can mark each pixel in the Nth image frame. Exemplarily, the electronic device 100 can mark the dirty pixel of the Nth image frame as a value L1, and mark the pixel of the Nth image frame that is not a dirty pixel as a value L2. The value L1 is different from the value L2. For example, the value L1 can be 1, and the value L2 can be 0. The embodiment of the present application does not limit the values ​​of the value L1 and the value L2.

[0317] The electronic device 100 may save the dirty pixel calculation result of the Nth frame in the memory. There may be a memory space in the memory of the electronic device 100 for storing the dirty pixel storage result. When the electronic device 100 displays the N-1th image frame of the activated layer, the dirty pixel storage result stored in the memory space is the dirty pixel calculation result of the N-1th image frame. When the electronic device 100 displays the Nth frame of the activated layer, the electronic device 100 may first clear the memory space used to store the dirty pixel calculation result, and then store the dirty pixel calculation result of the Nth image frame. In the dirty pixel calculation result, each pixel only needs to use 1 bit to be saved.

[0318] S1504, the electronic device 100 synthesizes the dirty pixels in the activated layer with the pixels in the background layer at the same position as the dirty pixels, and generates image 2, where image 2 includes multiple layers included in image 1, and the content in the activated layer in image 2 is different from the content in the activated layer in image 1.

[0319] After the electronic device 100 determines the dirty pixels of the Nth frame in the activated layer, the electronic device 100 can synthesize the dirty pixels. For the Nth image frame, the electronic device 100 can synthesize the dirty pixels in the activated layer with the pixels in the background layer at the same position as the dirty pixels. For other pixels in the activated layer that are not dirty pixels, the electronic device 100 does not change their color values. Finally, the electronic device 100 can render the synthesis results of multiple layers to obtain Image 2. Image 2 can be the Nth frame image, and Image 1 can be the N-1th frame image.

[0320] Step S1504 may refer to the description of the above step S1204 and will not be repeated here.

[0321] In this way, when the electronic device 100 performs layer synthesis, it only synthesizes dirty pixels, which can improve synthesis performance. In addition, each pixel in the dirty pixel result stored by the electronic device 100 only needs one bit of memory, which can save the memory of the electronic device 100.

[0322] Optionally, in the layer synthesis method provided in the embodiment of the present application, the electronic device 100 may record the dirty pixel information in a single-channel monochrome texture, which may be, for example, an r channel, a g channel, or a b channel.

[0323] When the electronic device 100 performs layer synthesis through the fragment shader, each pixel in the monochrome texture is confirmed one by one whether it is a dirty pixel. When the fragment shader determines that the pixel in the monochrome texture is a dirty pixel, the electronic device 100 synthesizes the pixel through the fragment shader. When the fragment shader determines that the pixel in the monochrome texture is not a dirty pixel, synthesis is skipped for the pixel.

[0324] For example, Fig.16 As shown, a layer synthesis method provided by an embodiment of the present application may include the following steps:

[0325] S1601. The electronic device 100 displays a user interface 1. The user interface 1 includes an image 1. The image 1 includes multiple layers, including an active layer and a background layer.

[0326] Step S1601 may refer to the description of the above step S1201 and will not be repeated here.

[0327] S1602: The electronic device 100 detects the user's operation 1 for activating a layer.

[0328] Step S1602 may refer to the description of the above step S1202 and will not be repeated here.

[0329] S1603. In response to operation 1, the electronic device 100 records texture information 2, where texture information 2 includes dirty pixel information in the activated layer.

[0330] In response to operation 1, the electronic device 100 may save the texture information 2 of each image frame in the activated layer, wherein the texture information 2 includes the depth value of the dirty pixel. The storage format of the texture information 2 may be a single-channel uint16 type. It is understandable that the storage format of the texture information 2 may also be other forms, not limited to a single-channel uint16, and the embodiment of the present application does not limit the storage format of the texture information 2.

[0331] S1604, when the GPU of the electronic device 100 performs synthesis through the fragment shader, the fragment shader determines whether each pixel in the active layer is a dirty pixel based on the texture information 2, and when it is determined that the pixel 1 in the active layer is a dirty pixel, the pixel 1 in the active layer is synthesized with the pixel in the background layer at the same position as the pixel 1, and an image 2 is generated, wherein the image 2 includes multiple layers included in the image 1, and the content in the active layer in the image 2 is different from the content in the active layer in the image 1.

[0332] The electronic device 100 does not need to first calculate the dirty pixels in the Nth image frame in the activated layer, and directly inputs the monochrome texture containing dirty information of the Nth image frame (i.e., texture information 2) into the fragment shader for synthesis. When the GPU synthesizes through the fragment shader, the fragment shader can first determine whether each pixel in the Nth image frame is a dirty pixel. If so, synthesis is performed, and if not, synthesis is skipped. That is, when synthesizing, the fragment shader needs to determine whether each pixel is a dirty pixel. The Nth image frame in the activated layer contains pixel 1 and pixel 2. When it is determined that pixel 1 in the activated layer is a dirty pixel, pixel 1 in the activated layer is synthesized with the pixel in the background layer at the same position as pixel 1 to obtain a synthesis result 1; when it is determined that pixel 2 in the activated layer is not a dirty pixel, the color value of pixel 2 is not changed.

[0333] Exemplarily, the fragment shader can determine whether a pixel is a dirty point based on the depth value of pixel 1 in the Nth image frame in the active layer. The color texture containing the dirty point information stores the depth value of pixel 1. The fragment shader determines the dirty point pixels in the active layer according to the dirty point pixel identification methods in different scenarios shown in Table 2.

[0334] After the electronic device 100 determines the dirty pixels of the Nth frame in the activated layer, the electronic device 100 can synthesize the dirty pixels. For the Nth image frame, the electronic device 100 can synthesize the dirty pixels in the activated layer with the pixels in the background layer at the same position as the dirty pixels. For other pixels in the activated layer that are not dirty pixels, the electronic device 100 does not change their color values. Finally, the electronic device 100 can render the synthesis results of multiple layers to obtain Image 2. Image 2 can be the Nth frame image, and Image 1 can be the N-1th frame image.

[0335] In this way, when the electronic device 100 determines the dirty pixels in the scenes of brush drawing, transformation operation, filter operation, layer adjustment and color quick filling, the electronic device 100 can synthesize the dirty pixels in the active layer of the user-drawn image with the pixels in other layers with the same position as the dirty pixels to obtain a synthesis result, and the electronic device 100 can display the synthesis result. In this way, when the electronic device 100 performs layer synthesis, it only synthesizes the dirty pixels, which can reduce the amount of calculation of layer synthesis and shorten the time consumption of layer synthesis, thereby improving the performance of real-time layer synthesis and improving user experience.

[0336] In the embodiment of the present application, the first image may be image 1, and the second image may be image 2. The first value may be N, the second value may be N, the third value may be 1, the fourth value may be 1, the fifth value may be N, the sixth value may be value L1, and the seventh value may be value L2. The first depth value may be N, the second depth value may be N+1, the third depth value may be N+1, the fourth depth value may be 1, the fifth depth value may be 1, and the sixth depth value may be N. The embodiment of the present application does not limit the specific values ​​of the first value, the second value, the third value, the fourth value, the fifth value, the sixth value, the seventh value, the first depth value, the second depth value, the third depth value, the fourth depth value, the fifth depth value, and the sixth depth value.

[0337] Exemplarily, a comparison of the frame rate of the layer synthesis method of the embodiment of the present application and the frame rate of the layer synthesis method based on the dirty area bounding box can be shown in Table 4 below.

[0338] Table 4

[0339]

[0340]

[0341] As shown in Table 4, in the brush drawing scene, transformation scene, filter scene, and layer adjustment scene, no matter the number of layers to be synthesized is 50, 100, or 150, the frame rate of the layer synthesis method provided by the embodiment of the present application is higher than the frame rate of the layer synthesis method based on the dirty area bounding box. The present application can improve the performance of real-time layer synthesis in multiple scenes of painting applications.

[0342] In one embodiment, the present application also provides a device 1700. Fig.17As shown, the device 1700 may include: a processor 1701 (e.g., a CPU), and a memory 1702. The memory 1702 may include a high-speed random-access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory. The memory 1702 may store various instructions for completing various processing functions and implementing the method steps of the present application.

[0343] Optionally, the device 1700 involved in the present application may further include: a power supply 1703, a communication bus 1704 and a communication port 1705. The above-mentioned communication port 1705 is used to realize the connection and communication between the electronic device and other peripherals. In the embodiment of the present application, the memory 1702 is used to store computer executable program code, and the program code includes instructions; when the processor 1701 executes the instruction, the instruction causes the processor 1701 of the device 1700 to perform the action performed by the processor of the electronic device 100 in the above-mentioned method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0344] Optionally, the device 1700 involved in the present application may further include: a display screen 1706. The display screen 1706 is used to display an interface of the electronic device.

[0345] It should be noted that the modules or components described in the above embodiments may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program codes, such as a controller. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0346] Optionally, the device 1700 may further include a GPU, which may be an independent chip, or the GPU may be integrated into the CPU 1701 .

[0347] The device 1700 can be used to execute a layer synthesis method executed by the above-mentioned electronic device 100 and any possible implementation method of the layer synthesis method.

[0348] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0349] In some examples, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0350] In some examples, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0351] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0352] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0353] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned related steps to implement the layer synthesis method in the above-mentioned embodiment.

[0354] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the layer synthesis method in the above-mentioned embodiment.

[0355] In addition, an embodiment of the present application further provides a device. The device may be a component or a module, and the device may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device executes the layer synthesis method in the above-mentioned method embodiments.

[0356] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0357] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0358] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0359] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0360] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0361] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A layer synthesis method, characterized in that: The method comprises: The electronic device displays a first user interface of a target application, wherein the first user interface displays a first image, wherein the first image includes an activation layer and a background layer; wherein the target application is a painting application; The electronic device detects a first operation on the activation layer; In response to the first operation, the electronic device determines that a plurality of pixels in the active layer are dirty pixels, and the dirty pixels include a first pixel; The electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a second image, the position of the second pixel in the background layer is the same as the position of the first pixel in the activation layer, the second image includes the background layer of the first image, and the content of the activation layer in the second image is different from the content of the activation layer in the first image.

2. The method according to claim 1, characterized in that In response to the first operation, the electronic device determines that a plurality of pixels in the activated layer are dirty pixels, specifically comprising: In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame; The electronic device determines that a plurality of pixels in the activation layer are dirty pixels based on the depth map.

3. The method according to claim 2, characterized in that The electronic device determines, based on the depth map, that a plurality of pixels in the activation layer are dirty pixels, specifically comprising: When the first operation is a brush drawing operation and the depth value of the first pixel in the depth map is greater than or equal to a first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value of a depth test in a brush drawing scenario; or When the first operation is a transformation operation and the depth value of the first pixel in the depth map is greater than or equal to a second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is a threshold value of a depth test in a transformation scenario; or When the first operation is a filter operation and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is a threshold value of a depth test in a filter scenario; or When the first operation is a layer adjustment operation and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is a threshold value of a depth test in a layer adjustment scenario; or When the first operation is a color filling operation, and when the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is a threshold value for depth testing in a layer adjustment scenario.

4. The method according to claim 2, characterized in that: The electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves a depth map of the activated layer in the current frame, specifically including: In response to the first operation, the CPU determines a depth value of a pixel of the activated layer in a current frame and stores the depth value in a first memory of the CPU; The CPU instructs the GPU to copy the depth value in the first memory to the second memory of the GPU; The GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map; The electronic device determines, based on the depth map, that a plurality of pixels in the activation layer are dirty pixels, specifically comprising: The GPU performs a depth test on the depth map to obtain a depth test result; The GPU determines that multiple pixels in the activated layer are dirty pixels based on the depth test result.

5. The method according to any one of claims 1 to 4, characterized in that: The electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a second image, specifically comprising: The electronic device synthesizes the first pixel with the second pixel in the background layer to obtain a first synthesis result, wherein the color values ​​of the pixels at the same position as the first pixel in the first synthesis result are updated, and the color values ​​of the pixels other than the dirty pixel in the first synthesis result are not updated; The electronic device renders the first synthesis result to obtain a second image.

6. The method according to claim 4, characterized in that The GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map, specifically including: In the case where the first operation is a brush drawing operation, the GPU writes the depth value of the pixel where the stable point of the activated layer in the current frame is located as the first depth value, and writes the depth value of the pixel where the unstable point of the activated layer in the current frame is located as the second depth value in the texture containing the dirty point pixel information of the activated layer in the current frame, to generate the depth map, and the second depth value is greater than the first depth value; or When the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformation content of the activated layer in the current frame is located in the texture containing the dirty pixel information of the activated layer in the current frame as a third depth value to obtain the depth map; or When the first operation is a filter operation, the GPU writes the depth value of the pixel having content of the activated layer in the current frame as a fourth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map; or When the first operation is layer adjustment, the GPU writes the depth value of the pixel having content of the activated layer in the current frame as the fifth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map; or When the first operation is color filling, the GPU writes the depth value of the newly added filled pixels of the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map.

7. The method according to claim 1, characterized in that: In response to the first operation, the electronic device determines that a plurality of pixels in the activated layer are dirty pixels, specifically comprising: In the case where the first operation is a brush drawing operation, the electronic device determines that pixels where stable points and unstable points drawn by the user of the activated layer in the current frame are located, and pixels where unstable points drawn by the user of the activated layer in the previous frame are located are dirty pixels; or In the case where the first operation is a transformation operation, the electronic device determines that pixels in the activated layer where the transformation content is located in the current frame and pixels in the activated layer where the transformation content is located in the previous frame are dirty pixels; or In the case where the first operation is a filter operation, the electronic device determines that pixels of the activated layer having content in the current frame are dirty pixels; or, In the case where the first operation is a layer adjustment operation, the electronic device determines that pixels of the activated layer having content in the current frame are dirty pixels; or, In the case where the first operation is color filling, the electronic device determines that the pixels where the newly added color filling area of ​​the activated layer in the current frame is located are dirty pixels.

8. The method according to claim 7, characterized in that After the electronic device determines that a plurality of pixels in the active layer are dirty pixels, the method further includes: The electronic device marks the dirty pixel in the activated layer as a sixth value, and marks the pixels other than the dirty pixel in the activated layer as a seventh value, to obtain a marking result; The electronic device stores the marking result.

9. The method according to claim 1, characterized in that: In response to the first operation, the electronic device determines that a plurality of pixels in the activated layer are dirty pixels, specifically including: In response to the first operation, the electronic device saves the monochrome texture of the activated layer in the current frame, wherein the monochrome texture includes the depth value of the pixel on which the first operation is applied in the current frame of the activated layer; The electronic device determines that a plurality of pixels in the activated layer are dirty pixels based on the monochrome texture.

10. The method according to claim 9, characterized in that the electronic device comprises a processor CPU and a graphics processor GPU, and in response to the first operation, the electronic device saves the monochrome texture of the activated layer in the current frame, specifically comprising: In response to the first operation, the CPU determines a depth value of a pixel of the active layer on which the first operation is applied in the current frame, and sends a first drawing instruction to the GPU; In response to the first drawing instruction, the GPU draws a monochrome texture of the active layer in the current frame, and saves the depth value of the pixel acted by the first operation in the current frame of the active layer to the monochrome texture; The electronic device determines, based on the monochrome texture, that a plurality of pixels in the activated layer are dirty pixels, specifically including: The GPU determines that a plurality of pixels in the activated layer are dirty pixels based on the monochrome texture.

11. The method according to claim 9 or 10, characterized in that: In the case where the first operation is a brush drawing operation, the pixels of the activated layer acted upon by the first operation in the current frame are pixels where stable points and unstable points drawn by the user are located, the depth value of the pixel where the stable point is located is a first depth value, and the depth value of the pixel where the unstable point is located is a second depth value; or, In the case where the first operation is a transformation operation, the pixel of the activated layer in the current frame on which the first operation acts is the pixel where the transformation content is located in the current frame of the activated layer, and the depth value of the pixel where the transformation content is located is the third depth value; or, In the case where the first operation is a filter operation, the pixels of the activated layer in the current frame that are acted upon by the first operation are pixels of the activated layer that have content in the current frame, and the depth value of the pixel that has content is a fourth depth value; or, In the case where the first operation is a layer adjustment operation, the pixels of the activated layer in the current frame that are acted upon by the first operation are pixels of the activated layer that have content in the current frame, and the depth value of the pixel that has content is a fifth depth value; or, When the first operation is a quick color fill operation, the pixels of the activated layer acted upon by the first operation in the current frame are the pixels where a newly added color fill area of ​​the activated layer is located in the current frame, and the depth value of the pixels where the newly added color fill area is located is the sixth depth value.

12. The method according to claim 11, characterized in that A format of the monochrome texture stored in the GPU is different from a format of a depth map stored in the GPU.

13. A layer synthesis method, characterized in that: The method comprises: The electronic device displays a first user interface of a target application, wherein the first user interface displays a first image, wherein the first image includes an active layer, a background layer, and a foreground layer; wherein the target application is a painting application; The electronic device detects a first operation on the activation layer; In response to the first operation, the electronic device saves the depth map of the activated layer in the current frame; The electronic device determines, based on the depth map, that a plurality of pixels in the activation layer are dirty pixels, wherein the dirty pixels include a first pixel; The electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a second image, wherein the position of the second pixel in the background layer is the same as the position of the first pixel in the activation layer, the position of the third pixel in the foreground layer is the same as the position of the first pixel in the activation layer, and the second image includes the background layer of the first image and the foreground layer of the first image, and the content of the activation layer in the second image is different from the content of the activation layer in the first image.

14. The method according to claim 13, characterized in that The electronic device determines, based on the depth map, that a plurality of pixels in the activation layer are dirty pixels, specifically comprising: When the first operation is a brush drawing operation and the depth value of the first pixel in the depth map is greater than or equal to a first value, the electronic device determines that the first pixel is a dirty pixel, and the first value is a threshold value of a depth test in a brush drawing scenario; or When the first operation is a transformation operation and the depth value of the first pixel in the depth map is greater than or equal to a second value, the electronic device determines that the first pixel is a dirty pixel, and the second value is a threshold value of a depth test in a transformation scenario; or When the first operation is a filter operation and the depth value of the first pixel in the depth map is greater than or equal to a third value, the electronic device determines that the first pixel is a dirty pixel, and the third value is a threshold value of a depth test in a filter scenario; or When the first operation is a layer adjustment operation and the depth value of the first pixel in the depth map is greater than or equal to a fourth value, the electronic device determines that the first pixel is a dirty pixel, and the fourth value is a threshold value of a depth test in a layer adjustment scenario; or When the first operation is a color filling operation, and when the depth value of the first pixel in the depth map is greater than or equal to a fifth value, the electronic device determines that the first pixel is a dirty pixel, and the fifth value is a threshold value for depth testing in a layer adjustment scenario.

15. The method according to claim 13, characterized in that The electronic device includes a central processing unit (CPU) and a graphics processing unit (GPU). In response to the first operation, the electronic device saves a depth map of the activated layer in the current frame, specifically including: In response to the first operation, the CPU determines a depth value of a pixel of the activated layer in a current frame and stores the depth value in a first memory of the CPU; The CPU instructs the GPU to copy the depth value in the first memory to the second memory of the GPU; The GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map; The electronic device determines, based on the depth map, that a plurality of pixels in the activation layer are dirty pixels, specifically comprising: The GPU performs a depth test on the depth map to obtain a depth test result; The GPU determines that multiple pixels in the activated layer are dirty pixels based on the depth test result.

16. The method according to any one of claims 13 to 15, characterized in that: The electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a second image, specifically including: The electronic device synthesizes the first pixel with the second pixel in the background layer and the third pixel in the foreground layer to obtain a first synthesis result, wherein the color values ​​of the pixels at the same position as the first pixel in the first synthesis result are updated, and the color values ​​of the pixels other than the dirty pixel in the first synthesis result are not updated; The electronic device renders the first synthesis result to obtain a second image.

17. The method according to claim 15, characterized in that The GPU writes the depth value into a texture containing dirty pixel information of the activated layer in the current frame to generate a depth map, specifically including: In the case where the first operation is a brush drawing operation, the GPU writes the depth value of the pixel where the stable point of the activated layer in the current frame is located as the first depth value, and writes the depth value of the pixel where the unstable point of the activated layer in the current frame is located as the second depth value in the texture containing the dirty point pixel information of the activated layer in the current frame, to generate the depth map, and the second depth value is greater than the first depth value; or When the first operation is a transformation operation, the GPU writes the depth value of the pixel where the transformation content of the activated layer in the current frame is located in the texture containing the dirty pixel information of the activated layer in the current frame as a third depth value to obtain the depth map; or When the first operation is a filter operation, the GPU writes the depth value of the pixel having content of the activated layer in the current frame as a fourth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map; or When the first operation is layer adjustment, the GPU writes the depth value of the pixel having content of the activated layer in the current frame as the fifth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map; or When the first operation is color filling, the GPU writes the depth value of the newly added filled pixels of the activated layer in the current frame as the sixth depth value in the texture containing the dirty pixel information of the activated layer in the current frame to obtain the depth map.

18. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1-17.

19. A chip system, the chip system is applied to electronic equipment, the chip system comprises one or more processors, characterized in that: The processor is configured to call computer instructions so as to execute the method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 17.

21. A computer program product, characterized in that When the computer program product is executed on a computer, the electronic device is enabled to execute the method according to any one of claims 1 to 17.

Images