Animation playback method and electronic device

By using an inter-frame incremental compositing scheme, the incremental dirty region of the animation frame is identified, and only pixels in the changed region are synthesized. This solves the problem of long processing time for a single frame in animation playback, and improves the animation playback frame rate and user experience.

WO2026232838A1PCT designated stage Publication Date: 2026-11-12HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/146553
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-09
Filing Date
2025-12-29
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

During animation playback, electronic devices need to generate animation frames quickly, but existing technology results in long processing times for each frame, serious waste of resources, low frame rates and stuttering during animation playback, which affects the user's creative experience and visual effects.

Method used

An inter-frame incremental compositing scheme is adopted. By determining the incremental dirty area of ​​the animation frame, only the pixels in the changed area are composited, avoiding the compositing and rendering of all canvas pixels. This includes bounding box, slicing and pixel-level compositing and rendering methods. The appropriate compositing scheme is selected according to the proportion of the dirty area.

Benefits of technology

It reduces the processing time per frame, increases the animation playback frame rate, avoids stuttering, and improves the user's creative experience and animation playback effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025146553_12112026_PF_FP_ABST
    Figure CN2025146553_12112026_PF_FP_ABST
Patent Text Reader

Abstract

The present application provides an animation playback method and an electronic device. The method comprises: a drawing application can obtain a target frame on the basis of image content in an incremental dirty region in a layer of the target frame and the previous frame of the target frame, and play the target frame, wherein the incremental dirty region is used for indicating a region in which the image content in the layer of the target frame changes relative to image content in a layer of the previous frame, that is, a region where dirty pixel points that need to be recomposited and rendered among all pixel points of a canvas are located. In the method, a drawing application can obtain a target frame on the basis of a region in which image content of the target frame is different from image content of the previous frame, thereby avoiding performing compositing and rendering processing on all pixel points of a canvas for the target frame. In this way, resource waste of the drawing application can be reduced, and the processing time for a single frame during animation playback can be reduced, so that an animation playback frame rate can be increased, thereby avoiding stuttering during animation playback, and ensuring the animation playback effect, thus improving the user's creation experience.
Need to check novelty before this filing date? Find Prior Art

Description

An animation playback method and electronic device

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510603834.X, filed on May 9, 2025, entitled "An Animation Playback Method and Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of electronic device technology, and in particular to an animation playback method and electronic device. Background Technology

[0004] When users create image artwork using a drawing application, they can create multiple layers to draw different parts of the artwork, and use appropriate blending modes to composite these layers to obtain the resulting texture. Then, the resulting texture is sent for display, so that the final image artwork is displayed on the screen of an electronic device.

[0005] Frame-by-frame animation is another powerful feature of the drawing application. Users can use this feature to flexibly create multiple independent animation frames. Each animation frame can consist of one or more layers, and the blending result of all layers corresponding to any given animation frame is its display effect. Users can change the display effect of an animation frame by drawing, setting properties, and manipulating each layer. With frame-by-frame animation enabled, the drawing application's user interface displays a timeline, visually showing thumbnails and the order of all animation frames created for the current animation. Users can adjust the frame order to adjust the animation's playback effect. When the animation is completed and playback begins, the drawing application will play it frame-by-frame according to the timeline, thus presenting the animation's dynamic effects.

[0006] Therefore, during animation playback, the electronic device needs to generate and play each animation frame in real time according to the order of the animation frames on the timeline. In the process of generating any animation frame, the electronic device needs to perform multi-layer composite rendering of all layers of the animation frame. During this process, it may also need to composite the animation frame layers with other layers such as foreground frames, background frames, and background color layers according to set principles. In addition, the electronic device can also perform pixel transformations such as rotation, scaling, and translation on the composite texture obtained based on the above composite steps according to the display parameter settings, and then send the resulting texture of the animation frame for display.

[0007] Because the images displayed on electronic devices change rapidly during animation playback, these devices need to be able to generate animation frames quickly in real time. However, each time an electronic device generates an animation frame, it needs to perform composite rendering processing on all the pixels of the entire canvas (i.e., all pixels in the canvas). This leads to unnecessary resource waste, resulting in long processing times for each frame, low animation playback frame rate, stuttering animation playback, and negatively impacts the user's creative experience and the visual effect of the work. Summary of the Invention

[0008] This application provides an animation playback method and an electronic device, which reduces the processing time per frame and improves the frame rate of animation playback in scenarios where animations are played on electronic devices.

[0009] In a first aspect, embodiments of this application provide an animation playback method, which can be applied to drawing applications of electronic devices. The method includes the following steps:

[0010] A drawing application draws first image content on a first layer and second image content on a second layer; the drawing application plays a first frame; wherein the first frame includes the first image content in the first layer; the drawing application obtains a second frame based on the image content in a first incremental dirty region in the second layer and the first frame; wherein the first incremental dirty region is used to indicate the area of ​​change of the second image content relative to the first image content; the second frame includes the second image content in the second layer; and the second frame is played.

[0011] This method allows the rendering application to obtain the second frame based on the areas where the image content of the second frame differs from that of the previous frame, thereby avoiding full-canvas pixel compositing rendering of the second frame. This method provides an inter-frame incremental compositing scheme that can reduce resource waste in the rendering application, reduce the processing time of a single frame during animation playback, thereby improving the animation playback frame rate, avoiding stuttering during animation playback, ensuring animation playback effect, and improving the user's creative experience.

[0012] In one possible design, the rendering application can also determine the first incremental dirty region corresponding to the second frame in the following way:

[0013] Method 1: The drawing application determines the first incremental dirty region based on the first drawing operation on the second layer; wherein, the first drawing operation is used to draw the content of the second image;

[0014] Method 2: The drawing application determines the first incremental dirty area based on the first dirty area of ​​the first layer and the second dirty area of ​​the second layer; wherein, the first dirty area is the area where the first image content is located, and the second dirty area is the area where the second image content is located;

[0015] Method 3: The drawing application compares the pixels at the same position in the first layer and the second layer to determine at least one first pixel; the pixel value of the at least one first pixel is different in the first layer and the second layer; the first incremental dirty region is determined based on the at least one first pixel.

[0016] This design allows the rendering application to determine the first incremental dirty region corresponding to the second frame in multiple ways. Optionally, during the rendering phase, if the image content of the two frames is correlated (i.e., the image content of the second frame is modified based on the image content of the first frame), and the rendering application records the rendering operations performed on the layers of the second frame, then method 1 can be used to determine the inter-frame incremental dirty region corresponding to the second frame. If the rendering application does not record the rendering operations performed on the layers of the second frame, or the user does not perform any rendering operations on the layers of the second frame, or if there is no correlation between the image content of the two frames, the rendering application can determine the inter-frame incremental dirty region corresponding to the second frame using method 2 or method 3.

[0017] Using this method 1, the drawing application can determine the first incremental dirty region based on the drawing operations performed on the layers of the second frame during the drawing phase of the second frame, thereby avoiding the drawing application determining the first incremental dirty region during the animation playback phase and reducing the amount of computation for the drawing application during the animation playback phase.

[0018] Using method 2, the drawing application can determine the first incremental dirty region based on the dirty regions of the layers in the preceding and following frames.

[0019] Method 3 allows the rendering application to determine the first incremental dirty region by comparing whether the pixel values ​​at the same position in the first and second layers are the same. This ensures that the first incremental dirty region contains no or a small number of pixels with the same pixel values ​​in both layers (i.e., pixels in the second layer that do not need to be composited and rendered, redundant pixels). As a result, during the compositing and rendering stage of the second frame, the rendering application can reduce the compositing and rendering processing of redundant pixels, thereby further reducing the waste of rendering application resources.

[0020] In one possible design, the drawing application can determine the first incremental dirty region based on the first dirty region of the first layer and the second dirty region of the second layer in two ways:

[0021] Method 1: The drawing application can map the first dirty region and the second dirty region onto the same plane, and take the pixel region obtained by merging the two dirty regions as the first incremental dirty region. That is, the region of the first incremental dirty region mapped onto the reference plane is the union of the regions of the first dirty region and the second dirty region mapped onto the reference plane.

[0022] Method 2: The drawing application determines a target region based on the first dirty region and the second dirty region, wherein the target region is mapped onto the reference plane as the union of the regions mapped onto the reference plane by the first dirty region and the second dirty region; the drawing application compares pixels at the same position within the target region in the first layer and the second layer to determine at least one second pixel; wherein the pixel value of the at least one second pixel is different in the first layer and the second layer; finally, the drawing application determines the first incremental dirty region based on the at least one second pixel.

[0023] Using Method 1, the rendering application can calculate the first incremental dirty region based on the first and second dirty regions with minimal computation. Using Method 2, the rendering application can determine the first incremental dirty region by comparing the pixel values ​​at the same location within the target area in the first and second layers. This ensures that the first incremental dirty region contains few or no pixels with the same pixel values ​​in the two layers. Therefore, during the compositing rendering stage of the second frame, the rendering application can reduce the compositing rendering processing of redundant pixels, further reducing resource waste.

[0024] In one possible design, the first incremental dirty region is a first bounding box surrounding the changed region; based on this, the rendering application can obtain the second frame by the following steps, according to the image content within the first incremental dirty region in the second layer and the first frame:

[0025] The drawing application can update the image content within the first bounding box in the first frame based on the image content within the first bounding box in the second layer, thereby obtaining the second frame.

[0026] This design allows the rendering application to update the image content within the first bounding box of the second layer to the first bounding box of the first frame during the compositing rendering stage of the second frame, thus achieving bounding box-level inter-frame incremental compositing rendering. Since the rendering application does not need to perform compositing rendering on pixels outside the first bounding box across the entire canvas, this design reduces resource waste during the compositing rendering of the second frame, decreases processing time, and improves the compositing rendering efficiency of the second frame.

[0027] In one possible design, each layer is divided into multiple slices of equal size, and the first incremental dirty region includes the target slice among the multiple slices; based on this, the rendering application can obtain the second frame by the following steps, according to the image content within the first incremental dirty region in the second layer and the first frame:

[0028] The drawing application can update the image content within the target slice in the first frame based on the image content within the target slice in the second layer, thereby obtaining the second frame.

[0029] This design allows the rendering application to update the image content within the target slice in the second layer to the target slice in the first frame during the compositing rendering stage of the second frame, thus achieving slice-level incremental inter-frame compositing rendering. Since the rendering application does not need to perform compositing rendering on pixels outside the target slice across the entire canvas, this design reduces resource waste during the compositing rendering of the second frame, decreases processing time, and improves the efficiency of compositing rendering in the second frame.

[0030] In one possible design, the first incremental dirty region includes at least one dirty pixel; based on this, the drawing application can obtain the second frame by the following steps, according to the image content within the first incremental dirty region in the second layer and the first frame:

[0031] The drawing application can update the pixel value of at least one dirty pixel in the first frame based on the pixel value of at least one dirty pixel in the second layer to obtain the second frame.

[0032] This design allows the rendering application to update the pixel value of at least one dirty pixel in the second layer to the position of that at least one dirty pixel in the first frame during the compositing rendering stage of the second frame, thereby achieving pixel-level incremental inter-frame compositing rendering. Since the rendering application does not need to perform compositing rendering on pixels other than that at least one dirty pixel across the entire canvas, this design reduces resource waste during the compositing rendering of the second frame, decreases processing time, and improves the efficiency of compositing rendering in the second frame.

[0033] In one possible design, the rendering application can obtain a second frame based on the image content within the first incremental dirty region in the second layer and the first frame if the area ratio of the first incremental dirty region is less than a first threshold; and if the area ratio of the first incremental dirty region is greater than or equal to the first threshold, a full canvas compositing scheme can be used to composite and render the second frame based on all pixels in the second layer.

[0034] This design allows rendering applications to use appropriate compositing schemes based on the area ratio of the inter-frame incremental dirty regions corresponding to each animation frame during the animation playback phase, ensuring the smoothest possible animation playback effect.

[0035] Secondly, embodiments of this application provide an animation playback method, which can be applied to drawing applications of electronic devices. The method includes the following steps:

[0036] A drawing application draws first image content in a first layer group and second image content in a second layer group; wherein the first layer group contains multiple first layers; the second layer group contains multiple second layers; the drawing application plays a first frame; wherein the first frame includes the first image content in the first layer group; the drawing application obtains a second frame based on the image content within a first incremental dirty region in the second layer group and the first frame; wherein the first incremental dirty region is used to indicate the area of ​​change of the second image content relative to the first image content; the second frame includes the second image content in the second layer group; and the second frame is played.

[0037] In one possible design, each first layer draws a portion of the first image content, and the image content from these multiple first layers constitutes the first image content. Furthermore, if a first layer does not draw any content, that first layer may not participate in the composition of the first layer group.

[0038] In one possible design, each second layer contains a portion of the second image content, and the image content from these multiple second layers constitutes the second image content. Furthermore, if a second layer does not contain any content, that second layer may not participate in the composition of the second layer group.

[0039] This method allows the rendering application to obtain the second frame based on the areas where the image content of the second frame differs from that of the previous frame, thereby avoiding full-canvas pixel compositing rendering of the second frame. This method provides an inter-frame incremental compositing scheme that can reduce resource waste in the rendering application, reduce the processing time of a single frame during animation playback, thereby improving the animation playback frame rate, avoiding stuttering during animation playback, ensuring animation playback effect, and improving the user's creative experience.

[0040] In one possible design, the rendering application can determine the first incremental dirty region corresponding to the second frame in the following way:

[0041] Method 1: The drawing application determines the first incremental dirty region based on the first drawing operation on the second layer group; wherein, the first drawing operation is used to draw the content of the second image;

[0042] Method 2: The drawing application determines the first incremental dirty region based on the first dirty region of the first layer group and the second dirty region of the second layer group; wherein, the first dirty region is the area where the first image content is located, and the second dirty region is the area where the second image content is located;

[0043] Method 3: The drawing application compares the pixels at the same position in the first texture and the second texture to determine at least one first pixel; the pixel values ​​of the at least one first pixel in the first texture and the second texture are different; the first incremental dirty region is determined based on the at least one first pixel; wherein, the first texture is obtained by compositing the plurality of first layers in the first layer group, and the second texture is obtained by compositing the plurality of second layers in the second layer group.

[0044] In one possible design, the drawing application can determine the second dirty area of ​​the second layer group through the following steps:

[0045] The drawing application determines the dirty region of each second layer in the second layer group; wherein, the dirty region of any second layer is the area where the image content in the second layer is located; then, based on the dirty regions of the plurality of second layers, the second dirty region of the second layer group is obtained; wherein, the area of ​​the second dirty region mapped to the reference plane is the union of the areas of the dirty regions of the plurality of second layers mapped to the reference plane.

[0046] With this design, the drawing application can map all the dirty areas of the second layer in the second layer group onto the same plane and merge these dirty areas to obtain the second dirty area of ​​the second layer group.

[0047] In one possible design, the drawing application can determine the first incremental dirty region based on the first dirty region of the first layer group and the second dirty region of the second layer group in two ways:

[0048] Method 1: The drawing application can map the first dirty region and the second dirty region onto the same plane, and take the pixel region obtained by merging the two dirty regions as the first incremental dirty region. That is, the region of the first incremental dirty region mapped onto the reference plane is the union of the regions of the first dirty region and the second dirty region mapped onto the reference plane.

[0049] Method 2: The drawing application determines a target region based on the first dirty region and the second dirty region, wherein the region of the target region mapped onto the reference plane is the union of the regions of the first dirty region and the second dirty region mapped onto the reference plane; pixels at the same position in the third texture and the fourth texture are compared to determine at least one second pixel; wherein the third texture is obtained by compositing the image content within the target region in the plurality of first layers, and the fourth texture is obtained by compositing the image content within the target region in the plurality of second layers; the pixel values ​​of the at least one second pixel in the third texture and the fourth texture are different; the first incremental dirty region is determined based on the at least one second pixel.

[0050] Using Method 1, the rendering application can calculate the first incremental dirty region based on the first and second dirty regions with minimal computation. Using Method 2, the rendering application can determine the first incremental dirty region by comparing the pixel values ​​at the same positions in the third and fourth textures obtained from the image content within the target areas of the first and second layer groups. This ensures that the first incremental dirty region contains few or no pixels with the same pixel values ​​in the two layers. Therefore, during the compositing rendering stage of the second frame, the rendering application can reduce the compositing rendering processing of redundant pixels, further reducing resource waste.

[0051] In one possible design, the first incremental dirty region is a first bounding box surrounding the changed region; based on this, the rendering application can obtain the second frame by the following steps, according to the image content within the first incremental dirty region in the second layer group and the first frame:

[0052] The rendering application composites the image content within the first bounding box in the plurality of second layers to obtain a composite result; and updates the image content within the first bounding box in the first frame according to the composite result to obtain the second frame.

[0053] This design allows the rendering application to update the compositing result (i.e., image content) within the first bounding box of the second layer group to the first bounding box in the first frame during the compositing rendering stage of the second frame. This achieves box-level incremental inter-frame compositing rendering. Since the rendering application does not need to perform compositing rendering on pixels outside the first bounding box across the entire canvas, this design reduces resource waste during the compositing rendering of the second frame, decreases processing time, and improves the compositing rendering efficiency of the second frame.

[0054] In one possible design, each layer is divided into multiple slices of equal size, and the first incremental dirty region includes the target slice among the multiple slices; based on this, the rendering application can obtain the second frame by the following steps, according to the image content within the first incremental dirty region in the second layer group and the first frame:

[0055] The drawing application synthesizes the image content within the target slice in the plurality of second layers to obtain a synthesized result; and updates the image content within the target slice in the first frame according to the synthesized result to obtain the second frame.

[0056] This design allows the rendering application to update the composited result (i.e., image content) within the target slice in the second layer group to the target slice in the first frame during the compositing rendering stage of the second frame, thereby achieving slice-level incremental inter-frame compositing rendering. Since the rendering application does not need to perform compositing rendering on pixels outside the target slice across the entire canvas, this design reduces resource waste during the compositing rendering of the second frame, decreases processing time, and improves the compositing rendering efficiency of the second frame.

[0057] In one possible design, the first incremental dirty region includes at least one dirty pixel; based on this, the drawing application can obtain the second frame by the following steps, according to the image content within the first incremental dirty region in the second layer group and the first frame:

[0058] The drawing application synthesizes at least one dirty pixel in the plurality of second layers to obtain a composite result; and updates the pixel value of at least one dirty pixel in the first frame according to the composite result to obtain the second frame.

[0059] This design allows the rendering application to update the position of at least one dirty pixel in the first frame with the composited result (i.e., image content or pixel value) of at least one dirty pixel in the second layer group during the compositing rendering stage of the second frame, thereby achieving pixel-level incremental inter-frame compositing rendering. Since the rendering application does not need to perform compositing rendering on pixels other than the at least one dirty pixel across the entire canvas, this design reduces resource waste during the compositing rendering of the second frame, decreases processing time, and improves the compositing rendering efficiency of the second frame.

[0060] In one possible design, the rendering application can obtain a second frame based on the image content within the first incremental dirty region in the second layer group and the first frame if the area ratio of the first incremental dirty region is less than a first threshold; and if the area ratio of the first incremental dirty region is greater than or equal to the first threshold, a full canvas compositing scheme is used to composite and render the second frame based on all pixels in the second layer group.

[0061] Thirdly, embodiments of this application provide an electronic device including a display screen, one or more processors, and a memory; the display screen is used to display a user interface, the one or more processors are coupled to the memory, and the one or more processors are used to read a computer program stored in the memory to execute the method provided in any aspect. The computer program code includes computer instructions.

[0062] Fourthly, embodiments of this application provide a chip including a processor coupled to a memory. The processor is used to call computer program instructions stored in the memory during runtime to implement the methods provided in any of the above aspects. Optionally, the chip may further include components such as a memory, a communication interface, and a power supply module. The memory is used to store computer programs; the communication interface is used to receive and send data; and the power supply module is used to supply power to the processor.

[0063] Fifthly, embodiments of this application provide a computer storage medium storing computer program instructions that, when executed on an electronic device, cause the computer to perform the methods provided in any of the above aspects.

[0064] Sixthly, embodiments of this application provide a computer program product, the computer program product including computer program instructions; when the computer program instructions are executed on a computer, the computer causes the computer to perform the method provided in any one aspect.

[0065] The technical effects that can be achieved by any of the second to sixth aspects mentioned above can be described with reference to the technical effects that can be achieved by any possible design in the first aspect mentioned above. The repetitions will not be discussed. Attached Figure Description

[0066] Figure 1 is an example diagram of a user interface for a drawing application provided in an embodiment of this application;

[0067] Figure 2 is a schematic diagram of the processing flow from drawing to display of a work provided in an embodiment of this application;

[0068] Figure 3 is a schematic diagram of layer composition provided in an embodiment of this application;

[0069] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0070] Figure 5 is a software structure diagram of an electronic device provided in an embodiment of this application;

[0071] Figure 6 is a flowchart of an animation playback method provided in an embodiment of this application;

[0072] Figure 7A is an example of an animation drawn by a drawing application according to an embodiment of this application;

[0073] Figure 7B is an example diagram of an inter-frame incremental dirty region provided in an embodiment of this application;

[0074] Figure 7C is an example diagram of an inter-frame incremental dirty region provided in an embodiment of this application;

[0075] Figure 7D is an example diagram of a dirty region in an animation frame provided in an embodiment of this application;

[0076] Figure 7E is an example diagram of an inter-frame incremental dirty region provided in an embodiment of this application;

[0077] Figure 7F is an example diagram of a target area provided in an embodiment of this application;

[0078] Figure 7G is an example diagram of an inter-frame incremental dirty region provided in an embodiment of this application;

[0079] Figures 8A to 8C are schematic diagrams of the inter-frame incremental synthesis process provided in the embodiments of this application;

[0080] Figure 8D is a schematic diagram of a bounding box-level inter-frame incremental synthesis provided in an embodiment of this application;

[0081] Figure 8E is a schematic diagram of slice-level inter-frame incremental synthesis provided in an embodiment of this application;

[0082] Figure 8F is a schematic diagram of pixel-level inter-frame incremental synthesis provided in an embodiment of this application;

[0083] Figure 8G is a schematic diagram of an atlas multi-slice adaptive synthesis scheme provided in an embodiment of this application;

[0084] Figure 9A is a flowchart of an animation playback method provided in an embodiment of this application;

[0085] Figure 9B is a schematic diagram of texture atlasing provided in an embodiment of this application;

[0086] Figure 10 is a flowchart of an animation playback method provided in an embodiment of this application;

[0087] Figure 11 is a flowchart of an animation playback method provided in an embodiment of this application;

[0088] Figure 12 is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0089] This application provides an animation playback method and an electronic device to reduce the processing time per frame and improve the frame rate of animation playback in scenarios where the electronic device plays animations. The method and electronic device are based on the same technical concept. Since the principles by which the method and electronic device solve the problem are similar, the implementation of the electronic device and the method can be mutually referenced, and repeated details will not be elaborated further.

[0090] The following explanations of some terms used in this application are provided to facilitate understanding by those skilled in the art.

[0091] 1) Electronic equipment refers to devices or apparatuses that have data connectivity, data calculation, and processing capabilities. Generally, electronic equipment has a display screen capable of showing a user interface.

[0092] For example, the electronic device in this application may be a tablet computer, personal computer (PC), laptop computer, computer, netbook, in-vehicle computer, in-vehicle terminal, smartphone, smart wearable device (e.g., smartwatch, smart bracelet, smart glasses, smart helmet, etc.), personal digital assistant (PDA), smart home device (e.g., smart TV, smart mirror, smart speaker, etc.). This application does not limit the specific form of the electronic device.

[0093] Electronic devices can perform their functions and provide services to users through an operating system they run. For example, the electronic device may, but is not limited to, running an operating system... Or other operating systems.

[0094] 2) An application (APP) is installed on an electronic device and provides services to the user through specific business logic after it runs. Applications can be developed by the electronic device manufacturer, the operating system vendor, or a third-party application vendor; this application does not limit the scope of the application. Applications can also be called application programs, application software, etc.

[0095] For example, applications can include: various drawing applications, calling applications, camera applications, SMS applications, chat applications, video applications, music applications, electronic map applications, social applications, etc.

[0096] 3) Pixel (px): The basic unit of image display, also known as a pixel. A pixel cannot be further divided, and each pixel is a small square of a single color.

[0097] The pixel value of any given pixel is used to indicate the pixel's color or brightness information. For example, pixel values ​​can be represented by red-green-blue (RGB) values, grayscale values, or other forms, depending on the type of image and the processing method.

[0098] 4) Texture: A data object that contains image information. For example, this data object could be an array of pixel values.

[0099] For example, a data object composed of all pixels in a layer is a texture and can be called a layer texture. A data object composed of all pixels obtained by compositing multiple layers is also a texture and can be called a composite texture or a result texture, etc.

[0100] 5) Final texture: The texture used to store the final composite result of the layer during the rendering stage.

[0101] 6) Image Layer. In image processing or drawing applications, layers are components of a work. A work can consist of one or more layers, each containing different image content or visual elements. Layers can be stacked and blended to create the final display effect of the work. Each layer can be modified or drawn independently without affecting the image content of other layers.

[0102] In image processing or drawing applications, the layer that the user is currently editing can be called the active layer.

[0103] 7) Synthesizing: This is a step in the image rendering stage that takes multiple layers as input and outputs a single texture (or image). Synthesizing can also be called blending.

[0104] 8) Canvas. In image processing or drawing applications, the canvas is an abstract representation of the artwork; one can also consider the canvas as the artwork itself.

[0105] During animation production, image processing or drawing applications can set up a canvas for the animation, and each frame of the animation can be drawn and edited on the same canvas (i.e., the animation canvas).

[0106] In the description of this application, "full canvas pixels" means all the pixels in the canvas.

[0107] 9) Canvas size: The length and width of the canvas, usually expressed in pixels (px), millimeters (mm), or inches (in).

[0108] For example, 7680px * 4320px means the canvas has 7680 pixels long and 4320 pixels wide; 210mm * 297mm means the canvas has 210mm long and 297mm wide. The conversion between pixels and length is usually mediated by dots per inch (DPI) or pixels per inch (PPI). For example, DPI = 300 means there are 300 pixels per inch.

[0109] It should be noted that once the canvas size (or the size of the artwork) is set, all layers created for that artwork will have the same size as the canvas.

[0110] 10) Frame-by-frame animation is an animation technique that creates a continuous animation effect during playback by drawing or creating each individual frame of the animation one by one.

[0111] It's important to note that in scenes created using frame-by-frame animation technology, each frame of the animation has the same canvas size, matching the overall animation canvas size. Typically, when creating an animation, you only need to set the canvas size.

[0112] 11) Foreground frame: This refers to an animation frame composed of static elements located in the foreground of an animation. During animation playback, the foreground frame is not played as an independent animation frame, but it participates in image compositing as the foreground of each animation frame. Therefore, the foreground frame can also be called the foreground layer. Typically, during the compositing and rendering stage, the foreground layer is located above all other layers of the animation frame, as shown in Figure 3.

[0113] 12) Background frame: This refers to an animation frame composed of static elements located in the background of an animation. During animation playback, the background frame is not played as an independent animation frame, but it participates in image compositing as the background of each animation frame. Therefore, the background frame can also be called a background layer. Typically, during the compositing and rendering stage, the background layer is located below all other animation frame layers, as shown in Figure 3.

[0114] 13) Background color layer: This refers to the layer containing the background color of the animation frame. Optionally, the background color in this layer can be set in various ways, such as as a solid color, gradient, specific texture style, pattern, or image. Typically, during the compositing and rendering stage, the background color layer is located below all other layers of the animation frame, including the background layer; that is, the background color layer is the bottommost layer, as shown in Figure 3.

[0115] 14) Bounding box: A concept commonly used in object collision detection, rendering, and space management. A bounding box is a geometric shape used to simply enclose complex objects so that operations can be performed with less computation.

[0116] 15) Tile Blocks. To save storage space, electronic devices can slice the texture of an image during storage, dividing a texture into multiple small slices. Each slice can store the pixel values ​​of a portion of the texture's pixels, and each slice can be called a tile block. The process of slicing textures described above is also called texture atlasing.

[0117] 16) Vertex buffer: A memory area used in computer graphics to store vertex data, typically used on a graphics processing unit (GPU).

[0118] 17) Rendering pipeline: In computer graphics, this is a collection of steps or stages that process and generate computer images. It is typically used to convert 3D models into 2D images and ultimately display them on a screen.

[0119] 18) Vertex shader: This is an early stage in the rendering pipeline, primarily responsible for processing vertex data. For example, the vertex shader can transform vertex data in a 3D model from 3D space (i.e., model space) to 2D space (clipping space).

[0120] 19) Fragment Shader: This is part of the rendering pipeline, primarily responsible for calculating how each pixel (or fragment) of an image should be displayed on the screen. The fragment shader can calculate the color and other attributes of each pixel on a pixel-by-pixel basis, achieving composite rendering of each pixel.

[0121] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0122] In the embodiments of this application, descriptions such as "when," "under the circumstances," "if," and "if" all refer to the electronic device making corresponding processing under certain objective circumstances. They are not time limits, nor do they require the electronic device to make a judgment action when implementing it, nor do they imply any other limitations.

[0123] It should be noted that in this application, "used for indicating" can include both direct and indirect indication. When describing information as being used to indicate A, it can include whether the information directly or indirectly indicates A, but does not necessarily mean that the information contains A. Taking first information used to indicate first content as an example, the first information can contain the first content, or a part of the first content, or an identifier, index, indicator, etc., of the first content, and can also contain algorithms, calculation parameters, etc., for determining the first content. This application does not limit the manner of "indication".

[0124] In this application, "containing / including A" may be equivalent to "containing / including information A". Information A is used to indicate A.

[0125] In addition, it should be understood that in the description of this application, the words "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0126] Currently, drawing applications generally support drawing animations using frame-by-frame animation technology. Users can create multiple independent animation frames in a drawing application and then play these animation frames one by one in a set order to play the animation. Figure 1 illustrates an example of the user interface of a drawing application on a tablet computer. Referring to Figure 1(a), the user interface of the drawing application may include: a drawing area 11, a layer management area 12, a playback control area 13, and at least one menu area (as shown in menu areas 14 and 15).

[0127] Users can use their fingers, knuckles, styluses (or pen styluses), or other media to perform drawing operations in the drawing area 11. This embodiment of the application does not limit the media used by the user when performing drawing operations. The drawing application can display corresponding drawing content in the drawing area 11 based on the drawing operations performed by the user. During the drawing process, the user can change the size of the drawing area 11 through scaling operations.

[0128] For example, drawing operations performed on any layer may include, but are not limited to: drawing strokes with a brush, adding graphics or lines, moving or scaling a graphic or line, setting or modifying the color or transparency of certain image content, setting the texture style, color, transparency, and other attributes of the layer, erasing or deleting image content, cutting or pasting image content, and other operations that can change the image content in the layer.

[0129] The layer management area 12 can display at least one layer created by the user for an animation frame, as well as foreground frames, background frames, or background color layers that can be shared by all animation frames created for the current animation. Users can create and delete layers in the layer management area 12, and can also select a layer from the multiple layers displayed in the layer management area 12 so that the drawing area 11 can display the image content of that layer.

[0130] The playback control area 13 can display at least one playback control button, such as the play button on the left and the pause button on the right in the figure; it can also display a timeline, which can intuitively show the thumbnails and arrangement order of all animation frames created for the current animation. As shown in Figure 1(a), the number displayed in the upper left corner of each animation frame thumbnail indicates the arrangement order of the animation frames, and Figure 1(b) shows the animation frame corresponding to each number. Optionally, the user can adjust the arrangement order of the animation frames in the playback control area 13, thereby adjusting the playback effect of the animation. When the user clicks the play button in the playback control area 13, the drawing application can play the animation frames in the animation in the drawing area 11 in sequence. While the drawing application is playing the animation, when the user clicks the pause button in the playback control area 13, the drawing application will stop playing the animation; at this time, the animation frames displayed in the drawing area 11 will no longer change. Optionally, the layer management area 12 on the right can display at least one layer contained in the animation frames currently displayed in the drawing area 11.

[0131] Optionally, multiple controls can be displayed in the menu area of ​​the drawing application. When there are multiple menu areas, controls with different functions can be located in different menu areas. Optionally, users can move the position of the menu areas according to their personal needs, and can also adjust the menu areas where the controls are located. For example, as shown in Figure 1(a), the drawing application may include a drawing menu area 14 and a control menu area 15.

[0132] The drawing menu area 14 can display multiple drawing controls, each corresponding to a different drawing function. For example, the drawing menu area 14 may include: a brush control corresponding to the brush function, an eraser control corresponding to the eraser function, a color picker control corresponding to the color picker function, a graphic control corresponding to the graphic function, a brightness control corresponding to the brightness adjustment function, etc. For example, the brush function of the drawing application may, but is not limited to, provide the following brush types: pencil, brush, oil paintbrush, crayon, etc.

[0133] The control menu area 15 can also display multiple control controls, each corresponding to a different control function. For example, the control menu area 15 may include: an undo control for the undo function, a redo control for the redo function, a text box control for the add text box function, a font control for the font management function, a color control for the color management function, a gallery control for the add image function, and so on.

[0134] In some embodiments, the user interface of the drawing application may also include other content, such as an exit control, a save control, etc., and this application embodiment does not limit this.

[0135] In addition, to increase user control over the animation's rhythm, the drawing application can support adjusting the animation frame rate. This means users can adjust the animation's playback frame rate, i.e., the number of frames displayed per second, to control the animation's speed and smoothness. A higher frame rate results in faster scene transitions and a more fluid and seamless animation.

[0136] Optionally, after the user has drawn all the animation frames of the animation using the drawing application's frame-by-frame animation technology, the animation can be saved. At this point, the drawing application can save the animation as a file in formats such as Graphics Interchange Format (GIF), Flash, Audio Video Interleaved (AVI), MP4, or MOV, for easy saving and transmission.

[0137] It should be noted that Figure 1 is used as an example and does not constitute any limitation on electronic devices, user interfaces of drawing applications, etc.

[0138] Figure 2 illustrates the overall processing flow of a work from drawing to display. The following explanation uses the i-th frame of an animation as an example. As shown in Figure 2, during the drawing stage, the user can create n layers (i.e., layer 1, layer 2, ..., layer n in the figure) for the i-th frame of the animation in the drawing application. The user can perform input operations on each layer to update the layer texture of each layer. For example, the user can perform drawing operations, attribute setting operations, layer control operations, etc., on any layer to change the display effect of the image content in that layer, thereby adjusting the display effect of the i-th frame of the animation. During the compositing stage of the animation playback phase, the drawing application can use the target blending mode to sequentially composite all layers of the i-th frame of the animation from bottom to top, generating a result texture. Finally, in the rendering and display stage, the drawing application can perform pixel processing such as transformation and scaling on each pixel in the result texture, as well as rendering processing, to obtain the display result texture and send it to the display screen, allowing the user to see the drawing result of the i-th frame of the animation on the display screen.

[0139] Typically, during frame-by-frame animation playback, rendering applications can composite the currently playing animation frames in real time, resulting in textures. The rendering application can then display the composited textures on the screen at a fixed frequency according to a set frame rate, seamlessly connecting them with the preceding and following animation frames to create a smooth animation effect. During the compositing process, rendering applications can select appropriate blending modes for multi-layer blending. Commonly used blending modes include Normal, Overlay, and Soft Light. Through these blending modes, rendering applications can precisely control the interaction of image content between different layers, thereby creating richly colored, realistically lit, and layered images, producing works with greater depth and dimension.

[0140] Figure 3 illustrates an exemplary layer compositing method. Continuing with the example of the i-th frame of the animation, referring to Figure 3, during the compositing stage, the rendering application can first composite all layers of the i-th frame from bottom to top to obtain an intra-frame composite result (i.e., an intra-frame composite texture). Then, the rendering application can sequentially composite the intra-frame composite result with the background frame, foreground frame, and background color layer to obtain the final texture. The intermediate result obtained during the compositing stage can also be called a temporary texture, as shown in the figure. The foreground frame, background frame, and background color layer are used to describe the static elements of the foreground and background in the animation, respectively. They do not participate in the dynamic changes of the animation and only participate in layer compositing as compositing elements.

[0141] It should be noted that, as an example, Figure 3 does not specify the composition order of multiple layers. In actual implementation, the drawing application can use other orders for composition. For example, the drawing application can first composite the background frame with layer 1, and then composite the intermediate results with the other layers of the i-th animation frame in a bottom-to-top order.

[0142] Currently, in frame-by-frame animation playback scenarios, rendering applications use an inter-frame full-canvas compositing scheme to composite each animation frame. Therefore, the rendering application needs to composite and render all pixels contained in the entire canvas, that is, to composite all pixels in the layer of each animation frame, update the texture of the composited result on the entire canvas, thereby enabling the switching between adjacent frames of the animation and achieving the playback effect.

[0143] However, during animation playback, the scenes switch rapidly, requiring rendering applications to provide fast real-time compositing and computational capabilities. Therefore, high single-frame processing performance is required. Furthermore, the differences in image content between adjacent animation frames are generally small, and the area requiring recompositing is limited. For example, as shown in Figure 1(b), in an animation of a cartoon character dancing, only the character's hand animation is involved. In this case, involving all pixels in the entire canvas in compositing and subsequent computational processing would lead to unnecessary resource waste, resulting in longer single-frame processing times, lower animation playback frame rates, and consequently, frequent stuttering during animation playback, affecting the user's creative experience and the visual effect of the work.

[0144] Based on this, this application provides an animation playback method and an electronic device. In this method, a rendering application can obtain and play a target frame based on the image content within the incremental dirty region of the target frame's layer and the previous frame. The incremental dirty region indicates the area where the image content in the target frame's layer changes relative to the image content in the previous frame's layer; that is, the area containing dirty pixels that need to be re-composited and rendered across the entire canvas. Through this method, the rendering application can obtain the target frame based on the area where the image content of the target frame differs from the image content of the previous frame, thereby avoiding full-canvas pixel-level compositing and rendering of the target frame. This method provides an inter-frame incremental compositing scheme, which can reduce resource waste in the rendering application, reduce the processing time of a single frame during animation playback, thereby improving the animation playback frame rate, avoiding stuttering during animation playback, ensuring animation playback quality, and improving the user's creative experience.

[0145] The solutions provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0146] This application provides an electronic device that can implement the animation playback method provided in this application. The structure of the electronic device provided in this application will be described below with reference to FIG4.

[0147] As shown in Figure 4, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a SIM card interface 195, etc.

[0148] Processor 110 may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a GPU, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0149] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. By providing memory, the number of times the processor 110 accesses data in the internal memory 121 can be reduced, thus reducing the processor 110's waiting time and improving system efficiency.

[0150] As a dedicated hardware module for image processing, the GPU can efficiently handle graphics and parallel computing tasks. For example, GPUs can perform computational tasks in scenarios such as rendering, compositing, machine learning, and scientific computing. In the electronic device 100, while the AP or CPU is processing general computing tasks, it can call the GPU through the GPU interface and GPU driver to handle image-related tasks, thereby achieving heterogeneous image processing and ensuring the performance of the AP or CPU.

[0151] Display screen 194 is used to display various user interfaces. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1. Display screen 194 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces (GUIs). For example, display screen 194 may display windows, photos, videos, web pages, or documents, etc.

[0152] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, program code for at least one application, etc. The operating system may include, but is not limited to, […]. The program storage area can store computer programs for implementing the input event reporting method provided in the embodiments of this application. When the processor 110 executes these computer programs, the electronic device can execute the animation playback method provided in this application. The data storage area can store data created during the use of the electronic device 100, etc.

[0153] In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0154] The sensor module 180 may include, but is not limited to, fingerprint sensors, touch sensors, pressure sensors, magnetic sensors, ambient light sensors, barometric pressure sensors, bone conduction sensors, etc.

[0155] A touch sensor, also known as a "touch panel," can be located on the display screen 194. The touch sensor detects touch operations applied to or near it. It transmits the detected touch operation to the access point (AP) to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some embodiments, the touch sensor can be integrated with the display screen 194 to form a touch display screen. For example, when an electronic device is running a drawing application, the device can display the user interface of the drawing application on the touch display screen, where the user can perform drawing operations using a finger, knuckle, stylus, or other medium. The touch display screen can report the detected drawing operations to the AP, thereby displaying the corresponding drawing content in the user interface. In other embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 194.

[0156] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide wireless communication solutions for electronic device 100, including 2G / 3G / 4G / 5G. Wireless communication module 160 can provide wireless communication solutions for electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), Beidou navigation satellite system (BDS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technology.

[0157] It should be understood that the electronic device 100 shown in Figure 4 is merely an example and does not constitute a limitation on the electronic device. In practical applications, the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0158] The software system of the electronic device 100 provided in this application embodiment can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes a layered architecture as an example to illustrate the software structure of the animation drawing business of the electronic device, as shown in Figure 5.

[0159] A layered architecture divides the software within an electronic device into several layers, each with a clear role and function. Layers can communicate with each other through software interfaces. As shown in Figure 5, the software structure of an electronic device, from top to bottom, includes at least: an application layer, an application framework layer (FWK), system libraries, a rendering hardware interface, and a kernel layer. In addition, the electronic device also includes a hardware layer to provide hardware resource support.

[0160] The application layer is the top layer of the operating system, including the operating system's native applications and installed third-party applications, such as camera, gallery, calendar, Bluetooth, music, video, messaging, calculator, chat applications, video applications, music applications, electronic map applications, social applications, and so on.

[0161] As shown in Figure 5, in this application, the application layer of the electronic device may include a drawing application. This drawing application has functions such as drawing images, animations, and image processing. Optionally, the drawing application may be configured with functional modules such as a user interface, artwork management, layer management, frame-by-frame animation, and drawing functions. The drawing function modules may include, but are not limited to, modules for brush drawing, filters, transformations, layer adjustments, and quick fill color. The brush drawing module is used to implement the drawing function. The artwork management module is used to delete, create, or modify artworks, and to set artwork attributes such as name and canvas size. The user interface module is used to implement the user interface control of the drawing application. The layer management module is used to create, delete, or select layers. The frame-by-frame animation module is used to implement the frame-by-frame animation drawing function of the drawing application.

[0162] It is understood that different users' electronic devices and different types of electronic devices may contain different application packages. This application embodiment does not limit the specific applications included in the application layer of electronic device 100.

[0163] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer includes predefined functions. As shown in Figure 5, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0164] The window manager provides window management services (WMS). The window manager can determine the screen size, check if a status bar is present in a window, lock the screen, take screenshots, and more.

[0165] Content providers are used to store and retrieve data, making that data accessible to applications. This data may include, but is not limited to, videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0166] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A user interface can consist of one or more visual controls.

[0167] The phone manager is used to provide telephone functionality for electronic devices 100. This includes managing call status (including connection and hang-up).

[0168] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0169] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0170] Optionally, in some scenarios, the software system of electronic device 100 may also include a runtime. For example, when the operating system of electronic device 100 is Android, the software system of electronic device 100 may also include the Android runtime. The runtime is responsible for the scheduling and management of the operating system. Optionally, the Android runtime may include core libraries and a virtual machine. The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library. The application layer and application framework layer can run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer into binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0171] The system library may include a drawing engine that implements drawing functions, as well as multiple other functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0172] In some embodiments, the rendering engine is used to implement the painting or image processing functions of the rendering application. As shown in Figure 5, the rendering engine may include modules such as layer composition, brush painting, filter operations, transformation operations, and rendering pipeline. These modules can assemble and configure the rendering pipeline according to their own logic, and the rendering pipeline is used to generate rendering instructions.

[0173] The brush drawing module is used to draw image content on layers based on user drawing operations. The layer compositing module is used to execute the layer compositing process, as shown in Figure 3. The filter operation module is used to manage filters on layers or artwork. The transformation operation module is used to transform layers or artwork, such as rotating and scaling. The animation compositing module is used to generate animations based on multiple drawn animation frames. For example, multiple animation frames can be played sequentially at a specific frame rate to switch scenes on the current screen and connect with the scenes of previous and subsequent animation frames to form a smooth animation effect.

[0174] The rendering pipeline module may also include a vertex shader and a fragment shader. The vertex shader can be used to perform calculations and transformations on the three vertices of each triangle region in the image to be rendered. The fragment shader can be used to calculate the color of each pixel in the image to be rendered and to color each pixel.

[0175] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. The Media Library supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics. In some examples, the drawing engine includes both the 3D Graphics Processing Library and the 2D Graphics Engine.

[0176] The kernel layer provides core system services for the operating system, such as security, memory management, process management, network protocol stack, and driver models, all of which are implemented based on the kernel layer. As an abstraction layer between the hardware and software stack, the kernel layer contains many hardware-related drivers for electronic devices, such as display drivers, GPU drivers, camera drivers, keyboard drivers, sensor drivers, audio drivers, Bluetooth drivers, and WiFi drivers.

[0177] Optionally, in scenarios where the hardware layer of the electronic device 100 includes a GPU, the kernel layer and system libraries include an interface for the operating system (or CPU) to call GPU storage and computing functions, such as a rendering hardware interface. As shown in the figure, in some examples, the rendering hardware interface may include a direct extension (DX)12 interface, a Metal interface, a Vulkan interface, an OpenGL interface, etc.

[0178] The hardware layer can include various sensors, displays, cameras, GPUs, and other hardware.

[0179] It should be noted that Figure 5 is merely an example of a software structure diagram for an electronic device, which only simply lists the layers and software modules related to the solution of this application. In practical applications, the operating system of an electronic device may also include other layers, and each layer may also include other software modules for implementing one or more functions or services. This application does not specifically limit this aspect. Furthermore, this application does not limit the specific layer where each software module resides; for example, the rendering engine may reside in the application framework layer, and the rendering hardware interface may reside in the kernel layer.

[0180] To reduce the processing time per frame and improve the animation playback frame rate when rendering applications play animations, this application provides an animation playback method. This method can be applied to the electronic device shown in Figure 4, whose software structure is shown in Figure 5. Referring to the flowchart shown in Figure 6, this application will be described in detail below, using the animation rendering service of an electronic device as an example.

[0181] Step 601: The drawing application draws the first image content on the first layer and the second image content on the second layer.

[0182] Optionally, during the rendering stage of the target animation scene using frame-by-frame animation technology, the rendering application can render image content in created layers based on the user's rendering operations. In this embodiment, the first layer is at least one layer created for the first frame, also referred to as the layer of the first frame. The second layer is at least one layer created for the second frame, also referred to as the layer of the second frame. In one example, when there are multiple first layers, these multiple first layers can also be referred to as a first layer group, with each first layer rendering a portion of the image content from the first image content, and the image content from these multiple first layers constitutes the first image content; when there are multiple second layers, these multiple second layers can also be referred to as a second layer group, with each second layer rendering a portion of the image content from the second image content, and the image content from these multiple second layers constitutes the second image content. Additionally, during the rendering stage, the rendering application can also determine the arrangement order of all animation frames contained in the target animation based on user operations. In this embodiment, the first frame and the second frame are two adjacent animation frames of the target animation, wherein the first frame precedes the second frame.

[0183] For example, after the electronic device starts the drawing application, a window of the drawing application can be displayed on the screen. The user interface of the drawing application displayed in the window can be as shown in Figure 1(a). The user can perform drawing operations in the user interface of the drawing application, thereby drawing the corresponding image content in each layer. Continuing with the example of drawing a cartoon character dancing in Figure 1, the cartoon character in each frame of the animation in Figure 1 can also be drawn through multiple layers, as illustrated in Figure 7A: In Figure 7A(a), layer 1 and layer 2 are layers created for the first frame, i.e., the first layer. The drawing application can draw the image content of the first frame in layer 1 and layer 2 respectively according to the user's drawing operations; In Figure 7A(b), layer 3 and layer 4 are layers created for the second frame, i.e., the second layer. The drawing application can draw the image content of the second frame in layer 3 and layer 4 respectively according to the user's drawing operations.

[0184] Additionally, when there are static, unchanging elements in the foreground and / or background of the target animation, the user can create at least one of the following for these static elements: a foreground frame, a background frame, or a background color layer. These layers describing static elements are shared by all animation frames in the target animation. Continuing with the example of the animation scene depicting a cartoon character dancing shown in Figure 1, as shown in Figure 7A(c), the user can create a foreground frame, a background frame, and a background color layer in the drawing application for the static elements in the foreground and background of the animation; the drawing application can draw image content in the foreground frame, background frame, and background color layer according to the user's drawing operations. The foreground frame, background frame, and background color layer in the embodiments of this application can be explained with reference to the terms used in points 11) to 13) above, and will not be repeated here.

[0185] Optionally, the drawing application can draw the corresponding image content in each layer using traditional drawing methods or other drawing methods. The specific drawing process, and the process by which the drawing application displays the currently drawn image content, will not be described further in this embodiment.

[0186] The following steps 602 to 604 are the steps performed during the target animation playback stage of the drawing application. Each step is explained below.

[0187] Step 602: Draw the application to play the first frame; wherein the first frame includes the first image content in the first layer.

[0188] In this embodiment of the application, the drawing application can play each animation frame of the target animation sequentially according to the order of the animation frames of the target animation.

[0189] Step 603: The drawing application obtains the second frame based on the image content in the first incremental dirty area in the second layer and the first frame; wherein, the first incremental dirty area is used to indicate the area of ​​change of the second image content relative to the first image content; the second frame includes the second image content in the second layer.

[0190] Since the first incremental dirty region is used to indicate the area where the content of the second image changes relative to the content of the first image, that is, the area where dirty pixels that need to be re-composited and rendered are located in the entire canvas, the first incremental dirty region can also be called the inter-frame incremental dirty region corresponding to the second frame. Here, dirty pixels are pixels that need to be re-composited and rendered, and can also be called dirty points.

[0191] Through the above step 603, the drawing application can obtain the current frame based on the area where the image content of the current frame differs from that of the previous frame, thereby avoiding the composite rendering processing of all canvas pixels of the current frame. This method can reduce the waste of resources of the drawing application, reduce the processing time of a single frame in animation playback, thereby improving the animation playback frame rate, avoiding stuttering in animation playback, ensuring the animation playback effect, and improving the user's creative experience.

[0192] In this embodiment of the application, the drawing application may determine the first incremental dirty region before performing step 603. Optionally, the drawing application may determine the first incremental dirty region through any of the following embodiments A1 to A3.

[0193] Implementation method A1: The drawing application can determine the first incremental dirty region based on the first drawing operation on the second layer. The first drawing operation is used to draw the content of the second image.

[0194] Optionally, during the drawing phase of the second layer, the drawing application can record all drawing operations performed on the second layer (including the aforementioned first drawing operation), thereby determining the first incremental dirty region based on the drawing area corresponding to the recorded first drawing operation. For example, in a scenario where the software structure of the animation drawing business of an electronic device is as shown in Figure 5, the brush drawing module in the drawing engine of the drawing application can record the drawing operations performed on the second layer and determine the first incremental dirty region.

[0195] For example, in a scenario where the second image content is a modification of some image content based on the first image content, the drawing application can record the drawing operations performed by the user on the second layer. These drawing operations may include: copying the first image content of the first layer into the second layer (or copying the first layer and using the copied first layer as the second layer), and performing at least one drawing operation in the second layer to modify some image content based on the first image content. For example, modifying some image content based on the first image content may include, but is not limited to, at least one of the following: erasing image content, adding image content, and modifying display attributes such as color, transparency, shape, and size of the image content.

[0196] Based on this, the drawing application can identify the first drawing operation from the drawing operations recorded for the second layer, which is used to modify part of the image content based on the first image content; then, based on the drawing area corresponding to each first drawing operation, the first incremental dirty region can be determined. That is, the first incremental dirty region is the effective drawing area of ​​the second frame.

[0197] It should be noted that when there are multiple second layers, different first drawing operations may be performed on different layers within these multiple second layers; that is, the drawing areas corresponding to different first drawing operations may not lie in the same plane. Therefore, to facilitate an accurate description of the relationship between the first incremental dirty region and the drawing area corresponding to each first drawing operation, the concept of a reference plane is introduced in this embodiment to aid understanding. The area of ​​the first incremental dirty region mapped to the reference plane is the union of the areas mapped to the drawing area corresponding to each first drawing operation. Optionally, the size of the reference plane is the same as the canvas size of the target animation. For example, the reference plane can be the canvas of the target animation or any layer in the second layers; this application does not limit this.

[0198] Continuing with the example of the layers corresponding to frames 1 and 2 in Figures 7A(a) and (b), and referring to Figure 7B: After the drawing application creates layer 3, it can copy the image content of layer 1 into layer 3 based on the user's drawing operation 1 on layer 3. After the drawing application creates layer 4, it can copy the image content of layer 2 into layer 4 based on the user's drawing operation 2 on layer 4, as shown in Figures 7B(a) and (b). Then, based on the user's drawing operation 3 on layer 4, the drawing application can modify some image content based on the original image content of layer 2, i.e., adjust the posture of the cartoon character's arm, as shown in Figures 7B(b) and (c). Since drawing operation 3 modifies some image content based on the image content of frame 1, the drawing application can determine the incremental dirty area corresponding to frame 2 based on the drawing area corresponding to drawing operation 3. As shown in Figure 7B(c), the pixel area within the dashed box in layer 4 is the drawing area corresponding to drawing operation 3. Based on the drawing area corresponding to drawing operation 3, the drawing application can determine the incremental dirty area corresponding to the second frame, which is the pixel area within the solid box in the canvas of the target animation in Figure 7B(d). In this embodiment, the pixel area is the area composed of pixels.

[0199] Through implementation A1, the rendering application can determine the first incremental dirty region during the rendering phase of the second frame, and then save the first incremental dirty region information used to indicate the first incremental dirty region. In one implementation, the rendering application can save the first incremental dirty region information to the image metadata of the second frame, so that the first incremental dirty region information can be directly obtained from the image metadata of the second frame during the playback phase of the target animation, thereby determining the first incremental dirty region. For example, the first incremental dirty region information can indicate the pixel position of the first incremental dirty region in the canvas of the target animation.

[0200] Implementation Method A2: Implementation Method A1 above determines the first incremental dirty region based on the drawing operation of the second layer. However, determining whether a pixel in the current frame needs to be re-rendered essentially requires determining whether the pixel value of that pixel in the current frame has changed compared to the pixel value of that pixel in the previous frame. For example, a pixel in the entire canvas may be within the drawing area of ​​the second frame, but its pixel value has not changed compared to the previous frame; therefore, this pixel is not a dirty pixel and does not need to be re-rendered. Therefore, in this implementation method, the drawing application can consider the pixel values ​​of pixels at the same position in two adjacent frames to determine the first incremental dirty region. Optionally, the drawing application can determine the first incremental dirty region through the following steps A2-1 to A2-2.

[0201] Step A2-1: The drawing application can compare pixels at the same position in the first texture and the second texture to determine at least one first pixel. The first texture is obtained from the first layer, and the second texture is obtained from the second layer; the pixel value of this at least one first pixel differs between the first texture and the second texture.

[0202] Optionally, in this embodiment, before executing step A2-1, the intra-frame composite texture (i.e., the first texture) of the first frame can be obtained based on the first layer, and the intra-frame composite texture (i.e., the second texture) of the second frame can be obtained based on the second layer, that is, intra-frame pre-compositing processing can be performed on the second layer. For example, when there are multiple first layers, the rendering application can composite these multiple first layers to obtain the first texture; or when there is only one first layer, the rendering application can use the first layer as the first texture. Similarly, when there are multiple second layers, the rendering application can composite these multiple second layers to obtain the second texture; or when there is only one second layer, the rendering application can use the second layer as the second texture.

[0203] Optionally, in a scenario where the software structure for animation rendering on an electronic device is as shown in Figure 5, the rendering application can compare pixels in two textures through the rendering pipeline during step A2-1. For example, the rendering application can input the first and second textures into the fragment shader in the rendering pipeline, so that the fragment shader can traverse the two textures pixel by pixel, compare pixels located at the same position in the two textures, and obtain at least one first pixel (i.e., a dirty pixel) with different pixel values ​​in the two textures.

[0204] Step A2-2: The drawing application determines the first incremental dirty region based on at least one first pixel.

[0205] Optionally, in step A2-2, the drawing application can directly use the at least one first pixel as the first incremental dirty region; or the drawing application can determine a pixel region with the smallest area containing the at least one first pixel based on the location of the at least one first pixel, and use that pixel region as the first incremental dirty region; or the drawing application can use the at least one tile block containing the at least one first pixel as the first incremental dirty region. In summary, the embodiments of this application do not limit the method of determining the first incremental dirty region based on the at least one first pixel.

[0206] For example, suppose a first texture and a second texture are shown in Figure 7C(a), each texture containing corresponding image content; the drawing application can compare the first texture and the second texture through a fragment shader to determine at least one first pixel with different pixel values ​​in the two textures, wherein the at least one first pixel is all the pixels within the two solid lines in Figure 7C(b).

[0207] In one example, the drawing application may use the at least one first pixel as the first incremental dirty region, as shown in (c) of Figure 7C.

[0208] In another example, the drawing application can determine the pixel region with the smallest area containing the at least one first pixel based on the location of the at least one first pixel, and use the pixel region as the first incremental dirty region, as shown in (d) of Figure 7C.

[0209] In another example, in a scenario where the entire canvas texture of the target animation is atlased, each full canvas texture can be divided into multiple equally sized slices (described below using tile blocks as an example). As shown in Figure 7C(e), the canvas of the target animation consists of multiple tile blocks; in this scenario, the drawing application can determine the tile block in which the at least one first pixel is located, and use these tile blocks as the first incremental dirty region, as shown in Figure 7C(e).

[0210] Optionally, in a scenario where the entire canvas of the target animation is atlas-ified, both the first and second textures are based on tile blocks (or slices). Therefore, when comparing the first and second textures, the rendering application can traverse each tile block in the target animation's canvas, comparing the texture data of the currently traversed tile block in both the first and second textures using a fragment shader to determine if the pixel values ​​of pixels at the same position in the two sets of texture data from the two textures are the same. If the pixel values ​​of at least one pixel in the two sets of texture data are different, the rendering application considers it a dirty tile block in the first incremental dirty region. If the pixel values ​​of all pixels in the two sets of texture data are the same, the rendering application does not consider it a dirty tile block in the first incremental dirty region. Finally, the rendering application can determine at least one dirty tile block in the target animation's canvas as the first incremental dirty region, as shown in Figure 7C(e).

[0211] Through implementation method A2, the rendering application can determine the first incremental dirty region by comparing whether the pixel values ​​at the same position in the first texture and the second texture are the same. This ensures that there are no or a small number of pixels with the same pixel values ​​in the two textures (i.e., pixels in the second texture that do not need to be composited and rendered, redundant pixels) in the first incremental dirty region. As a result, during the compositing and rendering stage of the second frame, the rendering application can reduce the compositing and rendering processing of redundant pixels, thereby further reducing the waste of resources of the rendering application.

[0212] Implementation method A3: The drawing application can determine the first incremental dirty area through the following steps A3-1 to A3-3.

[0213] Step A3-1: Draw the application to determine the first dirty area of ​​the first layer.

[0214] Step A3-2: Draw the application to determine the second dirty area of ​​the second layer.

[0215] The first dirty region is the area where the first image content is located, and the second dirty region is the area where the second image content is located.

[0216] Since the first dirty region is the pixel area where the first image content is located in the first frame, it can also be called the dirty region of the first frame (or the dirty region of the frame). The second dirty region is the pixel area where the second image content is located in the second frame, and it can also be called the dirty region of the second frame (or the dirty region of the frame).

[0217] Step A3-3: The drawing application can determine the first incremental dirty area based on the first dirty area and the second dirty area.

[0218] In one implementation of this embodiment, the drawing application can determine the first dirty area of ​​the first layer after drawing the first layer, that is, execute step A3-1 after drawing the first layer; after drawing the second layer, determine the second dirty area of ​​the second layer, and determine the first incremental dirty area based on the first dirty area and the second dirty area, that is, execute steps A3-2 and A3-3 after drawing the second layer.

[0219] Optionally, after determining the first incremental dirty region through step A3-3, the rendering application can save the first incremental dirty region information used to indicate the first incremental dirty region. Optionally, similar to embodiment A1, after determining the first incremental dirty region, the rendering application can save the first incremental dirty region information to the image metadata of the second frame, so that the first incremental dirty region information can be directly obtained from the image metadata of the second frame during the playback stage of the target animation, thereby determining the first incremental dirty region.

[0220] In another implementation, the rendering application can determine the first dirty region of the first layer after rendering the first layer, and save the first dirty region information used to indicate the first dirty region; after rendering the second layer, it can determine the second dirty region of the second layer, and save the second dirty region information used to indicate the second dirty region. During the playback phase of the target animation, the rendering application can obtain the first dirty region information and determine the first dirty region of the first layer based on the first dirty region information; and obtain the second dirty region information and determine the second dirty region of the second layer based on the second dirty region information; then, the rendering application can determine the first incremental dirty region based on the first dirty region and the second dirty region, that is, the rendering application can perform the above steps A3-1 to A3-3 during the playback phase of the target animation. Optionally, in this implementation, the rendering application can save the first dirty region information to the image metadata of the first frame and save the second dirty region information to the image metadata of the second frame; in this way, during the playback phase of the target animation, the rendering application can obtain the first dirty region information from the image metadata of the first frame; and obtain the second dirty region information from the image metadata of the second frame.

[0221] In this implementation A3, the drawing application can use the same method to determine the dirty area of ​​the layer for each animation frame. The following explanation uses the determination of the second dirty area of ​​the second layer created for the second frame as an example. The second dirty area can be determined, but is not limited to, through the following implementation methods B1 or B2:

[0222] Implementation method B1: When the second layer is a single layer, the drawing application can use the area containing the second image content in the second layer as the second dirty area.

[0223] Implementation method B2: When there are multiple second layers (i.e., these multiple second layers can form a second layer group), the drawing application can determine the second dirty area of ​​the second layer (or second layer group) through the following steps B2-1 to B2-2:

[0224] Step B2-1: The drawing application can determine the dirty area of ​​each of the multiple second layers. The dirty area of ​​any given second layer is the region containing the image content within that second layer.

[0225] Optionally, the outline of any dirty area in the second layer can be a set regular shape (such as a circle, rectangle, etc.) or the same as the outline of the image content in the second layer. This application does not limit this.

[0226] Step B2-2: The drawing application can obtain the second dirty region based on the dirty regions of the multiple second layers. The area of ​​the second dirty region mapped onto the reference plane is the union of the areas of the dirty regions of the multiple second layers mapped onto the reference plane.

[0227] Since these multiple second layers are on different planes, this application introduces the concept of a reference plane to aid understanding in order to accurately describe the relationship between the first incremental dirty region and the dirty regions of each second layer. The description of the reference plane can be found in the above embodiment A1, and will not be repeated here.

[0228] It should be noted that the outline of the second dirty area can be a set regular shape (such as a circle, rectangle, etc.) or the same as the outline of the second image content. This application does not limit this.

[0229] Continuing with the example of the layer corresponding to the second frame shown in Figure 7A(b), and referring to Figure 7D for further explanation: After the user performs a drawing operation on layer 3, the drawing application can determine the dirty area of ​​layer 3 based on the area where the image content is located, i.e., the pixel area within the dashed box in layer 3 as shown in Figure 7D(a); After the user performs a drawing operation on layer 4, the drawing application can determine the dirty area of ​​layer 4 based on the area where the image content is located, i.e., the pixel area within the dashed box in layer 4 as shown in Figure 7D(a); Then the drawing application can map the dirty areas of layer 3 and layer 4 onto the canvas of the target animation, as shown in Figure 7D(b), thereby merging the dirty areas of layer 3 and layer 4 to obtain the dirty area of ​​the second frame (also called the dirty area of ​​the layer (or layer group, i.e., layer 3 and layer 4) of the second frame, or the frame dirty area of ​​the second frame), as shown in Figures 7D(c) and (d). In one example, as shown in Figure 7D(c), the outline of the dirty area in the second frame can be an irregular shape that can be related to the outline of the third dirty area in each layer; in another example, as shown in Figure 7D(d), the outline of the dirty area in the second frame can be a regular shape (such as a rectangle).

[0230] Optionally, when the drawing application performs step A3-1, it can also determine the first dirty area of ​​the first layer through the above implementation method B1 or B2. For the specific process, please refer to the above description of implementation method B1 or B2, which will not be repeated here.

[0231] After the drawing application determines the first dirty region and the second dirty region, the drawing application may, but is not limited to, perform step A3-3 by means of the following implementation C1 or C2 to determine the first incremental dirty region based on the first dirty region and the second dirty region.

[0232] Implementation method C1: The drawing application can map the first dirty region and the second dirty region onto the same plane, and use the pixel region obtained by merging the two dirty regions as the first incremental dirty region. That is, the area of ​​the first incremental dirty region mapped onto the reference plane is the union of the areas of the first dirty region and the second dirty region mapped onto the reference plane.

[0233] The reference plane is a concept introduced in this application embodiment to facilitate an accurate description of the relationship between the first dirty region, the second dirty region, and the first incremental dirty region. For a description of the reference plane, please refer to the description in embodiment A1 above; it will not be repeated here.

[0234] For example, assuming the first and second dirty regions determined by implementation method B1 or B2 above are as shown in Figure 7E(a), since the first and second dirty regions are located on the same plane (the canvas of the target animation), the rendering application can merge the first and second dirty regions, as shown in Figure 7E(b). Furthermore, the rendering application can determine the first incremental dirty region based on this merged area, as shown in Figures 7E(c) and (d). In one example, as shown in Figure 7E(c), the outline of the first incremental dirty region can be a regular shape (such as a rectangle). In another example, as shown in Figure 7E(d), the outline of the first incremental dirty region can also be an irregular shape, related to the outlines of the first and second dirty regions.

[0235] Implementation method C2: The drawing application can determine the first incremental dirty region based on the first dirty region and the second dirty region through the following steps C2-1 to C2-3:

[0236] Step C2-1: The drawing application can determine the target region based on the first dirty region and the second dirty region, wherein the region of the target region mapped to the reference plane is the union of the regions of the first dirty region and the second dirty region mapped to the reference plane.

[0237] The target region is a pixel area used to compare the pixel values ​​at the same position in the texture of the first frame and the texture of the second frame. This target region can also be called the comparison region, the matching region, etc.

[0238] This reference plane is a concept introduced in this application embodiment to facilitate an accurate description of the relationship between the first dirty area, the second dirty area, and the target area. For a detailed explanation of the reference plane, please refer to the description in Embodiment A1 above; it will not be repeated here.

[0239] Continuing with the example of the first and second dirty areas shown in Figure 7E(a), since the first and second dirty areas are located on the same plane (the canvas of the target animation), as shown in Figure 7F(a), the drawing application can merge the first and second dirty areas, as shown in Figure 7F(b). Furthermore, the drawing application can determine the target area based on this merged area, as shown in Figures 7F(c) and (d). In one example, as shown in Figure 7F(c), the outline of the target area can be a regular shape (such as a rectangle). In another example, as shown in Figure 7F(d), the outline of the target area can also be an irregular shape, related to the outlines of the first and second dirty areas.

[0240] Step C2-2: The drawing application can compare pixels at the same position in the third and fourth textures to determine at least one second pixel. The third texture is obtained based on the image content of the target area in the first layer, and the fourth texture is obtained based on the image content of the target area in the second layer; at least one second pixel has a different pixel value in the third and fourth textures.

[0241] Optionally, in the first design, the rendering application can obtain the intra-frame composite texture (i.e., the first texture) of the first frame based on the first layer, and the intra-frame composite texture (i.e., the second texture) of the second frame based on the second layer. For example, when there are multiple first layers, the rendering application can composite these multiple first layers to obtain the first texture; or when there is only one first layer, the rendering application can use the first layer as the first texture. Similarly, when there are multiple second layers, the rendering application can composite these multiple second layers to obtain the second texture; or when there is only one second layer, the rendering application can use the second layer as the second texture. Then, the rendering application can determine a third texture from the first texture based on the target region; and determine a fourth texture from the second texture based on the target region. The third texture can include image content located within the target region in the first texture, and the fourth texture can include image content located within the target region in the second texture.

[0242] In the second design, the rendering application can obtain a third texture based on the image content of the target area in the first layer, and a fourth texture based on the image content of the target area in the second layer. For example, when there are multiple first layers, the rendering application can composite the image content of the target area in each of the multiple first layers to obtain a third texture; or when there is only one first layer, the rendering application can use the image content of the target area in the first layer as the third texture. Similarly, when there are multiple second layers, the rendering application can composite the image content of the target area in each of the multiple second layers to obtain a fourth texture; or when there is only one second layer, the rendering application can use the image content of the target area in the second layer as the fourth texture.

[0243] Optionally, similar to step A2-1 in implementation A2, in a scenario where the software structure of the animation rendering service of an electronic device is as shown in Figure 5, the rendering application can compare pixels in two textures through the rendering pipeline when executing step C2-2. For example, the rendering application can input a third texture and a fourth texture into the fragment shader in the rendering pipeline, so that the fragment shader can traverse the two textures pixel by pixel, compare pixels located at the same position in the two textures, and obtain at least one second pixel (i.e., a dirty pixel) with different pixel values ​​in the two textures.

[0244] Step C2-3: The drawing application can determine the first incremental dirty region based on at least one second pixel point.

[0245] Optionally, similar to step A2-2 in embodiment A2, in step C2-3, the drawing application can directly use the at least one second pixel as the first incremental dirty region; or the drawing application can determine a pixel region with the smallest area containing the at least one second pixel based on the location of the at least one second pixel, and use that pixel region as the first incremental dirty region; or the drawing application can use at least one tile block containing the at least one second pixel as the first incremental dirty region. In summary, the embodiments of this application do not limit the method of determining the first incremental dirty region based on the at least one second pixel.

[0246] For example, suppose a rendering application determines a target region as shown in Figure 7F(c) based on the first and second dirty regions shown in Figure 7F(a), the target region having a rectangular outline. The rendering application can then determine a third and a fourth texture based on the target region, as shown in Figure 7G(a); subsequently, the rendering application can compare the third and fourth textures using a fragment shader to determine at least one second pixel with different pixel values ​​in the two textures, wherein the at least one second pixel is all the pixels within the two solid-line boxes in Figure 7G(b).

[0247] In one example, the drawing application may use the at least one second pixel as the first incremental dirty region, as shown in (c) of Figure 7G.

[0248] In another example, the drawing application can determine the pixel region with the smallest area containing the at least one second pixel based on the location of the at least one second pixel, and use the pixel region as the first incremental dirty region, as shown in (d) of Figure 7G.

[0249] In another instance, in a scenario where the target animation is atlased across the entire canvas texture, each full canvas texture can be divided into multiple equally sized slices (using tile blocks as an example below). In this scenario, the rendering application can determine the tile block where the at least one second pixel is located, and use these tile blocks as the first incremental dirty region, as shown in (e) of Figure 7G. Alternatively, the target region determined by the rendering application is in units of tiles, meaning the target region contains multiple target tiles. Based on this, when comparing the third texture with the third texture, the rendering application can traverse each of these multiple target tiles, comparing the texture data of the third texture and the currently traversed tile in the third texture through the fragment shader to determine whether the pixel values ​​of the pixels at the same position in the two sets of texture data from the two textures are the same. If the pixel value of at least one pixel in the two sets of texture data is different, the rendering application will treat it as a dirty tile in the first incremental dirty region. If the pixel values ​​of all pixels in the two sets of texture data are the same, the rendering application will not treat it as a dirty tile in the first incremental dirty region. Finally, the rendering application can determine at least one dirty tile as the first incremental dirty region from the multiple target tiles in the target region, as shown in (e) of Figure 7G.

[0250] With the above implementation method C1, the rendering application can calculate the first incremental dirty region based on the first dirty region and the second dirty region with less computation. However, since the pixel values ​​of the pixels in the first dirty region and the second dirty region are not compared, the first incremental dirty region determined by implementation method C1 may contain pixels with the same pixel value in both dirty regions (i.e., pixels that do not need to be composited and rendered in the second frame relative to the image content of the first frame, which are redundant pixels). Subsequently, in the compositing and rendering stage of the second frame, the rendering application still needs to composite and render these redundant pixels, which cannot minimize the resource waste of the rendering application.

[0251] Compared to implementation C1, implementation C2 allows the rendering application to determine the first incremental dirty region by comparing the pixel values ​​at the same positions in the third and fourth textures. This ensures that the first incremental dirty region contains few or no pixels with the same pixel values ​​in both textures. Consequently, during the compositing rendering stage of the second frame, the rendering application can reduce the compositing rendering processing of redundant pixels, further reducing resource waste. Furthermore, compared to implementation A2, where the rendering application needs to compare the pixel values ​​of all pixels in the entire canvas of the first and second frames, implementation C2 only requires comparing the pixel values ​​of pixels within the target area in the entire canvas of the first and second frames. This further reduces the computational load and resource waste of the rendering application.

[0252] Through the above embodiments A1 to A3, the drawing application can determine the first incremental dirty region corresponding to the second frame in various ways. Optionally, the drawing application can determine the first incremental dirty region corresponding to the second frame through embodiment A1 when there is a correlation between the image content of the two frames (i.e., the image content of the later frame is modified based on the image content of the previous frame) and the drawing application records the drawing operations performed on the second layer. When the drawing application does not record the drawing operations on the second layer or the user does not perform any drawing operations on the second layer, or when there is no correlation between the image content of the two frames, the drawing application can determine the first incremental dirty region corresponding to the second frame through embodiment A2 or embodiment A3.

[0253] It should also be noted that the embodiments of this application do not limit the form of the first incremental dirty region. For example, the outline of the first incremental dirty region can be a set regular shape (such as a rectangle), as shown in (d) of Figure 7B, (d) of Figure 7C, (c) of Figure 7E, and (d) of Figure 7G; or it can be an irregular shape, as shown in (d) of Figure 7E. As another example, the first incremental dirty region can be composed of multiple dispersed incremental dirty regions, as shown in (c) of Figure 7C and (c) of Figure 7G; or it can be composed of multiple dirty pixels, as shown in (b) of Figure 8F; or it can be composed of multiple slices (or tile blocks) of uniform size, as shown in (e) of Figure 7C and (e) of Figure 7G.

[0254] Optionally, depending on the granularity, in this embodiment, the first incremental dirty region can be stored or parsed using, but is not limited to, the following three storage formats (or description methods): bounding box, slice (or tile block), or pixel. Depending on the storage format of the first incremental dirty region, the rendering application can use the corresponding compositing rendering method to execute step 603 to obtain the second frame, as detailed in embodiments D1 to D3 below. The second frame obtained by the rendering application through step 603 can also be referred to as the resulting texture of the second frame, i.e., the image used to be displayed on the screen.

[0255] Implementation method D1: The storage method of the first incremental dirty region is a bounding box, that is, the first incremental dirty region is a first bounding box that surrounds the area of ​​change of the second image content relative to the first image content. The first bounding box can also be called the incremental dirty region bounding box, or the bounding box incremental dirty region. Based on this, the rendering application can obtain the second frame through the following implementation method E1 or E2 to realize bounding box-level inter-frame incremental compositing rendering.

[0256] Implementation method E1: When the second layer is a single layer, the drawing application can update the image content within the first bounding box in the first frame based on the image content within the first bounding box in the second layer, thus obtaining the second frame.

[0257] Implementation method E2: When there are multiple second layers, the drawing application can composite and render the image content within the first bounding box of these multiple second layers to obtain a composite result; based on the composite result, update the image content within the first bounding box of the first frame to obtain the second frame.

[0258] Optionally, if the rendering application does not create other layers shared by multiple animation frames, such as foreground frames, background frames, or background color layers, for the target animation, in implementation E1, the rendering application can update the image content within the first bounding box of the second layer to the first bounding box of the first frame, thereby obtaining the second frame. In implementation E2, the rendering application can perform composite rendering processing on the image content within the first bounding boxes of the multiple second layers to obtain a composite result; and update the first bounding box of the first frame with the composite result to obtain the second frame. For example, in implementation E2, the compositing process performed by the rendering application on multiple second layers can be as shown in Figure 8A: layers 1 to n in the figure are multiple second layers; the rendering application can sequentially composite the image content within the first bounding boxes of all layers from layer 1 to layer n in a bottom-to-top order, and update the composite result to the result texture of the first frame to obtain the result texture of the second frame. As shown in the figure, the size of the pixel region where the pixel value is updated in the intermediate result and the result texture of the second frame obtained by the drawing application in the above compositing process is the same as the size of the first bounding box (the size of the inter-frame incremental dirty region (first incremental dirty region) of the second frame). Therefore, these textures can be called inter-frame incremental compositing textures, and the result texture can also be called inter-frame incremental compositing result textures.

[0259] Optionally, if the rendering application creates other layers shared by multiple animation frames, such as foreground frames, background frames, or background color layers, for the target animation, in implementation E1, the rendering application can composite and render the image content within the first bounding box of the second layer with the image content within the first bounding box of at least one other layer to obtain a composite result; and then update the first bounding box of the first frame with the composite result to obtain the second frame. In implementation E2, the rendering application can composite and render the image content within the first bounding box of the multiple second layers with the image content within the first bounding box of at least one other layer to obtain a composite result; then update the first bounding box of the first frame with the composite result to obtain the second frame. For example, in the current case, the compositing rendering process of the rendering application can be as shown in Figure 8B: Layers 1 to n in the figure are multiple second layers; in executing step 603 (compositing rendering process for the second frame), the rendering application can sequentially composite the image content within the first bounding box of all layers from layer 1 to layer n in a bottom-up order to obtain an intra-frame composite result; then the rendering application sequentially composites the intra-frame composite result with the image content within the first bounding box of the background frame, foreground frame, and background color layer, and updates the composite result to the result texture of the first frame to obtain the result texture of the second frame. As shown in the figure, the size of the pixel area where the pixel value is updated in the intermediate result and the result texture obtained in the compositing rendering process of the second frame is the same as the size of the first bounding box. Therefore, these textures can be called inter-frame incremental compositing textures, and the result texture can also be called inter-frame incremental compositing result textures.

[0260] Furthermore, if the rendering application has already performed intra-frame pre-compositing on the second layer to obtain the second texture before executing S603 (for example, the rendering application generates the second texture during the process of determining the first incremental dirty region), the rendering application can directly perform composite rendering based on the image content within the first bounding box of the second texture and the image content within the first bounding box of at least one other layer to obtain the composite result, thereby avoiding the waste of resources caused by performing intra-frame compositing on multiple second layers again. For example, in the current case, the compositing rendering process of the rendering application can be as shown in Figure 8C: Layers 1 to n in the figure are multiple second layers; during the intra-frame pre-compositing process, the rendering application can sequentially composite the image content in all layers from layer 1 to layer n in a bottom-up order to obtain the intra-frame composite result (i.e., the second texture); in executing step 603 (compositing rendering process of the second frame), the rendering application can sequentially composite the second texture, the background frame, the foreground frame, and the image content within the first bounding box of the background color layer, and update the composite result to the result texture of the first frame to obtain the result texture of the second frame. As shown in the figure, in the intermediate results and the second texture obtained during the intra-frame pre-compositing process, the size of the pixel region where the pixel value is updated is the same as the canvas size of the animation frame. Therefore, these textures can be called full-canvas composite textures. In the intermediate results and the result texture obtained during the compositing and rendering process of the second frame, the size of the pixel region where the pixel value is updated is the same as the size of the first bounding box. Therefore, these textures can be called inter-frame incremental composite textures, and the result texture can also be called inter-frame incremental composite result textures.

[0261] In this embodiment D1, the outline of the bounding box can be a rectangular border. When determining the first incremental dirty region, the drawing application can determine the position information of all pixels within that region. The position information of each pixel is typically represented by coordinates (x, y) in a two-dimensional coordinate system, where the two-dimensional coordinate system can be the canvas coordinate system of the target animation, with the x-axis representing the length of the canvas and the y-axis representing the width of the canvas. Therefore, the drawing application can find the maximum x-value (max_x) and minimum x-value (min_x) on the x-axis, and the maximum y-value (max_y) and minimum y-value (min_y) on the y-axis, among the coordinates of all pixels within the first incremental dirty region. In this embodiment, the above four extreme values ​​can be used to define the first bounding box. For example, min_x corresponds to the left boundary of the first bounding box, max_x corresponds to the right boundary of the first bounding box, min_y corresponds to the upper boundary of the first bounding box, and max_y corresponds to the lower boundary of the first bounding box. For example, the coordinates of the top left pixel of the first bounding box are (min_x, min_y), the coordinates of the bottom left pixel are (min_x, max_y), the coordinates of the top right pixel are (max_x, min_y), and the coordinates of the bottom right pixel are (max_x, max_y).

[0262] The drawing application can generate the position information of the first bounding box based on the four extreme values ​​mentioned above. The first incremental dirty region information, used to indicate the first incremental dirty region, may include the position information of the first bounding box. For example, the position information of the first bounding box can be the four extreme values ​​of the first bounding box, or the coordinates of the four vertices of the first bounding box.

[0263] For example, in the scene shown in Figure 8D(a), where the first image content in the first layer corresponding to the first frame and the second image content in the second layer corresponding to the second frame are respectively, the rendering application can determine the first incremental dirty region according to any one of the above embodiments A1 to A3, and determine the position information of the first bounding box through the coordinate values ​​of the pixels in the first incremental dirty region, as shown in Figure 8D(b). Optionally, before the rendering application composites and renders the second frame, the first bounding box can be divided into two triangles, and the vertex coordinate sequence of the two triangles can be determined. The distribution of the triangles in the first bounding box is shown in Figure 8D(c). The rendering application can determine two sets of vertex coordinate sequences, namely {(min_x,max_y),(min_x,min_y),(max_x,min_y)} and {(max_x,min_y),(max_x,max_y),(min_x,max_y)}. During the compositing rendering of the second frame, the rendering application can pass the two sets of vertex coordinate sequences to the vertex shader of the rendering pipeline through the vertex buffer. This allows the vertex shader to calculate or transform the position or attributes of the vertices in the two triangular regions. Then, the fragment shader of the rendering application performs pixel-by-pixel compositing rendering transformations on the two triangular regions based on the vertex processing results obtained from the vertex shader, thereby completing the compositing rendering of the second frame and obtaining the resulting texture for display.

[0264] Implementation D2: In a scene where the entire canvas texture of the target animation is atlased, the layers and related textures of each animation frame are atlased and divided into multiple equally sized slices (i.e., tiles). In this case, the storage and computation units of the first incremental dirty region can be slices; that is, the first incremental dirty region can include target slices from multiple slices. The number of these target slices can be at least one, and any target slice can also be called a dirty slice. Based on this, the rendering application can obtain the second frame through the following implementation method F1 or F2, achieving slice-level (or tile-level) inter-frame incremental compositing rendering.

[0265] Implementation method F1: When the second layer is a single layer, the drawing application can update the image content of the target slice in the first frame based on the image content within the target slice in the second layer, thus obtaining the second frame.

[0266] Implementation method F2: When there are multiple second layers, the drawing application can composite and render the image content within the target slices in these multiple second layers to obtain a composite result; based on the composite result, update the image content within the target slices in the first frame to obtain the second frame.

[0267] The specific execution process of the above implementation methods F1 and F2 can be referred to the relevant descriptions of implementation methods E1 and E2 in the above implementation method D1, which will not be repeated here.

[0268] Because tiles have a finer granularity and greater flexibility compared to bounding boxes, they can more accurately describe the first incremental dirty region, thereby reducing the number of redundant pixels in the first incremental dirty region. In this embodiment, when storing the layers created for each animation frame of the target animation and the various textures for each animation frame, the rendering application atlases them and stores them using tiles as the basic unit. In addition, during the determination of the inter-frame incremental dirty region, the rendering application can perform calculations or processing using tiles as the basic unit. For example, in embodiments A1 to A3 described above, the rendering application can perform calculations using tiles as the basic unit.

[0269] Since the slices are rectangular, similar to implementation D1 above, before the rendering application composites and renders the second frame, each target slice in the first incremental dirty region can be divided into two triangles, and the vertex coordinate sequence of the two triangles in each target slice can be determined. The rendering application can then pass the two sets of vertex coordinate sequences corresponding to each target slice to the vertex shader of the rendering pipeline through the vertex buffer, so that the vertex shader can perform calculations or transformations on the vertices of the two triangular regions in each target slice. Afterwards, the fragment shader of the rendering application can perform pixel-by-pixel compositing and rendering transformations on the two triangular regions in each target slice based on the vertex processing results obtained from the vertex shader, thereby completing the compositing and rendering of the second frame and obtaining the resulting texture of the second frame for display.

[0270] For example, the rendering application can atlase the layers and associated textures of each animation frame and divide them into multiple uniformly sized tile blocks, as shown in the various layers and the canvas of the target animation in Figure 8E. In the scene shown in Figure 8E(a), where the first image content in the first layer corresponding to the first frame and the second image content in the second layer corresponding to the second frame are as shown, the rendering application can determine the eight first tile blocks in the first incremental dirty region according to any of the above embodiments A1 to A3, as shown in Figure 8E(b). Optionally, before the rendering application composites and renders the second frame, the rendering application can determine the vertex coordinates of each first tile block. For example, as shown in Figure 8E(c), the vertex coordinates of tile block 1 in the first incremental dirty region are {(min_x1,min_y1),(max_x1,min_y1),(max_x1,max_y1),(min_x1,max_y1)}, and the vertex coordinates of tile block 2 are {(min_x2,min_y2),(max_x2,min_y2),(max_x2,max_y2),(min_x2,max_y2)}. Referring to Figure 8E(c), the rendering application can divide each first tile block in the first incremental dirty region into two triangles and determine the vertex coordinate sequence of the two triangles divided by each first tile block. During the composite rendering of the second frame, the rendering application can pass the two sets of vertex coordinate sequences corresponding to each first tile block to the vertex shader of the rendering pipeline through the vertex buffer for processing. The specific processing procedure can be referred to the above description and will not be repeated here.

[0271] Typically, the granularity or size of slices in a canvas can be controlled by adjusting their side lengths. Smaller slices have smaller areas, allowing for more flexible and precise representation of the inter-frame incremental dirty region. Larger slices have larger areas, resulting in coarser granularity but potentially reducing the number of slices contained within the inter-frame incremental dirty region. Clearly, with a fixed area, smaller slice sizes mean more slices within the region, increasing the number of triangle vertex coordinate sequences the rendering application needs to calculate, thus increasing CPU time and computational resources. The vertex shader also needs to process more vertex coordinate sequences, impacting rendering pipeline efficiency and increasing GPU computational resources and latency. Conversely, larger slice sizes result in fewer slices within the inter-frame incremental dirty region, reducing the number of vertex coordinate sequences the rendering application needs to calculate and process, thereby reducing CPU and GPU time and computational resources. In conclusion, a balanced slice size is needed to reduce CPU and GPU latency. In this embodiment, the size of the slice can be set by the user, set in the program of the drawing application, or determined by the drawing application based on factors such as the canvas size and brush settings of the target animation being drawn. This application does not limit this.

[0272] Implementation method D3: The storage and calculation unit of the first incremental dirty region is a pixel, that is, the first incremental dirty region includes at least one dirty pixel. Based on this, the rendering application can obtain the second frame through the following implementation method G1 or G2 to achieve pixel-level inter-frame incremental composite rendering.

[0273] Implementation method G1: When the second layer is a single layer, the drawing application can update the pixel value of the at least one dirty pixel in the first frame based on the pixel value of the at least one dirty pixel in the second layer to obtain the second frame.

[0274] Implementation method G2: When there are multiple second layers, the drawing application can composite and render at least one dirty pixel in the multiple second layers to obtain a composite result; update the pixel value of at least one dirty pixel in the first frame according to the composite result to obtain the second frame.

[0275] The specific execution process of the above implementation methods G1 and G2 can be referred to the relevant descriptions of implementation methods E1 and E2 in the above implementation method D1, and will not be repeated here.

[0276] In this embodiment, the rendering application can perform pixel-by-pixel composite rendering transformation and other computational processing on at least one dirty pixel through the fragment shader in the rendering pipeline, thereby completing the composite rendering process of the second frame and obtaining the resulting texture of the second frame for display.

[0277] For example, in the scene shown in Figure 8F(a) where the first image content in the first layer corresponding to the first frame and the second image content in the second layer corresponding to the second frame are respectively, the rendering application can determine 7 dirty pixels in the first incremental dirty region according to any of the above embodiments A1 to A3, as shown in Figure 8F(b). Then, the rendering application can perform pixel-by-pixel composite rendering transformation and other calculations on these 7 dirty pixels through the fragment shader in the rendering pipeline, thereby completing the composite rendering process for the second frame.

[0278] Step 604: Draw the application to play the second frame.

[0279] In step 604, the rendering application can send the resulting texture of the second frame obtained through the inter-frame incremental compositing scheme in S603 to the display.

[0280] Unlike traditional full-canvas compositing schemes where the rendering application displays the full-canvas composite result, in this embodiment, the rendering application displays the inter-frame incremental compositing result texture. Through the scheme provided in this embodiment, the rendering application can achieve inter-frame incremental compositing rendering, avoiding compositing rendering processing of all canvas pixels and reducing resource waste for the rendering application.

[0281] However, when the inter-frame incremental dirty region is very large, the number of pixels that the rendering application needs to composite and render is still large. If the rendering application composites and renders the frame in real time during the playback time of each frame, the problem of long single-frame compositing time may still occur, which may lead to stuttering during animation playback. Based on this, this application also provides an adaptive pre-compositing scheme. Continuing to use the second frame as an example, this scheme is described in one embodiment of this application. During the playback of the target animation, the rendering application can determine the first incremental dirty region corresponding to the second frame before the playback time of the second frame; then determine the area ratio of the first incremental dirty region; if the area ratio of the first incremental dirty region is greater than or equal to a set first threshold, the rendering application can pre-composite and render the second frame using a full canvas compositing scheme before the playback time of the second frame to obtain the result texture of the second frame, and display the result texture of the second frame during the playback time of the second frame to achieve playback of the second frame; if the area ratio of the first incremental dirty region is less than the first threshold, the rendering application can composite and render the second frame in real time using the inter-frame incremental compositing scheme provided in this application during the playback time of the second frame, and display the result texture of the second frame in real time to achieve playback of the second frame.

[0282] The area ratio of the first incremental dirty region is the ratio of its area to the area of ​​the target animation's canvas. Optionally, the drawing application can determine the area of ​​the first incremental dirty region and the area of ​​the target animation's canvas by the number of pixels. For example, if the first incremental dirty region contains 'a' pixels and the target animation's canvas contains 'b' pixels, then the area ratio of the first incremental dirty region is a / b.

[0283] Optionally, the rendering application can pre-synthesize the second frame based on the CPU or GPU's operating conditions without affecting the playback effect of the target animation. For example, the rendering application can pre-synthesize the second frame when there are idle computing resources on the CPU and GPU.

[0284] In one implementation, the rendering application can perform the aforementioned adaptive pre-composition method for every frame of the target animation, excluding the first m frames. Here, m is a positive integer, such as 1, 2, 3, etc.

[0285] In one implementation, the rendering application may disregard the area ratio of the inter-frame incremental dirty region corresponding to the animation frame and perform a pre-compositing scheme on all animation frames in the target animation except for the first m frames.

[0286] As described above in Implementation D2, in a scene where the rendering application atlases the entire canvas texture of the target animation into multiple slices, the storage and computation unit of the first incremental dirty region determined by the rendering application can be a slice. As described in Implementation D2 regarding the impact of slice data in the inter-frame incremental dirty region on the CPU and GPU, when the number of slices in the inter-frame incremental dirty region is large, the vertex shader in the rendering application and rendering pipeline needs to process a large number of triangle vertex coordinate sequences, thereby increasing the computing resources and computation time of the CPU and GPU. This can lead to long single-frame compositing times and cause stuttering during animation playback. Clearly, compared to the traditional full-canvas compositing scheme, the slice-level inter-frame incremental compositing scheme provided in this application may not only fail to improve performance but may also result in performance degradation. To address this issue, this application also provides an Atlas multi-slice adaptive compositing scheme, as shown in Figure 8G. The main principle of this scheme is as follows: After obtaining the inter-frame incremental dirty regions marked by slices, the rendering application judges the area ratio of these regions. When the area ratio of the inter-frame incremental dirty regions exceeds a set threshold, the rendering application directly adopts the full canvas compositing scheme; when the area ratio of the inter-frame incremental dirty regions does not exceed the set threshold, the rendering application continues to use the slice-level inter-frame incremental compositing scheme. In this way, the rendering application can use an appropriate compositing scheme during the animation playback stage, ensuring the smoothest frame-by-frame animation playback scene.

[0287] The description of the area ratio of the inter-frame incremental dirty region can be found in the above implementation. It should be noted that, in addition to calculating the area ratio of the inter-frame incremental dirty region based on the number of pixels, this implementation can also calculate the area ratio of the inter-frame incremental dirty region based on the number of slices.

[0288] Based on the above scheme, this application also provides another Atlas multi-slice adaptive compositing scheme, that is, after the rendering application obtains the inter-frame incremental dirty region marked by slices, it judges the total number of slices contained in the inter-frame incremental dirty region; when the total number exceeds a set threshold, the rendering application directly adopts the full-screen compositing scheme; when the total data does not exceed the set threshold, the rendering application continues to adopt the above-mentioned slice-level inter-frame incremental compositing scheme.

[0289] Continuing with the second frame as an example, the drawing application can execute step 603 only if it determines that the area ratio of the first incremental dirty region is less than the first threshold, or the number of target slices contained in the first incremental dirty region is less than the set number threshold.

[0290] It should be noted that the rendering application can simultaneously use the above-mentioned adaptive pre-compositing scheme and the atlas multi-slice adaptive compositing scheme. That is, when the area ratio of the first incremental dirty region corresponding to the second frame is less than the first threshold, the rendering application can use the slice-level inter-frame incremental compositing scheme to compose and render the second frame in real time; when the area ratio of the first incremental dirty region corresponding to the second frame is greater than or equal to the first threshold, the application can use the full canvas compositing scheme to pre-compose and render the second frame.

[0291] It should also be noted that this embodiment of the application uses the example of a rendering application obtaining the second frame through an inter-frame incremental compositing scheme. During the playback of the target animation, the rendering application can use this inter-frame incremental compositing scheme to composite and render every frame of the target animation except for the first m frames. The rendering application can use a full canvas compositing scheme to composite the first m frames.

[0292] In summary, this application provides an animation playback method. In this method, a rendering application can obtain and play a second frame based on the image content within a first incremental dirty region in the layers of the first and second frames. The first incremental dirty region indicates the area where the image content in the second frame's layer differs from that in the first frame's layer, i.e., the area containing dirty pixels that need to be re-composited and rendered across the entire canvas. This method allows the rendering application to obtain the second frame based on the areas where its image content differs from the previous frame, thus avoiding full-canvas pixel-level compositing and rendering of the second frame. This method provides an inter-frame incremental compositing scheme, reducing resource waste in the rendering application, decreasing single-frame processing time during animation playback, thereby improving the animation playback frame rate, preventing stuttering during playback, ensuring animation playback quality, and enhancing the user's creative experience.

[0293] Based on the methods provided in the above embodiments, this application also provides an animation playback method, which can provide a tile-level inter-frame incremental compositing scheme. The following description refers to the flowchart shown in Figure 9A.

[0294] Step 901: In the drawing stage of the target animation scene using frame-by-frame animation technology, the drawing application can draw the image content in layers created for each animation frame. For a detailed description of step 601 in the embodiment shown in Figure 6, it will not be repeated here.

[0295] The following steps are explained using the target animation frame in the target animation as an example.

[0296] Step 902: The drawing application can determine the inter-frame incremental dirty region corresponding to the target animation frame during the drawing phase (such as after the target animation frame is drawn) or during the playback phase of the target animation, and mark the inter-frame incremental dirty region by using a tile block.

[0297] Optionally, in this embodiment, the rendering application can determine the inter-frame incremental dirty region corresponding to the target animation frame through the implementation method of determining the first incremental dirty region shown in Figure 6 above, as described above. The following is an exemplary description with reference to the accompanying drawings.

[0298] In this embodiment, to save texture storage space, the drawing application can atlase each full canvas texture, switching it to multiple tile blocks. For example, Figure 9B illustrates the canvas of the target animation: the full canvas size is set to W*H, and each tile block is a square block of tileSize*tileSize. Then, in the width direction of the canvas, the number of tile blocks is Cols = W / tileSize, and in the height direction, the number of tile blocks is Rows = H / tileSize. The entire canvas contains Cols*Rows tile blocks. Here, tileSize is the side length of the tile block. A tile block (i,j) represents a tile block in the canvas, i.e., the i-th tile block in the width direction and the j-th tile block in the height direction, where i∈[0,Cols-1, j∈[0,Rows-1]. The four vertices of tile (i,j) are (i×tileSize,j×tileSize), ((i+1)×tileSize,j×tileSize), (i×tileSize,(j+1)×tileSize), and ((i+1)×tileSize,(j+1)×tileSize).

[0299] Typically, drawing applications need to establish a two-dimensional dataset to describe and represent the mapping relationship between different tiles and different areas in the canvas.

[0300] Taking the rendering of a target animation frame in a target animation as an example, the rendering application can respond to the user's rendering operation on the target layer and render image content in the target layer; where the target layer is created by the rendering application for the target animation frame. During the rendering process on the target layer, the rendering application can record the dirty regions of the target animation frame to calculate the inter-frame incremental dirty regions corresponding to the target animation frame.

[0301] Continuing with Figure 9B as an example: The drawing application can create a two-dimensional index matrix T of size Cols*Rows to describe whether each tile contains drawing content. Each element in matrix T has a value of 0 or 1. After drawing the target layer, the drawing application can determine whether tile (i,j) contains drawing content. If tile (i,j) contains drawing content, the element T[i][j] at the corresponding position in matrix T has a value of 1; if tile (i,j) does not contain drawing content, the value of T[i][j] in matrix T is 0. The drawing application can use f(x,y) to indicate whether the pixel at coordinate (x,y) in the canvas contains drawing content. When f(x,y) = 0, it means that the pixel value at coordinate (x,y) is 0, and there is no drawing content; when f(x,y) = 0, it means that the pixel value at coordinate (x,y) is not 0, and there is drawing content. Therefore, each element T[i][j] in matrix T can be represented by the following formula:

[0302] The rendering application can generate an index matrix T for the dirty regions of each frame in the target animation. Based on this, given that the rendering application obtains the index matrix T1 corresponding to the previous frame of the target animation frame and the index matrix T2 corresponding to the target animation frame, the union of the two animation frames mapped to the same plane can be obtained by combining the two index matrices, thus further determining the inter-frame incremental dirty region corresponding to the target animation frame. The union of the two index matrices T1 and T2 is T3, where T3 = T1 ∨ T2. Each element T3[i][j] in the index matrix T3 can satisfy the following formula:

[0303] In T3, the tile blocks corresponding to all elements with a value of 1 are the tile blocks in the inter-frame incremental dirty region corresponding to the target animation frame.

[0304] When the rendering application determines the inter-frame incremental dirty region corresponding to the target animation frame using the scheme provided by implementation method C1 in the embodiment shown in Figure 6, the index matrix of the inter-frame incremental dirty region corresponding to the target animation frame is the aforementioned T3. During the playback stage of the target animation, the rendering application can obtain the index matrix T3 of the inter-frame incremental dirty region corresponding to the target animation frame. The rendering application can traverse each element in T3. When the element value of a certain element T3[i][j] is 1, the rendering application can store the vertex coordinate sequence of the two triangles divided into the tile block corresponding to T3[i][j] in the vertex buffer, which will be used as input for the vertex shader stage in the subsequent composite rendering stage of the target animation frame. The four vertices of the tile block (i,j) are (i×tileSize,j×tileSize), ((i+1)×tileSize,j×tileSize), (i×tileSize,(j+1)×tileSize), ((i+1)×tileSize,(j+1)×tileSize), and the coordinate sequences of the two triangle vertices stored in the vertex buffer by the drawing application are {(i×tileSize,(j+1)×tileSize), (i×tileSize,j×tileSize), ((i+1)×tileSize,j×tileSize)} and {((i+1)×tileSize,j×tileSize), ((i+1)×tileSize,(j+1)×tileSize), (i×tileSize,j×tileSize)}.

[0305] Based on the previous step, the rendering application can store the vertex coordinate sequence of the triangles divided by each tile block in the inter-frame incremental dirty region into the vertex buffer.

[0306] Step 903: During the playback phase of the target animation, the rendering application can synthesize and render the resulting texture of the target animation frame using a tile-level inter-frame incremental compositing scheme based on the inter-frame incremental dirty region corresponding to the target animation frame and the layer corresponding to the target animation frame.

[0307] When compositing and rendering the target animation frame, the rendering application can perform vertex processing such as calculations and transformations on the triangle vertices stored in the vertex buffer using the vertex shader. Then, in the fragment shading stage, the rendering application can use the fragment shader to apply appropriate blending algorithms to the pixels in these triangles to complete the layer blending action and obtain the blending result, thus completing the pixel shading of all triangles. After the triangle shading of all tile blocks is completed, the inter-frame incremental dirty region compositing of the target animation frame is completed. The output of the fragment shader is a shared inter-frame incremental compositing result texture, so the compositing result of each pixel in the inter-frame incremental dirty region is directly updated on this texture.

[0308] Step 904: Draw the resulting texture applied to the playback time of the target animation frame to display the target animation frame on the screen.

[0309] The rendering application sends the composite texture of the target animation frame to the display, which means that all pixels in the inter-frame incremental dirty region corresponding to the target animation frame are updated once. The user will then see the composite texture of the target animation frame on the screen, which is connected with the composite texture sent to the screen of the previous frame, thus presenting the animation playback effect.

[0310] Based on the methods provided in the above embodiments, this application also provides an animation playback method, which can provide a bounding box-level inter-frame incremental composition scheme. The following description refers to the flowchart shown in Figure 10.

[0311] Step 1001: In the drawing stage of the target animation scene using frame-by-frame animation technology, the drawing application can draw the image content in layers created for each animation frame. For a detailed description of step 601 in the embodiment shown in Figure 6, please refer to the specific description here, which will not be repeated here.

[0312] Step 1002: The drawing application can determine the inter-frame incremental dirty region corresponding to the target animation frame during the drawing phase (such as after the target animation frame is drawn) or during the playback phase of the target animation, and mark the inter-frame incremental dirty region by using a bounding box.

[0313] Step 1003: During the playback phase of the target animation, the rendering application can use a bounding box-level inter-frame incremental compositing scheme to synthesize and render the resulting texture of the target animation frame based on the inter-frame incremental dirty region corresponding to the target animation frame and the layer corresponding to the target animation frame.

[0314] The specific execution process of steps 1002 and 1003 can be referred to the process of determining the first incremental dirty region in step 603 of the embodiment shown in Figure 6 and the description of implementation method D1, which will not be repeated here.

[0315] Step 1004: Draw the resulting texture applied to the playback time of the target animation frame and display the target animation frame to play the target animation frame on the display screen.

[0316] Based on the methods provided in the above embodiments, this application also provides an animation playback method, which can provide a pixel-level inter-frame incremental synthesis scheme. The following description refers to the flowchart shown in Figure 11.

[0317] Step 1101: In the drawing stage of the target animation scene using frame-by-frame animation technology, the drawing application can draw the image content in layers created for each animation frame. For a detailed description of step 601 in the embodiment shown in Figure 6, it will not be repeated here.

[0318] Step 1102: The drawing application can determine the inter-frame incremental dirty region corresponding to the target animation frame during the drawing phase (such as after the target animation frame is drawn) or during the playback phase of the target animation, and mark the inter-frame incremental dirty region by dirty pixels.

[0319] Optionally, the rendering application can determine the inter-frame incremental dirty region using implementation method A2 in the embodiment shown in FIG6, or implementation method C2 in implementation method A3.

[0320] Step 1103: During the playback phase of the target animation, the rendering application can synthesize and render the resulting texture of the target animation frame using a pixel-level inter-frame incremental compositing scheme based on the inter-frame incremental dirty region corresponding to the target animation frame and the layer corresponding to the target animation frame.

[0321] The specific execution process of step 1103 can be referred to the description of implementation method D3 in step 603 of the embodiment shown in Figure 6, which will not be repeated here.

[0322] Step 1104: Draw the resulting texture applied to the playback time of the target animation frame to display the target animation frame on the screen.

[0323] It should also be noted that each step in the above embodiments can be executed by the corresponding device, or by components such as chips, processors, or chip systems within that device. The embodiments of this application do not limit their execution. The above embodiments are merely illustrative examples of execution by the corresponding device. Furthermore, the specific implementation methods or examples in the above embodiments do not limit the solutions provided by the embodiments of this application.

[0324] It should be noted that in the above embodiments, some steps may be selected for implementation, and the order of the steps in the figures may be adjusted. This application does not limit this. It should be understood that performing some of the steps in the figures, adjusting the order of the steps, or combining them in a specific implementation all fall within the protection scope of this application.

[0325] It is understood that, in order to achieve the functions described in the above embodiments, each device involved in the above embodiments includes a hardware structure and / or software module corresponding to perform each function. Those skilled in the art should readily recognize that, based on the units and method steps of the various examples described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0326] It is understood that the architecture and application scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. As those skilled in the art will know, with the evolution of network architecture and the emergence of new services, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0327] It should be noted that the "steps" in the embodiments of this application are merely illustrative and are intended to better understand one method of presentation used in the embodiments. They do not constitute a substantial limitation on the execution of the solution of this application. For example, the "step" can also be understood as a "feature". Furthermore, the steps do not constitute any limitation on the execution order of the solution of this application. Any changes to the order of steps, or the merging or splitting of steps made on this basis without affecting the overall solution implementation, resulting in a new technical solution, are also within the scope of disclosure of this application.

[0328] Based on the same concept, this application also provides an electronic device. As shown in FIG12, the electronic device includes a display screen 1201, one or more processors 1202, and a memory 1203. Optionally, the electronic device may also include a transceiver 1204. For example, the above-mentioned devices can be connected via one or more communication buses. The one or more computer programs are stored in the memory 1203 and configured to be executed by the one or more processors 1202. The one or more computer programs include instructions that can be used to cause the electronic device to perform various steps of the methods in the above embodiments.

[0329] For example, the aforementioned one or more processors 1202 may specifically be processor 110 in the electronic device 100 shown in FIG. 4. The aforementioned memory 1203 may specifically be internal memory 121 in the electronic device 100 shown in FIG. 4 and / or external memory connected to the electronic device. The transceiver 1204 may be mobile communication module 150 and / or wireless communication module 160 shown in FIG. 4. The aforementioned display screen 1201 may specifically be display screen 194 shown in FIG. 4, and this application embodiment does not impose any limitation on it.

[0330] Based on the same concept, embodiments of this application also provide an electronic device, which includes units for performing the various steps of the methods provided in the above embodiments.

[0331] For details on the specific functions of each module, please refer to the above embodiments; they will not be repeated here.

[0332] Based on the above embodiments, this application also provides a computer program product, which includes instructions; when the computer program is run on a computer, it causes the computer to execute the method provided in the above embodiments.

[0333] Based on the above embodiments, this application also provides a computer-readable storage medium storing computer program instructions, which, when executed by a computer, cause the computer to perform the methods provided in the above embodiments.

[0334] Optionally, the aforementioned computer may include, but is not limited to, electronic devices.

[0335] The storage medium can be any available medium that a computer can access. For example, but not limited to, a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0336] Based on the above embodiments, this application also provides a chip for reading a computer program stored in a memory to implement the method provided in the above embodiments. Optionally, the chip may include a processor and a memory, wherein the processor is coupled to the memory and is used to read the computer program stored in the memory to implement the method provided in the above embodiments.

[0337] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the functions involved in the electronic devices described in the above embodiments. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. This chip system may be composed of chips or may include chips and other discrete components.

[0338] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0339] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0340] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0341] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

Claims

1. An animation playing method applied to a drawing application, characterized in that, The method comprises: drawing first image content in a first layer and drawing second image content in a second layer; playing a first frame; wherein the first frame comprises the first image content in the first layer; obtaining a second frame according to image content in a first incremental dirty region in the second layer and the first frame; wherein the first incremental dirty region is used for indicating a changed region of the second image content relative to the first image content; the second frame comprises the second image content in the second layer; playing the second frame.

2. The method of claim 1, wherein, The method further comprises: determining the first incremental dirty region according to a first drawing operation on the second layer; wherein the first drawing operation is used for drawing the second image content; or determining the first incremental dirty region according to a first dirty region of the first layer and a second dirty region of the second layer; wherein the first dirty region is a region where the first image content is located, and the second dirty region is a region where the second image content is located; or comparing a same-position pixel point in the first layer and the second layer to determine at least one first pixel point; pixel values of the at least one first pixel point are different in the first layer and the second layer; and determining the first incremental dirty region according to the at least one first pixel point.

3. The method of claim 2, wherein, A region of the first incremental dirty region mapped to a reference plane is a union of regions of the first dirty region and the second dirty region mapped to the reference plane; or The determining the first incremental dirty region according to the first dirty region of the first layer and the second dirty region of the second layer comprises: determining a target region according to the first dirty region and the second dirty region; wherein a region of the target region mapped to the reference plane is a union of regions of the first dirty region and the second dirty region mapped to the reference plane; comparing a same-position pixel point in the first layer and the second layer in the target region to determine at least one second pixel point; wherein pixel values of the at least one second pixel point are different in the first layer and the second layer; determining the first incremental dirty region according to the at least one second pixel point.

4. The method according to any one of claims 1 to 3, characterized in that, The first incremental dirty region is a first bounding box enclosing the changed region; The obtaining a second frame according to image content in a first incremental dirty region in the second layer and the second frame comprises: updating image content in the first bounding box in the first frame according to image content in the first bounding box in the second layer to obtain the second frame.

5. The method according to any one of claims 1 to 3, wherein Each layer is divided into multiple slices of equal size, and the first incremental dirty region comprises a target slice in the multiple slices; The obtaining a second frame according to image content in a first incremental dirty region in the second frame and the first frame comprises: updating image content in the target slice in the first frame according to image content in the target slice in the second layer to obtain the second frame.

6. The method according to any one of claims 1 to 3, wherein The first incremental dirty region comprises at least one dirty pixel point; The obtaining a second frame according to image content in a first incremental dirty region in the second image and the first frame comprises: According to a pixel value of the at least one dirty pixel point in the second layer, a pixel value of the at least one dirty pixel point in the first frame is updated to obtain the second frame.

7. The method according to any one of claims 1 to 6, wherein The second frame is obtained according to image content in the first incremental dirty area in the second layer and the first frame. In a case where an area proportion of the first incremental dirty area is less than a first threshold, the second frame is obtained according to image content in the first incremental dirty area in the second layer and the first frame.

8. An animation playback method applied to a drawing application, characterized by, The method comprises: a first image content is drawn in a first layer group, and a second image content is drawn in a second layer group; the first layer group comprises a plurality of first layers; the second layer group comprises a plurality of second layers; a first frame is played; the first frame comprises the first image content in the first layer group; a second frame is obtained according to image content in a first incremental dirty area in the second layer group and the first frame; the first incremental dirty area is used to indicate a changed area of the second image content relative to the first image content; the second frame comprises the second image content in the second layer group; the second frame is played.

9. The method of claim 8, wherein, The method further comprises: the first incremental dirty area is determined according to a first drawing operation of the second layer group; the first drawing operation is used to draw the second image content; or the first incremental dirty area is determined according to a first dirty area of the first layer group and a second dirty area of the second layer group; the first dirty area is a region where the first image content is located, and the second dirty area is a region where the second image content is located; or a first texture and a second texture are compared to determine at least one first pixel point; pixel values of the at least one first pixel point in the first texture and the second texture are different; the first incremental dirty area is determined according to the at least one first pixel point; the first texture is obtained by synthesizing the plurality of first layers in the first layer group, and the second texture is obtained by synthesizing the plurality of second layers in the second layer group.

10. The method of claim 9, wherein, The method further comprises: a dirty area of each second layer in the second layer group is determined; a dirty area of any second layer is a region where image content in the second layer is located; a second dirty area of the second layer group is obtained according to dirty areas of the plurality of second layers; a region where the second dirty area maps to a reference plane is a union of regions where dirty areas of the plurality of second layers map to the reference plane.

11. The method of claim 9 or 10, wherein, a region where the first incremental dirty area maps to a reference plane is a union of regions where the first dirty area and the second dirty area map to the reference plane; or the first incremental dirty area is determined according to the first dirty area of the first layer group and the second dirty area of the second layer group, comprising: a target region is determined according to the first dirty area and the second dirty area; a region where the target region maps to a reference plane is a union of regions where the first dirty area and the second dirty region map to the reference plane; ​ determine at least one second pixel point by comparing pixels at the same position in the third texture and the fourth texture, wherein the third texture is obtained by synthesizing image content in the target region in the plurality of first layers, and the fourth texture is obtained by synthesizing image content in the target region in the plurality of second layers; and the pixel values of the at least one second pixel point are different in the third texture and the fourth texture; determine the first incremental dirty region according to the at least one second pixel point.

12. The method according to any one of claims 8 to 11, wherein, The first incremental dirty region is a first bounding box surrounding the changed region. The second frame is obtained according to the image content in the first incremental dirty region in the second layer group and the first frame, including: synthesizing image content in the first bounding box in the plurality of second layers to obtain a synthesis result; and updating image content in the first bounding box in the first frame according to the synthesis result to obtain the second frame.

13. The method according to any one of claims 8 to 11, wherein, Each layer is divided into a plurality of slices of equal size, and the first incremental dirty region includes a target slice in the plurality of slices. The second frame is obtained according to the image content in the first incremental dirty region in the second layer group and the first frame, including: synthesizing image content in the target slice in the plurality of second layers to obtain a synthesis result; and updating image content in the target slice in the first frame according to the synthesis result to obtain the second frame.

14. The method according to any one of claims 8 to 11, wherein, The first incremental dirty region includes at least one dirty pixel point. The second frame is obtained according to the image content in the first incremental dirty region in the second layer group and the first frame, including: synthesizing the at least one dirty pixel point in the plurality of second layers to obtain a synthesis result; and updating pixel values of the at least one dirty pixel point in the first frame according to the synthesis result to obtain the second frame.

15. The method according to any one of claims 8 to 14, wherein, The second frame is obtained according to the image content in the first incremental dirty region in the second layer group and the first frame, including: in a case where an area ratio of the first incremental dirty region is less than a first threshold, obtaining the second frame according to the image content in the first incremental dirty region in the second layer group and the first frame.

16. An electronic device, comprising: The electronic device includes: a display screen configured to display a user interface; a memory configured to store computer program instructions; and a processor configured to execute the computer program instructions to support the electronic device to implement the method of any one of claims 1-15.

17. A computer-readable storage medium, characterized in that, The computer program instructions are stored on the computer readable storage medium and are executed by the processing circuit to implement the method of any one of claims 1-15.

18. A computer program product comprising instructions, characterized in that, The computer program product, when running on a computer, causes the computer to execute the method of any one of claims 1-15.