Interface generation method and electronic device

Abstract

The application discloses an interface generation method and an electronic device. The interface generation method provided by the application can determine content to be animatedly rendered and perform animation rendering on the content in the process of generating an interface based on a rendering tree, generate an interface with animation effects through one continuous rendering, and does not need to prepare the content in advance and the content after animation rendering before generating the interface based on the rendering tree, thereby reducing the number of required renderings for generating the interface with the animation effects, simplifying the process of generating the interface with the animation effects, and reducing the overhead of the electronic device for generating the interface with the animation effects.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to interface generation methods and electronic devices. Background Technology

[0002] With the development of electronic technology, more and more electronic devices are becoming part of users' daily lives. When users use electronic devices such as smartphones, the animation effects of the application's UI directly affect the user experience.

[0003] For example, when users switch applications or switch to a multitasking window via the navigation bar or full-screen gestures, electronic devices will display a blurred background; or, for example, when a user long-presses the icon of another application on a desktop application, the interface other than that icon will become blurred; this blurring effect allows users to perceive a smooth switching animation, thereby improving the user experience.

[0004] However, it is obvious that the implementation of various animation effects, including blur, involves multiple renderings of the view, which is relatively complex, consumes a lot of computing and memory resources, takes a long time, is prone to causing interface lag, and increases the power consumption and heat generation of electronic devices. Summary of the Invention

[0005] This application provides an interface generation method and an electronic device. The interface generation method defined in this application, when implementing animation effects, including blurring, on controls or arbitrary areas, does not require pre-generating the animated content. Instead, it modifies the rendering nodes and rendering attributes of the rendering tree, and completes the animation rendering of controls and arbitrary areas based on the modified rendering nodes and rendering attributes during the interface generation process, thereby generating an interface with animation effects. The interface generation method provided in this application reduces the number of rendering operations through a single continuous GPU call, eliminating the need to prepare the animation effects content in the interface before the rendering stage begins, thus reducing the power consumption and heat generation of the electronic device.

[0006] In a first aspect, the interface generation method provided in the embodiments of this application includes: a first process determining a target rendering tree, the target rendering tree being used to draw a target interface; during the process of the first process generating the target interface based on the target rendering tree, the first process determining first content based on at least one rendering node in the target rendering tree, and performing animation rendering on the first content based on first animation parameters to obtain the second content, wherein the animation parameters of the at least one rendering node are the first animation parameters; or, during the process of the first process generating the target interface based on the target rendering tree, the first process determining first content based on a first region on a first surface, and performing animation rendering on the first content based on the first animation parameters to obtain the second content, the first process generating the target interface on the first surface based on the target rendering tree, wherein the first animation parameters are determined by the first process; the first process generating the target interface, the target interface including the second content, and the animation rendering being rendering performed in an off-screen buffer.

[0007] In the above embodiments, during the process of generating an interface based on a rendering tree, the electronic device can draw the content that needs to be animated, perform animation rendering on the content to obtain the rendered content, and thus obtain the target interface. First, the process does not need to prepare the rendered content before the rendering stage, reducing the overhead of multiple renderings and thus reducing the overhead of the electronic device.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, before the first process determines the target rendering tree, the method further includes: the first process adding the at least one rendering node to the target rendering tree, the at least one rendering node being configured with the first animation parameter; or, the first process configuring the first animation parameter for the at least one rendering node.

[0009] In the above embodiments, the first process can modify the parameters in the target rendering tree. For example, it can configure animation parameters for the rendering nodes corresponding to the controls that need to be animated, so that the first process can animate the controls that need to be animated during the process of generating the interface based on the rendering tree. Alternatively, for example, the process can insert one or more rendering nodes configured with animation parameters into the rendering tree, where the one or more rendering nodes themselves and their child nodes are the content to be animated, and thus the animation is rendered.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, when the animation parameters of the at least one rendering node indicate a blurred foreground, the first content includes the view corresponding to the at least one rendering node; when the animation parameters of the at least one rendering node indicate a blurred background, the first content includes the content of the area corresponding to the at least one rendering node on the first surface.

[0011] In the above embodiments, when the animation parameters indicate that the foreground is blurred, the content to be animated is the view corresponding to the rendering node with the configured animation parameters; when the animation parameters indicate that the background is blurred, the content to be animated is the content of the area corresponding to the first surface of the rendering node with the configured animation parameters. Only the interface generation method provided in this application can realize the arbitrary selection of the content to be animated based on the value of the animation parameters, thereby achieving various different animation effects.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, before the first process determines the target rendering tree, the method further includes: the first process generating the target rendering tree based on the second rendering tree and the third rendering tree, wherein the second rendering tree is a rendering tree generated by the second process for drawing a frame of the interface of the second process; and the third rendering tree is a rendering tree generated by the third process for drawing a frame of the interface of the third process.

[0013] In the above embodiments, the target rendering tree can be synthesized from the rendering trees of multiple applications, and then a target interface containing multiple application interfaces can be generated through a single GPU call, i.e., a single rendering.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, the at least one rendering node includes rendering nodes in the second rendering tree and rendering nodes in the third rendering tree, and the first content includes part or all of the content of a frame of the second process and part or all of the content of a frame of the third process.

[0015] In the above embodiments, the target rendering includes rendering nodes of different applications, so that some or all of the content in multiple application interfaces can be selected at one time for animation rendering, thereby obtaining the target interface.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the first rendering node in the second rendering tree is configured with the first animation parameter by the second process; the second rendering node in the third rendering tree is configured with the first animation parameter by the third process; the at least one rendering node includes the first rendering node and the second rendering node.

[0017] In the above embodiments, the application can decide which controls need animation effects and configure animation parameters for the rendering nodes corresponding to these controls. The first process will identify these animation parameters, determine the content to be animated, complete the animation rendering, and then generate the interface during the process of generating the interface based on the target rendering tree.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, when the animation parameter indicates background blur or foreground blur, the animation parameter further includes at least one of blur degree, blur precision, and blur radius.

[0019] In the above embodiments, when the animation parameters indicate that the background or foreground is blurred, the blur degree, blur precision, and blur radius can be further configured. These parameters can be configured by the application or by the first process, enabling different types and effects of blur, thereby improving the user experience.

[0020] Secondly, embodiments of this application provide an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors calling the computer instructions to cause the electronic device to execute: a first process determining a target rendering tree, the target rendering tree being used to draw a target interface; in the process of the first process generating the target interface based on the target rendering tree, the first process determining first content based on at least one rendering node in the target rendering tree, and performing animation rendering on the first content based on first animation parameters to obtain the second content, the animation parameters of the at least one rendering node being the first animation parameters; or, in the process of the first process generating the target interface based on the target rendering tree, the first process determining first content based on a first region on a first surface, and performing animation rendering on the first content based on the first animation parameters to obtain the second content, the first process generating the target interface on the first surface based on the target rendering tree, the first animation parameters being determined by the first process; the first process generating the target interface, the target interface including the second content, the animation rendering being rendering performed in an off-screen buffer.

[0021] In the above embodiments, during the process of generating an interface based on a rendering tree, the electronic device can draw the content that needs to be animated, perform animation rendering on the content to obtain the rendered content, and thus obtain the target interface. First, the process does not need to prepare the rendered content before the rendering stage, reducing the overhead of multiple renderings and thus reducing the overhead of the electronic device.

[0022] In conjunction with some embodiments of the second aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the electronic device to perform: the first process adds the at least one rendering node to the target rendering tree, the at least one rendering node being configured with the first animation parameter; or, the first process configures the first animation parameter for the at least one rendering node.

[0023] In the above embodiments, the first process can modify the parameters in the target rendering tree. For example, it can configure animation parameters for the rendering nodes corresponding to the controls that need to be animated, so that the first process can animate the controls that need to be animated during the process of generating the interface based on the rendering tree. Alternatively, for example, the process can insert one or more rendering nodes configured with animation parameters into the rendering tree, where the one or more rendering nodes themselves and their child nodes are the content to be animated, and thus the animation is rendered.

[0024] In conjunction with some embodiments of the second aspect, in some embodiments, when the animation parameters of the at least one rendering node indicate a blurred foreground, the first content includes the view corresponding to the at least one rendering node; when the animation parameters of the at least one rendering node indicate a blurred background, the first content includes the content of the area corresponding to the at least one rendering node on the first surface.

[0025] In the above embodiments, when the animation parameters indicate that the foreground is blurred, the content to be animated is the view corresponding to the rendering node with the configured animation parameters; when the animation parameters indicate that the background is blurred, the content to be animated is the content of the area corresponding to the first surface of the rendering node with the configured animation parameters. Only the interface generation method provided in this application can realize the arbitrary selection of the content to be animated based on the value of the animation parameters, thereby achieving various different animation effects.

[0026] In conjunction with some embodiments of the second aspect, in some embodiments, the one or more processors are further configured to invoke the computer instructions to cause the electronic device to execute: the first process generates the target rendering tree based on the second rendering tree and the third rendering tree, wherein the second rendering tree is a rendering tree generated by the second process for drawing a frame of the interface of the second process; and the third rendering tree is a rendering tree generated by the third process for drawing a frame of the interface of the third process.

[0027] In the above embodiments, the target rendering tree can be synthesized from the rendering trees of multiple applications, and then a target interface containing multiple application interfaces can be generated through a single GPU call, i.e., a single rendering.

[0028] In conjunction with some embodiments of the second aspect, in some embodiments, the at least one rendering node includes rendering nodes in the second rendering tree and rendering nodes in the third rendering tree, and the first content includes part or all of the content of a frame of the second process and part or all of the content of a frame of the third process.

[0029] In the above embodiments, the target rendering includes rendering nodes of different applications, so that some or all of the content in multiple application interfaces can be selected at one time for animation rendering, thereby obtaining the target interface.

[0030] In conjunction with some embodiments of the second aspect, in some embodiments, the first rendering node in the second rendering tree is configured with the first animation parameter by the second process; the second rendering node in the third rendering tree is configured with the first animation parameter by the third process; the at least one rendering node includes the first rendering node and the second rendering node.

[0031] In the above embodiments, the application can decide which controls need animation effects and configure animation parameters for the rendering nodes corresponding to these controls. The first process will identify these animation parameters, determine the content to be animated, complete the animation rendering, and then generate the interface during the process of generating the interface based on the target rendering tree.

[0032] In conjunction with some embodiments of the second aspect, in some embodiments, when the animation parameter indicates background blur or foreground blur, the animation parameter further includes at least one of blur degree, blur precision, and blur radius.

[0033] In the above embodiments, when the animation parameters indicate that the background or foreground is blurred, the blur degree, blur precision, and blur radius can be further configured. These parameters can be configured by the application or by the first process, enabling different types and effects of blur, thereby improving the user experience.

[0034] Thirdly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0035] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0036] Fifthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0037] It is understood that the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer program product provided in the fourth aspect, and the computer storage medium provided in the fifth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0038] Figures 1A-1D This is an exemplary schematic diagram of the animation effect of a single application involved in this application.

[0039] Figure 2 An exemplary schematic diagram showing the interface of a single application provided in this application embodiment.

[0040] Figure 3 This is an exemplary schematic diagram of the animation rendering implementation transition interface provided in the embodiments of this application.

[0041] Figure 4A , Figure 4B This is an exemplary schematic diagram of an interface for cross-window / cross-application animation effects provided in an embodiment of this application.

[0042] Figure 5A , Figure 5B This is another exemplary schematic diagram of an interface with cross-window / cross-application animation effects provided in an embodiment of this application.

[0043] Figure 6A , Figure 6B This is another exemplary schematic diagram of an interface with cross-window / cross-application animation effects provided in an embodiment of this application.

[0044] Figure 7A This is an exemplary schematic diagram illustrating the implementation of cross-window / cross-application animation effects provided in the embodiments of this application.

[0045] Figure 7B for Figure 7A An exemplary schematic diagram of bitmap changes during the method shown.

[0046] Figure 7C This is another exemplary schematic diagram illustrating the implementation of cross-window / cross-application animation effects provided in the embodiments of this application.

[0047] Figure 7D for Figure 7C An exemplary schematic diagram of bitmap changes during the method shown.

[0048] Figure 8 This is an exemplary schematic diagram of the interface generation method provided in the embodiments of this application.

[0049] Figure 9 An exemplary schematic diagram of the rendering tree generated for an embodiment of this application.

[0050] Figure 10 This is an exemplary schematic diagram of a shared memory transfer rendering tree provided in an embodiment of this application.

[0051] Figure 11A, Figure 11B This is an exemplary schematic diagram of the storage structure of the rendering tree in shared memory provided in an embodiment of this application.

[0052] Figure 12 This is an exemplary schematic diagram of the architecture of UniRender provided in the embodiments of this application.

[0053] Figure 13 This is an exemplary schematic diagram of an application providing an embodiment of this application writing a render tree into shared memory.

[0054] Figure 14 Another exemplary schematic diagram illustrating the application provided in this application writing a render tree into shared memory.

[0055] Figure 15 This is an exemplary schematic diagram illustrating the synthesis of a target rendering tree from the rendering tree provided in the embodiments of this application.

[0056] Figure 16 This is another exemplary schematic diagram illustrating the synthesis of a target rendering tree from the rendering tree provided in the embodiments of this application.

[0057] Figure 17 Another exemplary schematic diagram of the process of building and rendering an interface for an electronic device provided in this application embodiment.

[0058] Figure 18A An exemplary schematic diagram illustrating the foreground blurring implementation provided in this application embodiment.

[0059] Figure 18B This is an exemplary schematic diagram illustrating the background blurring implementation provided in the embodiments of this application.

[0060] Figure 19 This is an exemplary schematic diagram of cross-application / cross-window animation rendering provided in an embodiment of this application.

[0061] Figure 20 This is another exemplary schematic diagram of cross-application / cross-window animation rendering provided in the embodiments of this application.

[0062] Figure 21 This is an exemplary schematic diagram of pixel processing for blur rendering provided in an embodiment of this application.

[0063] Figure 22A , Figure 22B This is an exemplary schematic diagram of a blur animation provided in an embodiment of this application.

[0064] Figure 23A , Figure 23B This is a schematic diagram illustrating an example of cross-application / cross-window interface animation effects provided in this application embodiment.

[0065] Figure 24 This is an exemplary schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0066] Figure 25 This is an exemplary schematic diagram of the software structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0067] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0068] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0069] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with a user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0070] To facilitate understanding, the relevant terms and concepts involved in the embodiments of this application will be introduced below. The terminology used in the embodiments section of this invention is only used to explain specific embodiments of the invention and is not intended to limit the invention.

[0071] To enhance user experience, electronic devices can display transitional interfaces in response to user interactions, such as when switching pages or applications, to improve UI smoothness and thus enhance the user experience. These transitional interfaces are typically achieved through a combination of animation effects, including blurring (Gaussian blur, radial blur, lens blur, etc.), sharpening, distortion, stylization (such as embossing, oil painting effects, 3D effects, etc.), lens flare, and adding noise. These animation effects require rendering by the electronic device.

[0072] (1) First, the interface and implementation of the animation effect in several cases involved in the embodiments of this application are described below by way of example.

[0073] (1.1) Animation effects in the interface of a single application

[0074] Figures 1A-1D This is an exemplary schematic diagram of the animation effect of a single application involved in this application.

[0075] Users can interact with electronic devices by clicking, long-pressing, etc., while using the application, thereby causing the application to change the displayed content or switch pages.

[0076] For example, while using a music application, a user can switch between playing songs by clicking control 1A01, such as switching from song 1 to song 2. Figure 1A As shown, before the user clicks control 1A01, the music application interface displayed on the electronic device includes multiple controls, the song title "Song 1," and a photo of the artist, etc. Figure 1D As shown, after the user clicks control 1A01, the electronic device plays the next song, and the music application interface displayed on the electronic device includes multiple controls, the song name "Song 2", and the artist's photo, etc.

[0077] The application interface displayed on the electronic device is directly from... Figure 1A The content changes shown are as follows Figure 1D The content shown is correct, but this direct interface switching method is very abrupt and results in a poor user experience.

[0078] To improve user experience, using transitional interfaces when the application's interface changes is more in line with human visual characteristics. For example, transitional interfaces can be like... Figure 1B and Figure 1C The content shown is shown. The transition interface is the interface after the animation rendering.

[0079] For the first animation transition method, i.e. Figure 1A , Figure 1B , Figure 1DAs shown, in response to the user clicking control 1A01, the electronic device blurs the screen except for control 1A01, as shown. Figure 1B As shown; then, as Figure 1D As shown, the interface of the music application displayed on the electronic device includes multiple controls, the song title "Song 2", and a photo of the artist.

[0080] Alternatively, for the second animation transition method, such as Figure 1A , Figure 1B , Figure 1C As shown, in response to a user clicking control 1A01, the electronic device zooms in on control 1A01 and changes the color of control 1A01, as... Figure 1C As shown; then, as Figure 1D As shown, the interface of the music application displayed on the electronic device includes multiple controls, the song title "Song 2", and a photo of the artist.

[0081] Clearly, with the first animation transition method, since the content on the interface to be changed is switched through a blurring animation effect, users do not experience any abrupt transitions, improving the continuity of the interface transition and thus enhancing the user experience. Similarly, with the second animation transition method, in response to the user clicking control 1A01, the electronic device changes the size, color, shape, and other attributes of control 1A01 to indicate to the user that the electronic device has received the user's click operation, thereby improving the user experience.

[0082] (1.2) Implementation of animation effects in a single application

[0083] With hardware acceleration enabled, the animation effects implemented in a single application are as follows: Figure 2 and Figure 3 As shown.

[0084] Figure 2 An exemplary schematic diagram showing the interface of a single application provided in this application embodiment.

[0085] like Figure 2 As shown, the generation of a single application's interface can include three steps:

[0086] ① Disable the view hierarchy, and the main thread (UI thread) traverses the application's views, determines and saves the drawing operations for each view, and records the view and the drawing operations (drawoperationstruct, DrawOP) involved in the view into the displaylist of the render node in the render tree.

[0087] The rendering node also includes rendering properties, and its drawing instruction list contains entries for the drawing instruction lists of its child rendering nodes. Rendering properties determine the position, transparency, and other attributes of the drawn content on the surface, while the drawing instruction list determines the drawn content, such as rectangles, lines, and circles.

[0088] Among them, the view is the basic element that constitutes the application interface, and one control on the interface can correspond to one or more views.

[0089] The drawing operation, as a data structure, is used to draw graphics, such as lines, widths, rectangles, and text. At the rendering node, drawing operations are translated into API calls to image processing libraries, such as OpenGL interface calls. For example, `DrawLineOp` is a data structure containing drawing data such as line length and width.

[0090] The drawing instruction list can be a buffer that records all drawing operations or identifiers of all drawing operations included in a single frame of the application's interface. When the application has multiple windows or is displayed on different display areas, multiple render trees need to be generated independently, resulting in multiple drawing instruction lists corresponding to different windows and display areas.

[0091] In this embodiment, the display area can be a screen or a virtual display. A virtual screen can be the area used by the electronic device to display content on the screen during screen recording.

[0092] The render tree, generated by the UI thread, is a data structure used to generate the application's interface. It can include multiple render nodes, each containing render attributes and a list of drawing instructions. The render tree records all the information needed to generate a single frame of the application's interface.

[0093] ②The UI thread passes / synchronizes the render tree to the render thread. The render tree resides in the stack of the application's corresponding process and may not be contiguous in physical memory.

[0094] ③ The Render thread first acquires a Hardware Canvas and performs drawing operations in the render tree on that Hardware Canvas. This Hardware Canvas resides on a surface held by the application, which carries bitmap or other data in other formats used to store image information. This surface is provided to the application by the SurfaceFlinger.

[0095] ① can be considered as the construction phase, which is mainly responsible for determining the size, position, transparency and other attributes of each view in the application, and recording the drawing operations and encapsulating them into a drawing instruction list.

[0096] For example, the `drawLine` method in a view can be encapsulated into a `DrawLineOp` during the construction process. This `DrawLineOp` contains drawing data such as the length and width of the line, and can also contain the corresponding interface calls of the underlying graphics processing library's `drawLine` method.

[0097] Similarly, ③ can be considered as the rendering stage, which is mainly responsible for performing drawing operations in the rendering tree on the surface to generate bitmaps. In this process, the Render thread calls the underlying graphics processing library, such as OpenGL, and then calls the GPU to complete the rendering to generate bitmaps.

[0098] The UI thread is the main thread of the application, and the Render thread is responsible for rendering. A bitmap, as a class, can hold image information, such as pixel position and pixel value.

[0099] The bitmap generated during the rendering phase is the bitmap used to present the transition interface, such as... Figure 1B As shown.

[0100] SurfaceFlinger is a system-level service used to manage and consume currently available surfaces that hold bitmaps generated by the application's render thread. SurfaceFlinger can also be used to composite multiple surfaces as layers.

[0101] The building phase requires CPU computing resources, while the rendering phase requires GPU resources.

[0102] It is worth noting that if hardware acceleration is not enabled, i.e., software rendering, all operations in the construction and rendering stages are completed through the UI thread, and there is no need to generate a rendering tree. After traversing the application's views and performing the view drawing operations, an anonymous shared memory is requested from SurfaceFlinger. On this memory, rendering is performed directly by calling the interfaces provided by the underlying graphics library, such as the Skia library, to generate a bitmap or other format of data for storing image information.

[0103] for Figure 1C The transition interface shown responds to the user's click on control 1A01. The application can modify the rendering properties and drawing operations of the rendering node corresponding to control 1A01 in the rendering tree, so that when the Render thread executes the drawing operations in the drawing instruction list, it can draw an enlarged and color-changed control 1A01 on the surface.

[0104] It's worth noting that while changing the color, size, and transparency of controls within the application's interface can generate a transitional interface by modifying the rendering properties of the corresponding rendering node, it's clear that the application cannot arbitrarily change the color, size, and transparency of any defined area.

[0105] for Figure 1B The transition interface shown needs to be implemented through animation rendering. The specific process is as follows: Figure 3 As shown.

[0106] Figure 3 This is an exemplary schematic diagram of the animation rendering implementation transition interface provided in the embodiments of this application.

[0107] like Figure 3 As shown:

[0108] ① First, the application's UI thread draws the background bitmap. The background bitmap does not include control 1A01.

[0109] Secondly, the application's UI thread performs animation rendering on the background bitmap, processing it pixel by pixel to achieve animation effects such as blurring. The UI thread can utilize the CPU for animation rendering.

[0110] ③ Furthermore, the application's UI thread adds drawing operations or rendering nodes when generating the render tree. These drawing operations, such as `drawBitmap`, can import the animated background bitmap as the background into the bitmap generated during the rendering phase; alternatively, the rendering node corresponds to the animated background bitmap, and during the rendering phase, this node will cause the Render thread to import the animated background bitmap as the background onto the surface.

[0111] ④ Finally, when the application's Render thread generates bitmaps based on the render tree, it generates something like... Figure 1B The transition interface shown.

[0112] Clearly, implementing in-application animation rendering requires the UI thread to prepare the bitmap for the animation rendering before the rendering phase begins. This occupies the UI thread with time-consuming animation rendering operations, easily causing application lag. Furthermore, the UI thread calls on the CPU for animation rendering, but the CPU is less efficient at performing image processing operations, further exacerbating application lag.

[0113] (1.3) Interface with cross-window / cross-application animation effects

[0114] Figure 4A , Figure 4B This is an exemplary schematic diagram of an interface for cross-window / cross-application animation effects provided in an embodiment of this application.

[0115] like Figure 4A As shown, the interface displayed on the electronic device includes the interface of the launcher application and the interface of the wallpaper application. The launcher application interface includes icon controls for several applications, such as the icon control 4A01 for the reading application.

[0116] In response to user interactions with the icon control 4A01, such as long-pressing, the electronic device displays the following interface: a blurred wallpaper application interface and a partially blurred desktop application interface, specifically as follows: Figure 4B As shown. Among them, the icons of the reading application (4A01) and the control bar (4A02) in the desktop application interface are not blurry.

[0117] Figure 5A , Figure 5B This is another exemplary schematic diagram of an interface with cross-window / cross-application animation effects provided in an embodiment of this application.

[0118] like Figure 5A As shown, the electronic device displays the interface of the gallery application. Responding to user interactions, such as full-screen gestures (e.g., swiping up from the bottom to the center of the screen) or tapping the multitasking controls in the bottom navigation bar, the electronic device displays a multitasking interface as shown... Figure 5B As shown.

[0119] like Figure 5B As shown, the multitasking interface includes a gallery interface and a blurred wallpaper interface.

[0120] Figure 6A , Figure 6B This is another exemplary schematic diagram of an interface with cross-window / cross-application animation effects provided in an embodiment of this application.

[0121] In split-screen scenarios, such as Figure 6A As shown, the upper half of the screen of the electronic device displays the interface of a desktop application, and the lower half displays the interface of a music application. The music application interface includes control 6A01.

[0122] After the user clicks control 6A01, the interface displayed on the electronic device is as follows: Figure 6B or Figure 6A As shown.

[0123] like Figure 6B As shown, when the user clicks control 6A01, the electronic device displays a blurry desktop application interface and a partially blurry music application interface. However, the interface of control 6A01 within the music application interface is not blurry.

[0124] Clearly, using animation effects such as blurring or other methods to present the application interface to the user on electronic devices helps improve the user experience.

[0125] It is worth noting that the interface for animation effects of multiple windows in a single application is similar to that of cross-application interfaces, and will not be elaborated on here.

[0126] (1.4) Implementation of cross-window / cross-application animation effects

[0127] Unlike the implementation of animation effects in a single application, the implementation of animation effects in multiple applications involves the process of compositing layers from different applications.

[0128] The following is combined with Figure 7A The content shown exemplifies the implementation of animation effects in multiple applications:

[0129] Figure 7A This is an exemplary schematic diagram illustrating the implementation of cross-window / cross-application animation effects provided in the embodiments of this application.

[0130] like Figure 7A As shown, the implementation of animation effects in multiple applications can include:

[0131] ① The UI thread of application 1 copies the interface of application 2. For example, application 1 obtains the interface of application 2 through the screenshot interface, denoted as bitmap 1. The screenshot interface triggers application 2 to rebuild and render bitmap 1. Bitmap 1 can reside in the heap memory of application 1. Alternatively, the screenshot interface can capture the interface of application 2 from the previous frame. ② The UI thread of application 1 performs animation rendering on bitmap 1, generating bitmap 1. Figure 2Animation rendering can be accomplished by utilizing CPU computing resources. ③ The UI thread of application 1 software draws the interface of application 1, which only includes the parts that need to be animated, denoted as bits. Figure 3 ④ UI thread alignment Figure 3 Perform animation rendering to generate bitmap 4. ⑤ Overlay bitmaps on the UI thread of application 1. Figure 2 ⑥ Using the content of bitmap 4, generate bitmap 5 and insert it as an animation layer into the application's UI view structure. ⑦ During the bitmap generation process based on the render tree, the Render thread of application 1 imports bitmap 5 as the background and generates bitmap 6. ⑧ The UI thread of application 2 generates the render tree. ⑨ The Render thread of the application generates bitmap 7 based on the render tree.

[0132] The SurfaceFlinger / Hardware Composer (HWC) module obtains bitmap 6 generated by application 1 and bitmap 7 generated by application 2 through the surface, and performs layer compositing to obtain a bitmap for display (also known as a layer for display).

[0133] Obviously, Figure 7A The animation effect shown requires the UI thread to prepare the bitmap for the animation rendering before the rendering phase begins. This causes the UI thread to be occupied by the time-consuming animation rendering operation, which can easily lead to application lag. In addition, the UI thread calls the CPU for animation rendering, but the CPU is less efficient at performing image processing operations, further exacerbating application lag.

[0134] In addition, capturing the interface of other applications via the screenshot interface may have a delay. If the interface of application 2 is constantly changing (for example, bitmap 1 and bitmap 7 have different contents), application 1 may not be able to obtain the latest interface of application 2. Furthermore, the screenshot interface calls the GPU, increasing the number of GPU calls and computational overhead.

[0135] Below, with Figure 4B Taking the interface shown as an example, combined with Figure 7B Exemplary introduction Figure 7A Contents of median plots 1 through 6.

[0136] Figure 7B for Figure 7A An exemplary schematic diagram of bitmap changes during the method shown.

[0137] exist Figure 4B In the interface shown, Figure 7A Application 1 is the desktop application, and application 2 is the wallpaper application.

[0138] like Figure 7B As shown, bitmap 1 is the interface of the wallpaper application obtained by the desktop application through the screenshot interface; bitmap Figure 2 This is the bitmap generated after blurring and rendering bitmap 1; bit Figure 3 This is the interface of the desktop application excluding control 4A01; ​​bitmap 4 is the alignment. Figure 3 Bitmap 5 is a bitmap generated after blur rendering; Bitmap 6 is a bitmap generated by overlaying the pixels of Bitmap 4 and Bitmap 5; Bitmap 7 is the interface of the desktop application; Bitmap 8 is the interface of the wallpaper application.

[0139] Blur rendering is a type of animation rendering.

[0140] It is worth noting that, Figure 7B In bitmaps 1 through 7, white pixels (such as those outside icons on a desktop application) represent pixels not drawn by the application. These pixels can be empty or have a default value, such as black pixels (255, 255, 255). These pixels may or may not be included in the animation rendering calculations.

[0141] Figure 7C This is another exemplary schematic diagram illustrating the implementation of cross-window / cross-application animation effects provided in the embodiments of this application.

[0142] and Figure 7A The content shown is different. Figure 7C The animation effects shown mainly rely on SurfaceFlinger to render the content that needs to be animated off-screen.

[0143] like Figure 7C As shown, the implementation of animation effects in multiple applications includes:

[0144] ① Application 1 and Application 2 are independently constructed and rendered to generate Bitmap 1 of Application 1 and Bitmap 1 of Application 2, respectively. Figure 2 Among them, bitmap 1 and bitmap 2 are generated. Figure 2 It requires separate calls to the CPU and GPU.

[0145] ② The Render thread of application 1 passes bitmap 1 to SurfaceFlinger, and SurfaceFlinger will then... Figure 2 The content that needs to be animated is copied to off-screen buffer 1. The content in off-screen buffer 1 is called bit depth. Figure 3 The off-screen buffer can be requested and built in advance, or it can be requested and built in response to an animation rendering request.

[0146] ③ The Render thread of application 2 will... Figure 2Pass it to SurfaceFlinger, and set the bit Figure 2 The content that needs to be animated is copied to off-screen buffer 2. The content in off-screen buffer 2 is called bitmap 4.

[0147] ④ SurfaceFlinger blurs the contents of off-screen buffer 1 and off-screen buffer 2 respectively, generating bitmap 5 and bitmap 6 respectively. Generating bitmap 5 and bitmap 6 requires separate calls to the GPU.

[0148] ⑤ SurfaceFlinger for bitmap 1, bit Figure 2 Bitmaps 5 and 6 are composited. This layer compositing process calls the GPU once. The composited content is bitmap 7, which is then copied to the on-screen buffer.

[0149] The off-screen buffer is a non-on-screen buffer allocated by SurfaceFlinger to store intermediate states of layers / bitmaps. The on-screen buffer is a buffer within the GPU that can be accessed by the display subsystem, carries bitmaps, and can be directly sent to the display.

[0150] Clearly, SurfaceFlinger uses off-screen rendering to achieve cross-window / cross-application animation effects when compositing bitmaps from multiple applications as layers. However, SurfaceFlinger's off-screen rendering for animation rendering incurs significant computational and memory overhead on electronic devices due to context switching caused by the graphics processing libraries called for animation rendering across different off-screen buffers.

[0151] Secondly, when implementing cross-window / cross-application animation effects, SurfaceFlinger needs to generate multiple off-screen buffers to store different intermediate results, and it also needs to composite multiple layers carrying intermediate results through layer compositing, resulting in significant computational and memory overhead. Furthermore, the intermediate results in different off-screen buffers contain a large amount of duplicate content, wasting the computing and memory resources of electronic devices.

[0152] The following is based on Figure 4B Taking the interface shown as an example, combined with Figure 7D Exemplary introduction Figure 7C The contents included in median maps 1 to 7.

[0153] Figure 7D for Figure 7C An exemplary schematic diagram of bitmap changes during the method shown.

[0154] like Figure 7DAs shown, bitmap 1 is a bitmap generated during the construction and rendering of a desktop application; bitmap Figure 2 Bitmaps generated for building and rendering wallpaper applications; bitmaps Figure 3 The content to be blurred in bitmap 1; bitmap 4 is the bitmap. Figure 2 The content that needs to be blurred; bitmap 5 is for alignment. Figure 3 The bitmap after blurring; Bitmap 6 is the bitmap after blurring Bitmap 4; Bitmap 7 is the bitmap after blurring Bitmap 1 and Bitmap 4. Figure 2 Bitmap 5 and Bitmap 6 are combined to form a bitmap for display.

[0155] Obviously, as mentioned above Figure 7A As shown, for an application to achieve cross-window / cross-application animation rendering, it needs to complete the animation rendering through software drawing. However, because the hardware architecture of the CPU is not as suitable for graphics computing as the GPU, software drawing has low energy efficiency, slow animation rendering speed, and is prone to causing lag.

[0156] Obviously, as mentioned above Figure 7C As shown, to achieve cross-window / cross-application animation rendering, SurfaceFlinger needs to allocate multiple off-screen buffers and complete the animation rendering through off-screen rendering. This process involves a large number of repetitive operations, resulting in slow animation rendering and potential stuttering.

[0157] (2) Next, the interface generation method provided in the embodiments of this application will be introduced below.

[0158] The interface generation method provided in this application first obtains the rendering trees of one or more applications through the UniRender process, and then reassembles the one or more rendering trees into a target rendering tree; second, the UniRender process renders based on the target rendering tree to obtain a bitmap carrying image information of one or more applications. The UniRender process can perform animation rendering during the bitmap rendering process.

[0159] Obviously, firstly, the interface generation method provided in this application merges the target rendering trees of one or more applications into a single target rendering tree, and can generate a bitmap carrying image information of one or more applications in a single rendering, avoiding the need to obtain bitmaps of other applications through screenshot calls, reducing the number of GPU calls, and thus reducing power consumption.

[0160] Secondly, the interface generation method provided in this application reduces the number of off-screen buffers that need to be allocated by performing animation rendering during the bitmap generation process, thereby reducing computational and memory overhead.

[0161] Furthermore, since the rendered bitmap is a bitmap that carries image information from one or more applications, layer composition can be achieved without SurfaceFlinger / HWC, reducing at least one GPU call and further saving computational and memory overhead.

[0162] The interface generation method provided in the embodiments of this application is described below by way of example.

[0163] Figure 8 This is an exemplary schematic diagram of the interface generation method provided in the embodiments of this application.

[0164] like Figure 8 As shown, the interface generation method provided in this application includes:

[0165] S801: After receiving the vertical signal, construct and generate the rendering tree.

[0166] When an application needs to update its interface, it can request a vertical synchronization signal (Vsync-APP) from the UniRender process. Upon receiving the Vsync signal, the application executes the `measure()`, `layout()`, and `draw()` methods on the UI thread. During the `draw()` method, the UI thread iterates through the application's views, determines the drawing instructions required for each view's rendering, and continuously records / inserts these instructions into the drawing instruction list of the corresponding rendering node for that view.

[0167] The application's interface consists of multiple nested views. The drawing instruction list corresponding to the root view (DecorView) includes the drawing instruction list entries for the root view's child views. Therefore, the nesting relationship between drawing instruction lists is the same as the nesting relationship between views, and consequently, the nesting relationship of rendering nodes is the same as the nesting relationship of views. For definitions of the rendering tree and rendering nodes, please refer to the above text. Figure 3 The corresponding text description.

[0168] Figure 9 This is an exemplary schematic diagram of constructing a rendering tree provided in an embodiment of this application.

[0169] After performing measure(), layout(), and draw(), the application's UI thread can obtain the parent-child structure of multiple views of the interface to be updated. During the process of traversing the views, it determines the content to be displayed on each view and the interface calls required to generate that content, such as drawCircle, drawLine, etc.

[0170] The application encapsulates the drawing interface calls such as drawCircle and drawLine into corresponding DrawOp such as DrawCircleOp and DrawLineOp, and saves them in the drawing instruction list.

[0171] like Figure 9 As shown, the application's interface view structure is as follows: root view, and the root view's subviews are view 1 and view 2. Figure 2 ,See Figure 2 The subview is the view Figure 3 Among them, vision Figure 2 If the shape includes rectangles and circles, then it is considered... Figure 2 The drawing operations in the drawing instruction list of the corresponding rendering node 2 include: drawing rectangles, drawing circles, etc.

[0172] When the application's UI thread executes the `draw()` method, it starts from the root view and traverses all views according to their parent-child relationships. It then determines the drawing operations for each view and encapsulates these operations into `DrawOp` objects. After generating the drawing instruction list, the application's UI thread further encapsulates this list into a render tree.

[0173] A rendering node in the rendering tree includes a list of drawing instructions and rendering properties. The rendering properties determine the position, size, transparency, and other attributes of the view to be rendered by the rendering node on the surface. The list of drawing instructions determines the content of the view to be rendered by the rendering node, such as lines, rectangles, circles, etc.

[0174] The surface is requested by the application, and the application determines its size. If SurfaceFlinger exists, the application can request the surface from SurfaceFlinger; if SurfaceFlinger does not exist, it can request it from the UniRender process. SurfaceFlinger may choose not to allocate a surface to the application.

[0175] Optionally, in some embodiments of this application, the UniRender process can determine the screen refresh rate of each display area and then adjust the frequency of the vertical synchronization signal (Vsync-APP) on each display area so that the application displayed on display area 1 generates a render tree at the frequency of the screen refresh rate of display area 1.

[0176] In this embodiment, the display area can be a screen or a virtual display. A virtual screen can be the area used by the electronic device to display content on the screen during screen recording.

[0177] It is worth noting that an application's UI thread can generate multiple render trees, such as in multi-screen, virtual screen, multi-window, and other multi-display scenarios.

[0178] S802: Cross-process transfer of render tree

[0179] After generating the render tree, the application's UI thread passes it to the UniRender process via IPC communication. The corresponding UniRender process needs to receive the render trees passed by different applications and determine the correspondence between the render trees and the applications.

[0180] In this process, multiple foreground applications pass the render tree to the UniRender process. An application is considered a foreground application if it meets any of the following three conditions: the application has a visible activity, the application has a foreground service, or other foreground applications are associated with the application.

[0181] Since different processes do not share memory, data exchange between processes requires inter-process communication (IPC). Specifically, the application can pass the rendering tree to the UniRender process via IPC communication through methods such as Binder, AIDL, shared memory, and Socket, which are not limited here.

[0182] The following example uses shared memory as an example of IPC communication to illustrate how to pass the rendering tree across processes.

[0183] (a) The application writes the render tree to shared memory

[0184] Figure 10 This is an exemplary schematic diagram of a shared memory transfer rendering tree provided in an embodiment of this application.

[0185] like Figure 10 As shown, firstly, the application requests shared memory from the UniRender process each time it starts. After receiving the shared memory request from the application, the UniRender process requests a block of shared memory and a handle to the corresponding shared memory through the Anonymous Shared Memory (Ashmem) subsystem.

[0186] Secondly, after successfully requesting shared memory from the Ashmem subsystem, the UniRender process receives a handle from the Ashmem subsystem for reading and writing physical memory. The UniRender process returns the handle to the application, allowing the application to use it to write the rendering tree into physical memory. The UniRender process can directly read this physical memory from its own process space, and thus directly read the application's rendering tree.

[0187] The shared memory can be a virtual file created in RAM through a temporary file system (tmpfs) and mapped to the user space of different processes.

[0188] Cross-process communication, such as the application requesting shared memory from the UniRender process, the UniRender process requesting shared memory from the Ashmem subsystem, and the UniRender process returning the handle to the obtained shared memory to the application, can be achieved through Binder communication.

[0189] It is understandable that the rendering tree stored on the application process's stack can be passed to the UniRender process through other IPC methods such as shared memory, and this is not a limitation here.

[0190] Optionally, in some embodiments of this application, when the application has multiple layers, that is, when the application generates multiple render trees, the application can request two shared memory blocks from the UniRender process to store data for different layers, that is, to store data for different render trees.

[0191] (b) Data structure for storing the rendering tree in shared memory

[0192] To further improve the efficiency of IPC communication, in this embodiment, the rendering tree is stored in shared memory as a memory tree. The data structure of the rendering tree stored in shared memory is described below as an example:

[0193] The memory tree can consist of multiple data segments, with different segments storing layer information, rendering data, etc. The following example... Figure 11A , Figure 11B Using the content as an example, the structure of a memory tree is illustrated. The offsets of different data segments in shared memory can be fixed or variable and adaptive.

[0194] Figure 11A , Figure 11B This is an exemplary schematic diagram of the storage structure of the rendering tree in shared memory provided in an embodiment of this application.

[0195] like Figure 11A As shown, the storage structure of the rendering tree includes three data segments: the HEAD field, the MAPPING field, and the NODES field.

[0196] The HEAD field includes layerkey and rootid; the MAPPING field includes nodeid and the address corresponding to nodeid; and the NODES field includes currentproperties, stagingproperties, stagingdisplaylist, and currentdisplaylist.

[0197] Here, layerkey is the ID of the entire rendering tree as a layer; rootid is the ID of the root node of the rendering tree; nodeid is the ID of the rendering node other than the root node, and each nodeid corresponds to an address, which is the starting address of the rendering properties (renderproperties / properties) and the display list (displaylist) in the rendering tree node; stagingproperties are the rendering properties written by the application, stagingdisplaylist is the display list written by the application, currentproperties are the rendering properties read by the UniRender process, and currentdisplaylist is the display list read by the UniRender process.

[0198] It is worth noting that if we consider "stagingproperties and stagingdisplaylist" as the first set of data and "currentproperties and currentdisplaylist" as the second set of data, then the data written by the application is the first set of data, and the data written by the application next time is the second set of data, thus implementing a double buffering mechanism; similarly, the data read by the UniRender process is the first set of data, and the data read next time is the second set of data.

[0199] Optionally, in some embodiments of this application, the storage structure of the rendering tree in shared memory can be as follows: Figure 11B As shown, the NODES field can contain only one set of data, namely only the list of rendering properties and drawing instructions.

[0200] The layerkey is used by the UniRender process to obtain the application to be displayed and the layer IDs of the application that need to participate in layer compositing from the WMS before reading layer data from the shared memory via handles. The UniRender process then verifies this layer ID against the layer IDs included in the layerkey from the shared memory.

[0201] The rootid is used as the entry point of the rendering tree, storing the entry points of other rendering nodes. After obtaining the rootid, the UniRender process can read the data of the rendering tree and restore the nested structure of the rendering tree.

[0202] The values ​​of currentproperties, stagingproperties, stagingdisplaylist, and currentdisplaylist are used to: swap the values ​​of currentproperties and stagingproperties after the application has finished writing the drawing instruction list and rendering properties, and swap the values ​​of currentdisplaylist and stagingdisplaylist. The UniRender process reads the rendering properties and drawing instruction list from currentproperties and currentdisplaylist.

[0203] For example, Application 1 is an application displayed in the foreground. Application 1 can have multiple layers. That is, after receiving the vertical synchronization signal (Vsync-APP), Application 1 will generate multiple render trees, such as render tree 1 and render tree 2. The UniRender process determines from the WMS that the render tree corresponding to the layer participating in layer compositing is render tree 1.

[0204] Because the offsets of layerkey and rootid are fixed, the UniRender process can determine the address of the root node. The UniRender process finds the root node's position at the NODES end in the MAPPING segment, reads the rendering instructions for that node, and if there are child nodes, there will be corresponding DrawRenderNode instructions, which store the child node's ID. The hash value is used to find the child node's position in the MAPPING segment. For example, the parent-child relationship of rendering nodes in the rendering tree is stored in the DrawOP operations. For instance, the drawing instruction list for rendering node 2 includes several DrawOP operations and the "Draw RenderNode3" operation, thus the UniRender process can determine that rendering node 3 is a child node of rendering node 2.

[0205] Understandably, the rendering tree in shared memory still maintains the same nesting relationship as the application's views, so the UniRender process can read data starting from the root node and then read the data of the entire rendering tree.

[0206] Specifically, by dividing currentproperties, stagingproperties, stagingdisplaylist, and currentdisplaylist, the security of the data read by the UniRender process and written by the application is ensured, preventing data that is only partially written by the application from being read by the UniRender process as the latest data for rendering the application interface. For details on the security of simultaneous read and write operations on the rendering tree, please refer to the textual description of read and write operations on the rendering tree in shared memory in section (2.3) below; it will not be repeated here.

[0207] Optionally, in some embodiments of this application, the size of the three data segments can be fixed. That is, when the UniRender process requests shared memory from the Ashmem subsystem, the obtained shared memory is of size (a+b+c). Here, the positions from the starting address (physical address) to the starting address + a are for filling the HEAD field; the positions from the starting address + a+1 to the starting address + a+b are for filling the MAPPING field; and the positions from the starting address + a+b+1 to the starting address + a+b+c are for filling the NODES field.

[0208] Understandably, with the three data segments of fixed size, the UniRender process can determine the start of each data segment based on the fixed offset, and then find the Mapping segment. The Mapping segment stores the offset of the rendering node of the rendering tree in the NODES field, and then finds the data of each rendering node.

[0209] Optionally, in some embodiments of this application, when the sizes of the three data segments are fixed, if the rendering node written by the application exceeds the size of b, a second shared memory block is requested from the UniRender process. The second shared memory block can have the same format as the first shared memory block, and the NODES field of the second shared memory block continues to store the drawing instruction list and rendering node attributes of the rendering node. The HEAD field and / or MAPPING field of the second shared memory block can be empty or may not exist. That is, in some embodiments of this application, the second shared memory block only includes the NODES field.

[0210] (c) UniRender reads the rendering tree from shared memory

[0211] (c.1) UniRender Architecture

[0212] Figure 12 This is an exemplary schematic diagram of the architecture of UniRender provided in the embodiments of this application.

[0213] like Figure 12 As shown, the UniRender provided in this application may include four parts, namely NodeManager, LayerManager, DisplayManager, and UniRenderCore.

[0214] NodeManager is the node management module within the UniRender process, responsible for receiving the rendering tree and other data sent by the application. The composition of the target rendering tree can be referred to the textual description in step S903.

[0215] LayerManager is the layer management module within the UniRender process, responsible for synchronizing layer information from the Window Manager Service (WMS), such as layer creation, destruction, and attribute changes. A bitmap is equivalent to one layer.

[0216] Among them, DisplayManager is the display device management module within the UniRender process, which is responsible for synchronizing display device information, such as screen size, from the Display Manager Service (DMS).

[0217] UniRenderCore is the rendering management module within the UniRender process, responsible for: establishing corresponding rendering nodes for each layer; receiving rendering trees corresponding to different applications maintained in NodeManager, removing layer information instructions for the application from LayerManager and inserting them into rendering nodes; merging the rendering trees corresponding to all visible layers of the active display devices maintained in DisplayManager; traversing the merged rendering tree for each display area, and generating bitmaps on the buffer allocated by UniRender, etc.

[0218] (c.2) UniRender reads the rendering tree

[0219] The UniRender process first identifies the applications displayed on each display area from the DMS and WMS; these applications are those participating in layer compositing. The UniRender process can further utilize a whitelist to determine the applications performing layer compositing within the UniRender process. Specifically, the UniRender process uses the WMS to determine the layer ID for each application.

[0220] The DisplayerManager in the UniRender process is responsible for communicating with the DMS, and the LayerManager in the UniRender process is responsible for communicating with the WMS.

[0221] Because the UniRender process holds a handle to the shared memory corresponding to the application, after determining the application participating in layer compositing, the UniRender process can use the handle to determine the shared memory corresponding to that application. The UniRender process then reads the render tree from the shared memory using this handle.

[0222] The NodeManager in the UniRender process is responsible for managing the handles to the shared memory and reading the rendering tree from the shared memory.

[0223] The process by which the UniRender process reads the render tree from shared memory includes:

[0224] First, the UniRender process reads the layer key from the starting address of shared memory and verifies the layer ID. Specifically, the UniRender process compares the layer ID determined by the WMS with the layer ID determined from the layer key. If they match, the UniRender process reads the rendering tree starting from the root node (root ID).

[0225] Secondly, after finding the address of the root node, the UniRender process determines the starting address of the root node in the NODES segment from the address field in the MAPPING segment, and then begins reading the drawing instruction list and rendering attributes of that rendering node. If the root node has child nodes, the entry points for the child nodes, such as the DrawRenderNode instruction, are stored in the root node's drawing instruction list. Since the DrawRenderNode instruction includes the child node's ID, the UniRender process uses a hash operation to find the corresponding node ID in the MAPPING segment, thereby determining the location of the child node's drawing instruction list and rendering attributes in the NODES segment, and then reading these attributes.

[0226] (d) Reading and writing the rendering tree in shared memory

[0227] The rendering tree, located in shared memory, can be read and written by two or more processes. To reduce or avoid read-write conflicts that could lead to errors in the rendering tree data, inter-process synchronization locks can be configured to ensure the safety of reading and writing the rendering tree.

[0228] The following is combined with Figure 13 , Figure 14 The content shown exemplifies the process of reading and writing the rendering tree.

[0229] Figure 13 This is an exemplary schematic diagram of an application providing an embodiment of this application writing a render tree into shared memory.

[0230] Each application maintains at least one lock variable A to prevent the application and the UniRender process from simultaneously reading and writing to shared memory. The UniRender process obtains the status (held or released) of the lock variable held by different applications via IPC communication.

[0231] like Figure 13 As shown, ① First, when the application needs to update the interface, it requests and receives the vertical synchronization signal (Vsync-APP), then obtains the lock variable A of the shared memory corresponding to this application, and calculates and updates the attributes and drawing instruction list of the rendering node starting from the root node.

[0232] ② Secondly, the application writes the updated properties and drawing instruction list of the rendering node into the staging properties and staging displaylist data segments of the NODES segment in shared memory, and adds the changed rendering node ID to the properties_dirty queue and displaylist_dirty queue, which are stored in the shared memory management class singleton on the application side.

[0233] Understandably, the properties_dirty queue and the displaylist_dirty queue mark the rendered nodes that have changed, which can enable differential updates to the rendered tree.

[0234] Optionally, in some embodiments of this application, the properties_dirty queue and displaylist_dirty queue may not be saved, thus enabling a full update of the render tree.

[0235] ③ Next, the application copies the stagingproperties segment of the corresponding rendering node from the properties_dirty queue to the current properties segment. The application swaps the draw_pointer and record_pointer of the corresponding rendering node from the displaylist_dirty queue, that is, copies the staging displaylist segment of the corresponding rendering node from the displaylist_dirty queue to the current displaylist; or the application copies the staging displaylist segment to the current displaylist.

[0236] Understandably, in response to the current vertical synchronization signal (Vsync-APP), the application only changes the data of the rendering node corresponding to displaylist_dirty, compared to the previous vertical synchronization signal (Vsync-APP). The application swaps the draw_pointer and record_pointer of the corresponding rendering node in the displaylist_dirty queue, thus achieving differential updates to the current displaylist.

[0237] Understandably, the application can copy the staging displaylist segment to the current displaylist to achieve a full update, that is, write all the data of the rendering tree generated by the application in response to the vertical synchronization signal (Vsync-APP) directly into shared memory, which is relatively simple to implement.

[0238] Optionally, in some embodiments of this application, without saving the properties_dirty queue and the displaylist_dirty queue, the staging properties segment of all rendering nodes of the application is copied to the current properties segment, and the staging displaylist segment is copied to the current displaylist segment.

[0239] ④ Next, the application transmits the information that lock variable A has been released to the UniRender process via IPC communication.

[0240] ⑤ Again, the UniRender process holds lock variable A.

[0241] ⑥ Finally, corresponding to ③, the UniRender process reads the current displaylist and current properties from shared memory, or reads the stagingdisplaylist segment of the corresponding rendering node from the displaylist_dirty queue and copies it to the current displaylist.

[0242] After reading the data, the UniRender process can release lock variable A and notify the application that lock variable A has been released. Then, when the next vertical synchronization signal (Vsync-APP) arrives, it holds lock variable A and writes the render tree into shared memory. At this point, the roles of the staging data segment and the current data segment are reversed. The UniRender process finally reads the staging displaylist and staging properties segments, implementing a "double buffering" mechanism to ensure the robustness of the generated interface.

[0243] Optionally, in some embodiments of this application, when a lock variable exists between the application and the UniRender process, the NODES field may include only the staging displaylist and staging properties, or only the current displaylist and current properties. In this case, the application and the UniRender process achieve read-write safety through the lock variable, ensuring that the UniRender process reads the correct rendering tree.

[0244] Optionally, in some embodiments of this application, each application may hold more lock variables, for example, each application holds lock variable A and lock variable B. Furthermore, after an application releases lock A, it does not need to wait for the UniRender process to release lock A; after holding lock variable B, it directly writes the render tree data to the shared memory upon receiving the next vertical synchronization signal (Vsync-APP).

[0245] The holding, releasing, and inter-process synchronization of lock variable B are the same as those of lock variable A. For details, please refer to [link / reference needed]. Figure 14 The textual description shown will not be repeated here.

[0246] Figure 14 Another exemplary schematic diagram illustrating the application provided in this application writing a render tree into shared memory.

[0247] like Figure 14As shown, when the application holds a lock variable A, upon receiving the vertical synchronization signal 1 (Vsync-APP), the application acquires lock variable A and writes the generated render tree into shared memory during the holding phase. After the application writes the render tree into shared memory, it releases lock variable A. Once the UniRender process determines that lock variable A has been released by the application, it acquires lock variable A and begins reading the render tree generated by the application in response to the vertical synchronization signal 1 from shared memory.

[0248] While the UniRender process holds lock variable A, the application receives vertical synchronization signal 2 (Vsync-APP) after vertical synchronization signal 1. Since lock variable A is held by UniRender, the application cannot write the rendering tree to shared memory in a timely manner and needs to wait for a period of time until the application determines that the UniRender process releases lock A.

[0249] However, it is quite obvious that in Figure 13 The UniRender process reads the current displaylist and current properties fields, while the application writes to the staging displaylist and stagingproperties fields. Concurrent reading and writing of different fields in shared memory can be achieved by increasing the number of lock variables.

[0250] like Figure 14 As shown, when the application holds two lock variables B, upon receiving the vertical synchronization signal 1 (Vsync-APP), the application acquires lock variable A and writes the generated render tree into shared memory during the holding phase. After the application writes the render tree into shared memory, it releases lock variable A. Once the UniRender process determines that lock variable A has been released by the application, it acquires the lock variable and begins reading the render tree generated by the application in response to the vertical synchronization signal 1 from shared memory.

[0251] While the UniRender process holds lock variable A, the application receives vertical synchronization signal 2 (Vsync-APP) after vertical synchronization signal 1. At this time, it holds lock variable B, and the application can write the rendering tree to shared memory in a timely manner without waiting for a period of time until the application determines that the UniRender process releases lock A.

[0252] Correspondingly, after determining that lock variable B has been released by the application, the UniRender process holds lock variable B and begins to read the rendering tree generated by the application in response to vertical synchronization signal 2 from shared memory.

[0253] It is worth noting that the number of lock variables held by the application can be related to the contents of the NODES field in shared memory, or it can be related to the value of Vsync-offset.

[0254] For example, if we consider `current displaylist` and `current properties` as the first group, and `staging displaylist` and `staging properties` as the second group, then we can configure two synchronization lock variables in the application and the UniRender process. Similarly, if the `NODES` field contains three groups of data, then we can configure three synchronization lock variables in the application and the UniRender process.

[0255] In this system, each lock variable corresponds to a set of data. For example, if lock variable A corresponds to current displaylist and current properties, then the change of lock variable A from the held state to the released state indicates that the application has successfully updated the data of current displaylist and current properties in shared memory, and the data of current displaylist and current properties can be read by the UniRender process. Alternatively, the change of lock variable A from the held state to the released state indicates that the UniRender process has completed reading the data of current displaylist and current properties from shared memory, and the data of current displaylist and current properties can be updated by the application.

[0256] The number of lock variables can be related to the value of Vsync-offset.

[0257] That is, it can be related to the difference between the vertical synchronization signal (Vsync-APP) and the vertical synchronization signal (Vsync-UR), Vsync-offset. If Vsync-offset is large, lock variables do not need to be configured. Without configuring lock variables, the UniRender process reads the render tree from shared memory after receiving the vertical synchronization signal (Vsync-UR). Because Vsync-offset is large, the application has already completely written the render tree into shared memory when the UniRender process reads it.

[0258] S803: Generate the target rendering tree based on the obtained rendering tree.

[0259] The UniRender process can either begin reading the render tree from shared memory after receiving the vertical synchronization signal (Vsync-UR), and start compositing the render tree after acquiring one or more render trees; or it can hold a lock variable, begin reading the render tree from shared memory, and start compositing the render tree upon receiving the vertical synchronization signal (Vsync-UR).

[0260] If the UniRender process receives only one render tree, that render tree is the target render tree; if the UniRender process receives more than one render tree, the multiple render trees are combined to generate the target render tree.

[0261] Figure 15 This is an exemplary schematic diagram illustrating the synthesis of a target rendering tree from the rendering tree provided in the embodiments of this application.

[0262] like Figure 15 As shown, the UniRender process can create a new root rendering node, such as... Figure 15 The new root rendering node is created, and the processed rendering trees of multiple applications are used as child nodes of the root rendering node. That is, the root rendering node can store the parent-child relationship of the processed rendering trees of different applications, which allows the UniRender process to traverse the target rendering tree from the root rendering node to find the processed rendering trees of different applications.

[0263] Figure 16 This is another exemplary schematic diagram illustrating the synthesis of a target rendering tree from the rendering tree provided in the embodiments of this application.

[0264] The UniRender process can first determine the Z-order of layers corresponding to different applications based on the window management service, that is, determine the top and bottom occlusion relationships between application layers. Then, during the generation of the target rendering tree, the rendering nodes corresponding to completely occluded views are deleted, thereby reducing the amount of computation in the bitmap generation process in step S905 and improving the bitmap generation speed.

[0265] For example, such as Figure 16As shown, the rendering tree generated by application 1 corresponds to layer 1, and the rendering tree generated by application 2 corresponds to layer 2. The Z-order of layer 1 is higher than that of layer 2. The rendering tree corresponding to layer 1 consists of root rendering node 1, rendering node 1, rendering node 2, and rendering node 3. Rendering node 1 and rendering node 2 are child rendering nodes of root rendering node 1, and rendering node 3 is a child rendering node of rendering node 2. The rendering tree corresponding to layer 2 consists of root rendering node 2, rendering node 4, rendering node 5, and rendering node 6. Rendering node 4 and rendering node 5 are child rendering nodes of the new root rendering node, and rendering node 6 is a child rendering node of rendering node 5.

[0266] The UniRender process can traverse the rendering child nodes of the rendering tree corresponding to layer 1 and the rendering child nodes of the rendering tree corresponding to layer 2, determine the position of the view corresponding to each rendering node in the surface (as allocated by the UniRender process), and combine the layer Z-order of layer 1 and the Z-order of layer 2 to determine the completely occluded view and the rendering node of the completely occluded view.

[0267] For example, UniRender can determine that the view corresponding to rendering node 6 in the rendering tree of layer 2 is completely obscured, and then delete rendering node 6.

[0268] After deleting the rendering node corresponding to a completely obscured view, the UniRender process can proceed as follows: Figure 16 The content shown merges all the rendered trees after the off-screen rendering instructions are moved forward into the target rendered tree.

[0269] Optionally, in some embodiments of this application, UniRender can also traverse rendering nodes and optimize the parameters of the target rendering tree at the granularity of DrawOP drawing operations. For example, DrawOP drawing operations that do not affect the interface can be deleted, where "does not affect" can mean that the graphics drawn by the DrawOP drawing operations are not displayed on the interface; or, for different applications, the position of DrawOP operations in the drawing nodes can be modified so that the same type of DrawOP drawing operations can be executed together, such as modifying the DrawOP drawing operation of rendering node 2 of application 1 to the drawing instruction list of rendering node 1 of application 2, which is not limited here.

[0270] Optionally, in some embodiments of this application, after step S804 and before step S805, after the UniRender process generates the target rendering tree, it decapsulates the drawing instruction list of the RenderNode to obtain a series of DrawOPs, and then performs DrawOP operations on the target rendering tree as a whole to perform batching and merging. In step S805, a target rendering tree that can generate a bitmap using less computation is obtained.

[0271] It is understandable that by merging multiple render trees and optimizing the merged render tree, over-drawing can be avoided or reduced, thereby increasing the speed of bitmap generation in step S905, saving power consumption of electronic devices, and improving the user experience.

[0272] It is worth noting that the UniRender process can also optimize other parameters of the target render tree during the generation process, which will not be limited here.

[0273] S804: Generates a bitmap and completes animation rendering.

[0274] After generating the target render tree, the UniRender process can execute drawing operations from the drawing instruction list in the render tree. During the process of generating bitmaps on surfaces allocated to multiple applications by the UniRender process, the animation rendering of these bitmaps is completed. These bitmaps carry image information from one or more applications.

[0275] The surface can be the same size as the display area. That is, the surface can be the same size as the screen of the electronic device. For example, in... Figure 6A and Figure 6B In the scenario shown, the surface is the same size as the screen. This surface can be located on the screen buffer.

[0276] Figure 17 This is an exemplary schematic diagram illustrating the animation rendering process during the generation of a bitmap, as provided in an embodiment of this application.

[0277] like Figure 17 As shown, the UniRender process obtains the rendering trees generated by the UI threads of one or more applications through IPC communication, combines one or more rendering trees into a target rendering tree, and then calls the GPU through the underlying graphics library to execute the drawing operation of the drawing instruction list in the target rendering tree to generate a bitmap, and completes the animation rendering in the process of generating the bitmap.

[0278] Below, we will exemplify the methods used by the UniRender process to render animations during the bitmap generation process, focusing on whether the animation crosses windows or applications.

[0279] (a) Animations within a single application

[0280] To facilitate application development, the interface generation method provided in this application allows developers to configure different animation parameters, such as foreground blur, background blur, and other animation parameters, for any view in the application. Correspondingly, during the generation of the rendering tree, the application's UI thread adds these animation parameters to the rendering properties of the rendering node corresponding to that view.

[0281] The UniRender process can identify the animation parameters in the rendering properties of the rendering nodes in the rendering tree corresponding to the view's animation parameters, and execute the corresponding animation rendering operation. Alternatively, the UniRender process can add animation parameters to the rendering properties of any rendering node in the target rendering tree.

[0282] In other words, animation parameters are parameters that application developers can configure on the view, or parameters that the UniRender process adds to the rendering node, used to add animation effects to the selected content on the interface.

[0283] The interface generation method provided in this application also provides developers with a BlurView, whose coverage area is the area where the animation takes effect. The BlurView can inherit from the view class and, in addition to having all the configurable properties of the view, can also have other properties.

[0284] The UniRender process can identify the corresponding rendering node of BlurView in the rendering tree and execute the corresponding animation rendering operation. This animation rendering operation includes: the UniRender process copies the content already drawn on the surface within the coverage area of ​​BlurView to the off-screen buffer for animation rendering, and then copies it back to the surface, or directly copies it to the on-screen buffer.

[0285] ①The UniRender process can allocate a surface for the application's bitmap. Based on the target rendering tree, the UniRender process continuously draws the application's view onto this surface to generate the application's bitmap.

[0286] ② When the UniRender process draws to a rendering node with animation parameters, if the animation parameters are scaling, simple color change, or transparency change, it will modify the drawing operation parameters in the drawing instruction list of the rendering node accordingly, and then execute the drawing operation to achieve animation rendering.

[0287] For example, if the animation parameter of the rendering attribute of the rendering node is magnified by 2, and the drawing instruction list of the rendering node includes DrawCircleOp((x0, y0), 3), then when the UniRender process performs the drawing operation, it will convert DrawCircleOp((x0, y0), 3) to DrawCircleOp((x0, y0), 6) and then execute DrawCircleOp((x0, y0), 6).

[0288] DrawCircleOp((x0, y0), 3) means: draw a circle with a radius of 3 at the center (x0, y0); DrawCircleOp((x0, y0), 6) means: draw a circle with a radius of 6 at the center (x0, y0).

[0289] When the animation parameters are filter effects, such as blur (Gaussian blur, radial blur, lens blur, etc.), sharpening, distortion, stylization (such as emboss effect, oil painting effect, 3D effect, etc.), lens flare, or adding noise, the UniRender process will enable off-screen rendering, copy the content at the rendering node location to the off-screen buffer, perform animation rendering, and then copy it back to the surface.

[0290] ③ Continue drawing to generate a bitmap.

[0291] The following example uses filter effects such as foreground blur and background blur to illustrate the process of UniRender process animation rendering.

[0292] Figure 18A An exemplary schematic diagram illustrating the foreground blurring implementation provided in this application embodiment.

[0293] like Figure 18A As shown, during the process of generating a bitmap based on the render tree, when the UniRender process traverses a render node whose render attributes include animation parameters, such as RenderNode3, it first determines the type of animation parameter, for example, foreground blur. The view corresponding to RenderNode3 is view3.

[0294] After determining that the animation parameters are foreground blur, the UniRender process executes the drawing operations in the drawing instruction list of RenderNode3 in off-screen buffer 3, and then draws view3.

[0295] The UniRender process performs animation rendering on the content in the off-screen buffer, i.e., blur rendering, and the generated content is called view31. Then, the UniRender process copies the content in the off-screen buffer to the surface and continues to traverse other nodes in the render tree.

[0296] The size of the off-screen buffer can be the same as the size of view3, and the UniRender process can determine the size of view3 from the properties of RenderNode3.

[0297] Figure 18B This is an exemplary schematic diagram illustrating the background blurring implementation provided in the embodiments of this application.

[0298] like Figure 18B As shown, during the process of generating a bitmap based on the render tree, when the UniRender process traverses a render node whose render attributes include animation parameters, such as RenderNode3, it first determines the type of animation parameter, for example, whether it is background blur. The view corresponding to RenderNode3 is view3.

[0299] After determining that the animation parameter is background blur, the UniRender process copies the content at position 3 on the surface to the off-screen buffer and performs animation rendering on the content in the off-screen buffer, that is, performs blur rendering.

[0300] Then, the UniRender process copies the contents of the off-screen buffer to the surface and executes the drawing instruction list of RenderNode3 to continue drawing view3 on the surface.

[0301] The UniRender process continues to traverse the other nodes of the render tree.

[0302] Before drawing view3, the UniRender process copies the content at the position of view3 on the surface in the off-screen buffer to be the background of view3.

[0303] Optionally, in some embodiments of this application, if the reference coordinate system of the rendering tree generated by the application is inconsistent with the reference coordinate system of the surface allocated by the UniRender process, the UniRender process will add coordinate transformation instructions to the rendering attributes of the rendering tree after receiving the rendering tree from the application. For example, the UniRender process will convert DrawCircleOp((x0, y0), 3) to DrawCircleOp((x1, y1), 3).

[0304] (b) Animations across applications / windows

[0305] The implementation methods for cross-application / cross-window animations are the same as those for single-application animations, as described in the text above, and will not be repeated here.

[0306] Firstly, the UniRender process allocates a surface for multiple applications and completes the animation rendering of the bitmaps while generating bitmaps for each application within that surface. This surface can reside in the screen buffer.

[0307] Secondly, the UniRender process can insert a surface at any position on the surface, and / or insert a BlurView between any rendering nodes in the target rendering tree; the UniRender process can also recognize the application's BlurView. The area covered by this BlurView is the content to be animated.

[0308] Figure 19 This is an exemplary schematic diagram of cross-application / cross-window animation rendering provided in an embodiment of this application.

[0309] like Figure 19 As shown, cross-application / cross-window animation rendering can include the following three steps:

[0310] ① First, after traversing and executing the list of drawing instructions in the target rendering tree, the UniRender process generates a bitmap in the surface, which carries image information from one or more applications.

[0311] ②Secondly, the UniRender process adds a BlurView to the surface and copies the content at the BlurView's location to the off-screen buffer.

[0312] ③ Finally, the UniRender process animates the content of the off-screen buffer and copies the content of the off-screen buffer back to the surface.

[0313] The overlay of content in the off-screen buffer with the original content on the surface can be considered a layer composition. That is, the content in the off-screen buffer can cover the content on the surface, or the content in the off-screen buffer can be overlaid with the content on the surface in various ways.

[0314] Figure 20 This is another exemplary schematic diagram of cross-application / cross-window animation rendering provided in the embodiments of this application.

[0315] like Figure 20 As shown, cross-application / cross-window animation rendering can include the following three steps:

[0316] ① First, during the process of traversing and executing the drawing instruction list of the target rendering tree, the UniRender process can add rendering nodes corresponding to BlurView at any level of the rendering tree (such as...). Figure 20 (As shown). Alternatively, the UniRender process can identify the rendering node corresponding to the application's BlurView while traversing and executing the list of drawing instructions in the target rendering tree.

[0317] If BlurView is not the root node of the target rendering tree, the UniRender process will first perform drawing operations on the surface to generate a bitmap.

[0318] ②Secondly, the UniRender process draws the child nodes of the BlurView in the off-screen buffer. After drawing all the child nodes of the BlurView, the UniRender process animates the content in the off-screen buffer and copies it back to the surface after the animation is rendered.

[0319] ③ Finally, the UniRender process continues to traverse the target rendering tree to complete the drawing of the bitmap.

[0320] It is worth noting that the position and size of the BlurView specified by the application can be generated based on the coordinate system of the UniRender process's surface, or it can be generated based on other coordinate systems; no limitation is made here.

[0321] Optionally, in some embodiments of this application, BlurView can also be configured with a sigma attribute, which represents different meanings in different animation renderings. For example, in the case of blurred animation rendering, sigma represents the degree of blur, and its value range is integer. When sigma = 0, BlurView is ineffective, that is, no blurring is applied. The definition of the degree of blur can be referred to in (c) Blur Animation Rendering, and will not be repeated here.

[0322] (c) Blur animation rendering

[0323] In this embodiment of the application, as long as the content to be animated has been copied to the off-screen buffer or drawn directly in the off-screen buffer, the content in the off-screen buffer can be animated.

[0324] The following example uses animation as a case study to illustrate the process of blur rendering.

[0325] This application provides multiple levels of blur rendering, wherein the lower the blur level, the closer the value of the blurred pixel is to the value of the pixel before blurring.

[0326] This application provides various levels of blur rendering precision, which, for ease of description, are referred to as high-precision blur rendering, medium-precision blur rendering, and low-precision blur rendering, respectively. Furthermore, this application also provides blur rendering with different blur radii at different levels of blur rendering precision.

[0327] The blur level, blur precision, and blur radius can be configured by the application or the UniRender process. The application can configure these parameters by configuring the properties of the BlurView. These properties are encapsulated in the rendering properties of the corresponding rendering node and then retrieved by the UniRender process. If the application does not configure the blur level, blur precision, blur radius, or other animation parameters for the BlurView, the UniRender process can adaptively configure these parameters.

[0328] This application provides an embodiment where the UniRender process can select different levels of animation rendering precision based on the current GPU load and capabilities. For example, when the GPU load of the electronic device is high, lower precision animation rendering is selected; when the GPU load of the electronic device is low, higher precision animation rendering is selected.

[0329] Figure 21 This is an exemplary schematic diagram of pixel processing for blur rendering provided in an embodiment of this application.

[0330] Blur, as a form of animation, can reset the value of each pixel in an image. The reset value of each pixel is related to the values ​​of the pixels surrounding that pixel before the reset.

[0331] For example, such as Figure 21 As shown, the bitmap before blurring is bitmapA, where bitmapA has a resolution of 8*8, meaning there are 8 pixels per row and 8 pixels per column, for a total of 64 pixels in bitmapA, which are A... 11 A 12 To A 88 .

[0332] After the electronic device performs a blur rendering of bitmapA, it generates bitmapB. BitmapB can have the same resolution as bitmapA, or it can have the same resolution as bitmapA. If the generated bitmapB has a different resolution than bitmapA, downsampling and interpolation can be used to make the resolution of bitmapB the same as bitmapA.

[0333] After the electronic device generates bitmapB, the value of any pixel in bitmapB is related to the values ​​of multiple pixels in bitmapA.

[0334] For example, pixel B in bitmapB ij Its generation process is as follows (1):

[0335]

[0336] In equation (1), k(r(i,j,k,l)) is a coefficient related to the distance r(i,j,k,l), where r(i,j,k,l) ​​represents the distance between (i,j) and (k,l), and A kl For the pixel in row k and column l, B ij Let be the pixel in the i-th row and j-th column.

[0337] Here, k(r(i,j,k,l)) can be related to the configured fuzzy radius. For example, when r(i,j,k,l) ​​> fuzzy radius, k(r(i,j,k,l)) = 0. That is, if A kl With B ij If the distance is greater than the fuzzy radius, then B ij The pixel value will not be affected by A kl The value of the pixel has an impact.

[0338] For example, if the fuzzy radius is 2, then B 45 =avg(A45+0.5*(A35+A44+A46+A55)+0.3*(A34+A36+A54+A56)), where avg() is used to calculate the average.

[0339] Here, k(0) is related to the degree of ambiguity. For example, when k(0) = 1, B 45 =avg(A45+0.5*(A35+A44+A46+A55)+0.3*(A34+A36+A54+A56)), when k(0)=2, B 45 =avg(2*A45+0.5*(A35+A44+A46+A55)+0.3*(A34+A36+A54+A56)). Therefore, the smaller the value of k(0), the greater the degree of blurring.

[0340] Gaussian blurring can be achieved when the value of k(r(i,j,k,l)) has a Gaussian function relationship with the distance.

[0341] Considering that pixels near the edge of the blurred bitmap A, such as A... 18 When rendering with blur, content outside bitmapA is needed. Therefore, when copying the blurred content from the screen buffer to the off-screen buffer, the copied content can be expanded to achieve a better blur effect.

[0342] Figure 22A , Figure 22B This is an exemplary schematic diagram of a blur animation provided in an embodiment of this application.

[0343] like Figure 22A As shown, after determining the BlurView, the electronic device determines the size of BlurView1 based on the blur radius and the BlurView itself, and then copies the content within the defined area of ​​BlurView1 to the off-screen buffer. After blurring the content in the BlurView in the off-screen buffer, the content within the defined area of ​​the blurred BlurView is copied to the on-screen buffer.

[0344] like Figure 22B As shown, after the electronic device obtains the content that needs to be blurred and copies the content to be blurred to the off-screen buffer, it first downsamples the content to be blurred and then blurs the bitmap after downsampling; then, after obtaining the blurred bitmap, it performs interpolation to restore the resolution of the bitmap, and then copies it back to the on-screen buffer.

[0345] For high-precision rendering, downsampling can be performed once or multiple times during the downsampling stage (with a fixed downsampling rate), and calculations can be performed pixel-by-pixel during blur rendering. Finally, a bitmap copied back to the screen buffer is obtained through interpolation.

[0346] For medium-precision rendering, downsampling can be performed once during the downsampling stage (with a fixed downsampling rate), and during blur rendering, when blurring any pixel, only the influence of pixels in the same row and column as that pixel is calculated. For example, for B... 45 Consider only A 44 A 45 A 46 A 35 A 55 The impact of this. Finally, the bitmap copied back to the screen buffer is obtained through interpolation.

[0347] For low-precision rendering, downsampling can be omitted, and when blurring any pixel, only the influence of pixels in the same row and column as that pixel is calculated. Finally, the bitmap copied back to the screen buffer is obtained through interpolation.

[0348] It is worth noting that other parameters such as the number of downsampling operations, whether downsampling is performed, the blur radius during blur rendering, and the number of pixels to be considered during blur rendering can be combined arbitrarily to achieve rendering with different precisions. No limitations are imposed here.

[0349] (3) Again, with Figure 23A , Figure 23B For example, this section introduces how electronic devices generate animations through rendering. Figure 4B The process shown on the interface.

[0350] Figure 23A , Figure 23B This is a schematic diagram illustrating an example of cross-application / cross-window interface animation effects provided in this application embodiment.

[0351] ①After generating the target rendering tree, the UniRender process can traverse the list of drawing instructions in the target rendering tree and draw the generated bitmap 1 on the surface.

[0352] Among them, the target rendering tree is as follows Figure 23B As shown, the root node for target rendering is node 1, and its child nodes are nodes 2, 3, 4, and 5. Node 2 corresponds to the view of the wallpaper application; node 3 determines the layout of the views corresponding to its child nodes; node 4 is the rendering node for BlurView; and node 5 determines the layout of the views corresponding to its child nodes. Node 3 has two child nodes: nodes 6 and 7. Node 6 corresponds to the icon and text for the "Gallery" section of the desktop application; node 7 corresponds to the icons and text for "Dial, Messages, Contacts"; node 8 corresponds to icon control 4A01; ​​and node 9 corresponds to control bar 4A02.

[0353] The UniRender process traverses the target rendering tree in the following order: node 1, node 2, ..., node 9.

[0354] Bitmap 1 may include only the content that needs to be animated in the interfaces of two or more applications, or the complete interfaces of two or more applications.

[0355] exist Figure 23A In the image, bitmap 1 shows the content in the interfaces of the two applications that needs to be animated.

[0356] When the UniRender process traverses to node 4, it determines that node 4 is the rendering node corresponding to BlurView. Then, the UniRender process determines the content that needs to be animated and rendered based on the size and range of BlurView.

[0357] exist Figure 23A In this context, the size and extent of the BlurView cover the entire surface. That is, the content rendered by the animation is bitmap 1.

[0358] ②The UniRender process creates an off-screen buffer and copies the content covered by BlurView to the off-screen buffer; or the content drawn after BlurView is drawn directly in the off-screen buffer.

[0359] exist Figure 23A In the process, the UniRender process copies bitmap 1 to the off-screen buffer, which is simply called bitmap 1. Figure 2 .

[0360] ③The UniRender process performs animated rendering of the content in the off-screen buffer. Animated rendering includes blur rendering.

[0361] exist Figure 23A In the UniRender process, the position is... Figure 2 Perform blur rendering to obtain the position. Figure 3 .

[0362] ④ The UniRender process copies the contents of the off-screen buffer back to the surface and continues traversing the target rendering tree. After traversing the target rendering tree, the final bitmap 4 used for display is obtained.

[0363] exist Figure 23A In the process, the UniRender process will be positioned Figure 3 The image is copied back to the surface and drawing continues, completing the rendering and generating bitmap 4. Bitmap 4 is then passed to the display subsystem by the UniRender process for display.

[0364] It is understood that the interface generation method provided in this application can generate a bitmap carrying image information of multiple applications through a single continuous rendering (one GPU call), avoiding additional calls to the GPU during the layer compositing process.

[0365] It is understood that the interface generation method provided in this application embodiment can reduce the size of the area to be repeatedly drawn by combining multiple rendering trees of the application into a target rendering tree.

[0366] It is understood that the animation parameters of the view and BlurView provided in the embodiments of this application can realize the animation rendering of any control and any area, reducing the workload of application developers.

[0367] It is worth noting that the rendering tree received by the UniRender process is not limited to the RenderNode data structure coupled in a tree structure; it can also be other data structures that include a list of drawing instructions and rendering properties, which are not limited here.

[0368] (4) Finally, the hardware architecture and software architecture of the electronic device provided in the embodiments of this application are introduced.

[0369] Figure 24 This is an exemplary schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0370] The following describes the embodiments using an electronic device as an example. It should be understood that the electronic device may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0371] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0372] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0373] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0374] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0375] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0376] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0377] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device.

[0378] The I2S interface can be used for audio communication. The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. The UART interface is a universal serial data bus used for asynchronous communication.

[0379] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to achieve the electronic device's shooting function. The processor 110 and the display screen 194 communicate via the DSI interface to achieve the electronic device's display function. The GPIO interface can be configured via software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc. The SIM interface can be used to communicate with the SIM card interface 195 to achieve the function of transferring data to or reading data from the SIM card. USB interface 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc.

[0380] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0381] The charging management module 140 is used to receive charging input from the charger. The power management module 141 is used to connect the battery 142, and the charging management module 140 is connected to the processor 110.

[0382] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. Mobile communication module 150 can provide solutions for wireless communication applications in electronic devices, including 2G / 3G / 4G / 5G. Mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. Mobile communication module 150 can receive electromagnetic waves through antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to modem processor for demodulation. Mobile communication module 150 can also amplify the signal modulated by modem processor and convert it into electromagnetic waves for radiation through antenna 1. In some embodiments, at least some functional modules of mobile communication module 150 can be disposed in processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the same device as at least some modules of the processor 110.

[0383] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0384] The wireless communication module 160 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0385] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the electronic device to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0386] Electronic devices implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0387] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N displays 194, where N is a positive integer greater than 1.

[0388] Electronic devices can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor, thereby acquiring real-time video data.

[0389] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0390] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device may include one or N cameras 193, where N is a positive integer greater than 1.

[0391] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device is selecting a frequency, a DSP can perform a Fourier transform on the frequency energy.

[0392] Video codecs are used to compress or decompress digital video. Electronic devices can support one or more video codecs. This allows the electronic device to play or record video in various encoded formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0393] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, voice recognition, and text understanding.

[0394] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0395] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, fifth generation DDR SDRAM is generally called DDR5 SDRAM), etc.

[0396] Non-volatile memory can include disk storage devices and flash memory.

[0397] In this embodiment, the non-real-time video can be stored in non-volatile memory.

[0398] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0399] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0400] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0401] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0402] Electronic devices can implement audio functions such as music playback and recording through audio modules 170, speakers 170A, receivers 170B, microphones 170C, headphone jacks 170D, and application processors.

[0403] Audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. Speaker 170A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. Receiver 170B, also called a "handpiece," is used to convert audio electrical signals into sound signals. Microphone 170C, also called a "microphone" or "microphone unit," is used to convert sound signals into electrical signals.

[0404] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0405] A pressure sensor 180A is used to sense pressure signals and can convert them into electrical signals. In some embodiments, the pressure sensor 180A may be located on a display screen 194. A gyroscope sensor 180B can be used to determine the motion posture of the electronic device. A barometric pressure sensor 180C is used to measure barometric pressure. A magnetic sensor 180D includes a Hall effect sensor. An accelerometer sensor 180E can detect the magnitude of acceleration of the electronic device in various directions (typically three axes). A distance sensor 180F is used to measure distance. A proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photodetector, such as a photodiode. An ambient light sensor 180L is used to sense ambient light intensity. A fingerprint sensor 180H is used to collect fingerprints. The electronic device can utilize the collected fingerprint characteristics to achieve fingerprint unlocking, access app locks, fingerprint photography, fingerprint answering of calls, etc. A temperature sensor 180J is used to detect temperature.

[0406] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of the electronic device, in a different position than display screen 194.

[0407] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. The electronic device can receive button input and generate key signal inputs related to user settings and function control. Motor 191 can generate vibration alerts. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The electronic device interacts with the network through the SIM card to achieve functions such as calls and data communication.

[0408] Figure 25This is an exemplary schematic diagram of the software structure of the electronic device provided in the embodiments of this application.

[0409] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system library layer, and the kernel layer.

[0410] The application layer can include a series of application packages. For example... Figure 20 As shown, the application package may include applications (also known as apps) such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0411] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0412] like Figure 25 As shown, the application framework layer may include window management services, display management services, content providers, view systems, phone managers, resource managers, notification managers, local profile management assistants (Local Profile Assistant, LPA), etc.

[0413] The window management service is responsible for starting, adding, and deleting windows. It can determine the application displayed on the window, as well as the creation, destruction, and attribute changes of the application's layers. It can also determine whether there is a status bar, lock the screen, and capture the screen.

[0414] The display management service can obtain the number and size of display areas, and is responsible for starting, adding, and deleting display areas.

[0415] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0416] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0417] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0418] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog-style notifications on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0419] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0420] The view system also includes UniRender, which can receive the rendering trees of one or more applications. UniRender can synchronize layer information, such as layer creation, destruction, and attribute changes, through the window management service. UniRender can also synchronize display area information, such as screen size, from the display management service.

[0421] Optionally, in some embodiments of this application, the view system also includes SurfaceFlinger. On an electronic device configured with a whitelist, when an application is not on the whitelist, the application's UI thread generates a render tree, the application's Render thread generates a bitmap, and then SurfaceFlinger performs layer composition.

[0422] Optionally, in some embodiments of this application, when both whitelisted and non-whitelisted applications are displayed on the display area, the generation of bitmaps for whitelisted applications is handled by UniRender. After generating the bitmap, UniRender passes it to SurfaceFlinger, which then composites the bitmap with other non-whitelisted applications to generate a bitmap for display.

[0423] The runtime includes the core libraries and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0424] The core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part is the core library itself.

[0425] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0426] System libraries can include multiple functional modules. For example: surface manager, media libraries, and graphics processing libraries, where the graphics processing libraries include: 3D graphics processing libraries (e.g., OpenGLES), 2D graphics engines (e.g., SGL), etc.

[0427] The Surface Manager is used to manage the display subsystem and provides the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0428] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0429] 3D graphics processing libraries are used to implement 3D graphics drawing, image rendering, layer compositing, and layer processing. 2D graphics engines are drawing engines for 2D graphics.

[0430] The kernel layer is the layer between hardware and software. The kernel layer includes at least the display driver, camera driver, audio driver, sensor driver, and virtual card driver.

[0431] In this embodiment, one or more applications in the application layer pass their respective UI thread-generated render trees to the UniRender process of the view system. The UniRender process obtains window control information and display area information from the window management service and the display management service, and then composites the render trees of the applications on a display area into a target render tree. After generating the target render tree, the UniRender process calls the layer processing library to execute the DrawOP operation in the drawing instruction list of the target render tree, thereby generating a bitmap. UniRender then passes the generated bitmap to the display driver for display.

[0432] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0433] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0434] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An interface generation method, applied to electronic devices, characterized in that, include: The first process generates a target rendering tree based on the second rendering tree and the third rendering tree, wherein the second rendering tree is a rendering tree generated by the second process for drawing a frame of the interface of the second process; the third rendering tree is a rendering tree generated by the third process for drawing a frame of the interface of the third process, and the target rendering tree is used to draw the target interface; During the process of the first process generating the target interface based on the target rendering tree, the first process determines the first content based on at least one rendering node in the target rendering tree, and performs animation rendering on the first content based on the first animation parameters to obtain the second content, wherein the animation parameters of the at least one rendering node are the first animation parameters; Alternatively, during the process of the first process generating the target interface based on the target rendering tree, the first process determines the first content based on a first region on the first surface, and performs animation rendering on the first content based on the first animation parameters to obtain the second content. The first process generates the target interface on the first surface based on the target rendering tree, and the first animation parameters are determined by the first process. The first process generates the target interface, which includes the second content, and the animation rendering is performed in an off-screen buffer.

2. The method according to claim 1, characterized in that, Before the first process determines the target rendering tree, the method further includes: The first process adds the at least one rendering node to the target rendering tree, and the at least one rendering node is configured with the first animation parameter; Alternatively, the first process configures the first animation parameters for the at least one rendering node.

3. The method according to claim 1, characterized in that, When the animation parameters of at least one rendering node indicate that the foreground is blurred, the first content includes the view corresponding to the at least one rendering node; When the animation parameters of the at least one rendering node indicate a blurred background, the first content includes the content of the area corresponding to the at least one rendering node on the first surface.

4. The method according to any one of claims 1-3, characterized in that, The at least one rendering node includes rendering nodes in the second rendering tree and rendering nodes in the third rendering tree, and the first content includes part or all of the content of a frame of the second process and part or all of the content of a frame of the third process.

5. The method according to any one of claims 1-3, characterized in that, The first rendering node in the second rendering tree is configured with the first animation parameters by the second process; The second rendering node in the third rendering tree is configured with the first animation parameters by the third process; The at least one rendering node includes the first rendering node and the second rendering node.

6. The method according to any one of claims 1-3, characterized in that, When the animation parameters indicate that the background or foreground is blurred, the animation parameters also include at least one of blur degree, blur precision, and blur radius.

7. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer-readable instructions, which the one or more processors invoke to cause the electronic device to perform the method as described in any one of claims 1-6.

8. A chip system applied to an electronic device, the chip system comprising one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-6.

10. A computer program product comprising computer-readable instructions that, when executed by one or more processors, implement the method as described in any one of claims 1-6.

Images