Rendering method and electronic device

Abstract

A rendering method and an electronic device, which can perform frame-by-frame rendering on an interface to be displayed, for example, completing the rendering of an interface within a plurality of rendering frames, thereby avoiding the failure to complete interface rendering within a single rendering frame, which results in the interface not being delivered to a display in a timely manner. For example, the graphics rendering frame rate of the electronic device is a first frame rate, and the first frame rate is used for indicating the number of times the electronic device performs rendering within a first duration. After determining a first interface to be displayed, the electronic device may determine, on the basis of the first frame rate, that the duration of each instance of rendering is a second duration. The electronic device may determine N rendering subtasks of the first interface on the basis of the second duration, wherein the N rendering subtasks are used for rendering all content of the first interface, and the rendering duration of each of the N rendering subtasks is less than or equal to the second duration, N being a positive integer. The electronic device completes the N rendering subtasks within N second durations and then displays the first interface.

Description

A rendering method and electronic device

[0001] Cross-reference to Related Applications

[0002] This application claims priority to the Chinese Patent Application No. 202411304228.X, filed on September 18, 2024, and entitled "A rendering method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of terminals, and in particular to a rendering method and electronic device. BACKGROUND

[0004] With the development of electronic technology, the interface design of electronic devices is becoming more and more complex, for example, there are more and more display elements in the interface, and it takes a long time and consumes a large amount of power to render such an interface. Assuming that the graphic rendering frame rate of the electronic device is 120 Hertz (Hz), that is, one interface needs to be rendered every 8 ms, if the interface is complex, it is difficult to complete the rendering within 8 ms, which will cause the interface to be unable to be displayed in time, affecting the user experience. SUMMARY

[0005] The embodiments of the present application provide a rendering method and electronic device, which can perform frame splitting rendering on a to-be-displayed interface, for example, rendering one interface in multiple rendering frames, so as to avoid the situation that the interface rendering cannot be completed in a single rendering frame, resulting in the interface being unable to be displayed in time.

[0006] In a first aspect, a rendering method is provided, which can be applied to an electronic device. The graphic rendering frame rate of the electronic device is a first frame rate, and the first frame rate is used to indicate the number of renderings of the electronic device within a first time length. For example, the method can include: determining a first interface to be displayed; determining a second time length for each rendering according to the first frame rate; determining N rendering tasks of the first interface according to the second time length, the N rendering tasks being used to render all contents of the first interface, and the rendering time length of each rendering task in the N rendering tasks being less than or equal to the second time length, N being a positive integer; completing the N rendering tasks within N second time lengths; and displaying the first interface.

[0007] In the embodiments of the present application, the electronic device can perform frame splitting rendering on the to-be-displayed interface, for example, the rendering time length of the electronic device is a second time length (for example, 8 ms), and then the electronic device can complete N rendering tasks of the to-be-displayed interface within N second time lengths. Since the rendering time length of each rendering task is less than or equal to the second time length, the situation that a certain rendering task cannot be rendered in time will not occur, resulting in the accumulation of rendering tasks and the situation that the display cannot be displayed in time.

[0008] In a possible design, a difference between the rendering duration of each rendering task and the second duration is less than or equal to the first threshold.

[0009] In an embodiment of this application, the rendering duration of each rendering task is less than or equal to the second duration, and a difference between the rendering duration and the second duration is less than the first threshold. On the one hand, this can avoid the situation that a rendering task cannot be rendered in time, resulting in accumulation of rendering tasks and loss of frames due to failure to display in time. On the other hand, this can avoid the situation that the electronic device is idle due to lack of enough rendering tasks within the second duration, resulting in waste of time.

[0010] In a possible design, the first threshold is less than or equal to the rendering duration of the smallest display element in the first interface.

[0011] In an embodiment of this application, the rendering duration of each rendering task is less than or equal to the second duration, and a difference between the rendering duration and the second duration is less than or equal to the rendering duration of the smallest display element in the interface to be displayed, so as to minimize the idle time of the electronic device within the second duration and avoid waste of time.

[0012] In a possible design, each of the N rendering tasks is used to render P display elements, P is a positive integer, and the display elements rendered by any two of the N rendering tasks are not repeated.

[0013] In an embodiment of this application, the electronic device can determine the N rendering tasks of the interface to be displayed, and the display elements rendered by any two of the N rendering tasks are not repeated, so as to avoid repeated rendering and waste of resources.

[0014] It should be noted that the number of display elements included in each of the N rendering tasks is not necessarily the same. For example, the rendering task 1 includes the display element 1 and the display element 2, the rendering task 2 includes the display element 3, and the rendering task 3 includes the display element 4.

[0015] In a possible design, the rendering order of the N rendering tasks is related to at least one of the following: the layers where the display elements corresponding to the N rendering tasks are located; and the display positions of the display elements corresponding to the N rendering tasks.

[0016] In an embodiment of this application, the electronic device can determine the N rendering tasks of the interface to be displayed, and the electronic device can sort and render the N rendering tasks according to the layers where the display elements corresponding to the N rendering tasks are located and / or the display positions of the display elements corresponding to the N rendering tasks, so as to ensure that the rendering is performed in order.

[0017] In a possible design, the rendering sequence of the N rendering tasks is related to the layers in which the display elements corresponding to the N rendering tasks are located, and the rendering sequence of the N rendering tasks includes: rendering in a sequence from back to front of the layers in which the display elements corresponding to the N rendering tasks are located.

[0018] In an embodiment of the present application, the electronic device can render the N rendering tasks in a sequence from back to front of the layers in which the display elements corresponding to the N rendering tasks are located. For example, the rendering sequence of the N rendering tasks is: the rendering task at the front is used to render the display element located in the back layer, and the rendering task at the back is used to render the display element located in the front layer. That is, the display element in the back layer is rendered preferentially, and then the display element in the front layer is rendered, which is consistent with the layer superposition sequence in computer graphics technology, for example, the superposition sequence of multiple layers is to superimpose the front layer on the back layer.

[0019] In a possible design, the display element in the back layer can be used to accommodate the display element in the front layer. For example, the display position of the display element 1 in the front layer can be located in the display area of the display element 2 in the back layer, and after layer superposition, the effect that the display element 1 is located inside the display element 2 is displayed.

[0020] In a possible design, the rendering sequence of the N rendering tasks is related to the display positions of the display elements corresponding to the N rendering tasks, and the rendering sequence of the N rendering tasks includes: rendering in a sequence from top to bottom or from left to right of the display positions of the display elements corresponding to the N rendering tasks.

[0021] In an embodiment of the present application, the electronic device can render the N rendering tasks in a sequence from top to bottom or from left to right of the display positions of the display elements corresponding to the N rendering tasks. For example, the rendering sequence of the N rendering tasks is: the rendering task at the front is used to render the display element with a display position at the top or the left, and the rendering task at the back is used to render the display element with a display position at the bottom or the right. That is, the display element at the top or the left is rendered preferentially, and then the display element at the bottom or the right is rendered. In this way, the display screen refresh sequence is also to refresh the display element at the top or the left preferentially, and then refresh the display element at the bottom or the right. Therefore, the user can see the display element at the top or the left preferentially, and then see the display element at the bottom or the right, which is more consistent with the user habit.

[0022] In a possible design, the display elements corresponding to multiple rendering tasks in the N rendering tasks are located in the same layer, and the multiple rendering tasks are rendered in a sequence according to the display positions of the display elements corresponding to the multiple rendering tasks; or the display elements corresponding to multiple rendering tasks in the N rendering tasks are located in the same position, and the multiple rendering tasks are rendered in a sequence according to the layers in which the display elements corresponding to the multiple rendering tasks are located.

[0023] In an embodiment of the present application, if the plurality of rendering tasks correspond to the same layer, the rendering is sorted according to the display position; if the plurality of rendering tasks correspond to the same position, the rendering is sorted according to the layer in which the display element is located, so as to ensure that the N rendering tasks are performed in order.

[0024] It can be understood that the plurality of rendering tasks correspond to the display elements located at the same position, which can include that the display positions of the display elements corresponding to the plurality of rendering tasks overlap. For example, rendering task 1 corresponds to display element 1, and rendering task 2 corresponds to display element 2. Display element 1 is located in the display area of display element 2 (for example, the attribute of display element 1 is a container for accommodating display element 2), so that the display elements corresponding to rendering task 1 and rendering task 2 are located at the same position.

[0025] In a possible design, the rendering order of the P display elements is related to at least one of the following: the layer in which the P display elements are located; and the display position of the P display elements.

[0026] In an embodiment of the present application, each rendering task can include P display elements, and the electronic device can sort and render the P display elements according to the layer in which the P display elements are located and / or the display position, so that the rendering of each display element is performed in order.

[0027] In a possible design, the rendering order of the P display elements is related to the layer in which the P display elements are located, and includes that the rendering order of the P display elements is: rendering in the order from back to front according to the layer in which the display elements are located.

[0028] In an embodiment of the present application, one rendering task can be used to render P display elements, and the electronic device can sort and render the P display elements in the order from back to front according to the layer in which the P display elements are located. For example, the rendering order of the P display elements is: first rendering the display elements located in the back layer, and then rendering the display elements located in the front layer. That is, the display elements in the back layer are preferentially rendered, and then the display elements in the front layer are rendered, which is consistent with the layer superposition order in computer graphics technology.

[0029] In a possible design, the rendering order of the P rendering tasks is related to the display position of the P display elements, and includes that the rendering order of the P display elements is: rendering in the order from top to bottom or from left to right according to the display position.

[0030] In an embodiment of the present application, one rendering task can be used to render P display elements, and the electronic device can render the P display elements in a display order from top to bottom or from left to right. For example, the rendering order of the P display elements is: first, the display elements at the top or left are rendered, and then the display elements at the bottom or right are rendered. That is, the display elements at the top or left are rendered first, and then the display elements at the bottom or right are rendered. Therefore, the user can first see the display elements at the top or left, and then see the display elements at the bottom or right, which is more in line with user habits.

[0031] In a possible design, a plurality of display elements in the P display elements are located in the same layer, and the plurality of display elements are rendered in a display position order; or a plurality of display elements in the P display elements are located in the same position, and the plurality of display elements are rendered in a layer order.

[0032] In an embodiment of the present application, one rendering task can be used to render P display elements, and if the P display elements are located in the same layer, the P display elements are rendered in a display position order, and if the P display elements are located in the same position, the P display elements are rendered in a layer order. In summary, the rendering of each display element can be completed in order.

[0033] In a possible design, the first task in the N rendering tasks includes a first subtask for rendering an interface background in the first interface.

[0034] In an embodiment of the present application, the electronic device can first render the interface background of the to-be-displayed interface, so that the user can first see the overall appearance of the interface.

[0035] In a possible design, the last task in the N rendering tasks includes a second subtask for rendering interface details in the first interface.

[0036] In an embodiment of the present application, the electronic device can lastly render the interface details of the to-be-displayed interface. For example, the details in the interface are rendered after the interface background is rendered. It can be understood that when a user looks at an interface, the user often first pays attention to the overall interface and then pays attention to the details in the interface. Therefore, rendering the interface background first and then rendering the interface details is more in line with user habits.

[0037] In a possible design, each display element in the P display elements does not contain other display elements.

[0038] In an embodiment of the present application, one rendering task can include P display elements, and each of the P display elements does not include other display elements, that is, each display element is a non-divisible display element. Taking one display element as an example, if the attribute of the display element is a container, that is, the display element contains other display elements, the contained other display elements are separated from the display element, and after the separation of the contained other display elements, the display element with the attribute of the container does not include any other display element, that is, the display element with the attribute of the container cannot be separated from any other display element.

[0039] In a possible design, according to the second time length, the N rendering tasks of the first interface are determined, including: determining all display elements in the first interface; and dividing all the display elements into N sets according to the second time length and the rendering time consumption of each display element in the all display elements, wherein the sum of the rendering time consumption of the display elements in each set in the N sets is less than or equal to the second time length, and each set corresponds to one rendering task.

[0040] In an embodiment of the present application, the electronic device can divide all the display elements into N sets according to the rendering time consumption of each display element in the display elements in the to-be-displayed interface, to obtain N rendering tasks, and the rendering time consumption of each rendering task is less than or equal to the second time length, so as to avoid the situation that a rendering task cannot be rendered in time, resulting in the accumulation of rendering tasks and the situation of frame loss due to the failure to display in time.

[0041] In a possible design, dividing the all display elements into N display element sets includes: performing traversal according to a first traversal order, to determine a first display element, wherein the rendering time consumption of the first display element is less than the second time length, and the first traversal order is a traversal order of the all display elements; continuing to perform traversal according to the first traversal order according to the rendering time consumption of the first display element, to determine a second display element, wherein the sum of the rendering time consumption of the first display element and the second display element is less than or equal to the second time length; and determining that the first display element and the second display element belong to the same rendering task.

[0042] In an embodiment of the present application, the electronic device can perform traversal on all the display elements in the to-be-displayed interface according to a certain traversal order, to divide all the display elements into N sets, to obtain N rendering tasks, and the rendering time consumption of each rendering task is less than or equal to the second time length, so as to avoid the situation that a rendering task cannot be rendered in time, resulting in the accumulation of rendering tasks and the situation of frame loss due to the failure to display in time.

[0043] In a possible design, the first display element and the second display element are located in the same layer or different layers.

[0044] In the embodiments of the present application, the electronic device can traverse the display elements in the same layer or across layers, without limitation. In summary, all the display elements can be divided into N sets, N rendering tasks are obtained, and the rendering time of each rendering task is less than or equal to the second time length, so as to ensure that each rendering task can be rendered in time.

[0045] In a possible design, the first display element and the second display element are located in different layers, and the first display element is used to accommodate the second display element.

[0046] In the embodiments of the present application, the electronic device can traverse the display elements across layers, for example, the first display element is found in a certain layer, the second display element is found in another layer, and the second display element is accommodated in the first display element, and then the two display elements are taken as a set, corresponding to a rendering task. In this way, the display element and other display elements accommodated in the display element can be rendered in the same rendering task.

[0047] In a possible design, the traversing according to the first traversal order includes traversing according to a layer order or traversing according to a display order.

[0048] In the embodiments of the present application, the electronic device can traverse the display elements according to a layer order or a display order, so as to avoid missing display elements.

[0049] In a possible design, the traversing according to the layer order includes traversing all the display elements in each layer in a layer order from back to front; and the traversing according to the display order includes traversing all the display elements in a display order from top to bottom or from left to right.

[0050] In the embodiments of the present application, the electronic device can traverse all the display elements in each layer in a layer order from back to front or traverse all the display elements in a display order from top to bottom or from left to right, so as to avoid missing display elements.

[0051] In a possible design, the method further includes determining the rendering time of each display element according to at least one of the type, size, and style of the display element.

[0052] In the embodiments of the present application, the electronic device can determine the rendering time consumption of each display element according to the type, size, style, etc. of each display element, which helps to improve the accuracy of the rendering time consumption of each display element. In this way, the N rendering tasks determined by the electronic device according to the rendering time consumption of each display element are more accurate, which can ensure that the rendering time consumption of each rendering task is less than or equal to the second time length, thereby ensuring that each rendering task can be rendered in time.

[0053] In a possible design, the method further includes: adjusting the graphic rendering frame rate to a second frame rate, the second frame rate being used to indicate the number of times of rendering of the electronic device within a third time length; determining a second interface to be displayed; determining, according to the second frame rate, a fourth time length as the rendering time length each time; determining, according to the fourth time length, M rendering tasks of the second interface, the M rendering tasks being used to render all the content of the second interface, and the rendering time length of each rendering task in the M rendering tasks being less than or equal to the fourth time length, M being a positive integer; completing the M rendering tasks within M fourth time lengths; and displaying the second interface.

[0054] In the embodiments of the present application, the electronic device can adjust the graphic rendering frame rate, for example, from a first frame rate to a second frame rate. After being adjusted to the second frame rate, the electronic device adjusts the rendering time length each time, for example, to a fourth time length. Therefore, for the interface to be displayed, the electronic device can complete M rendering tasks of the interface to be displayed within M fourth time lengths. Since the rendering time length of each rendering task is less than or equal to the fourth time length, a rendering task cannot be rendered in time, resulting in the accumulation of rendering tasks and the failure to display in time, which does not occur.

[0055] In a possible design, the adjusting of the graphic rendering frame rate to the second frame rate includes: adjusting the graphic rendering frame rate to the second frame rate according to the device running state or the user operation of the electronic device.

[0056] In the embodiments of the present application, the electronic device can adjust the graphic rendering frame rate, for example, automatically or manually. Taking automatic adjustment as an example, the electronic device adjusts the graphic rendering frame rate according to the device running state. Taking manual adjustment as an example, the electronic device adjusts the graphic rendering frame rate according to the user operation. In summary, in the embodiments of the present application, even if the image rendering frame rate is adjusted, the frame rendering can be performed, and the rendering time length of each rendering task of the interface to be displayed is less than or equal to the rendering time length of the electronic device each time, so that a rendering task cannot be rendered in time, resulting in the accumulation of rendering tasks and the failure to display in time, which does not occur.

[0057] The second aspect further provides an electronic device, comprising:

[0058] a processor, a memory, and one or more programs;

[0059] The one or more programs are stored in the memory and include instructions which, when executed by the processor, cause the electronic device to perform the method according to any one of the first aspect.

[0060] The third aspect further provides an electronic device including modules / circuits for performing the method according to any one of the first aspect. These modules / circuits can be implemented by hardware, or implemented by hardware executing corresponding software.

[0061] The fourth aspect further provides a computer-readable storage medium for storing a computer program, which, when executed on a computer, causes the computer to perform the method according to the first aspect.

[0062] The fifth aspect further provides a computer program product including a computer program, which, when executed on a computer, causes the computer to perform the method according to the first aspect.

[0063] The sixth aspect further provides a chip coupled with a memory in an electronic device, for invoking a computer program stored in the memory and performing the technical solution provided by the first aspect of the present application. In the present application, “coupled” means that two components are directly or indirectly combined with each other.

[0064] The seventh aspect further provides a chip system including a processing circuit and a storage medium, the storage medium storing instructions; the instructions are executed by the processing circuit to implement the method according to the first aspect.

[0065] The technical effects achieved by the second aspect to the seventh aspect can refer to the description of the technical effects achieved by the corresponding design solutions in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0066] FIGS. 1A-1C are a schematic diagram of a rendering process according to an embodiment of the present application;

[0067] FIGS. 2A-2B are another schematic diagram of a rendering process according to an embodiment of the present application;

[0068] FIG. 3 is a schematic diagram of a display element in an interface according to an embodiment of the present application;

[0069] FIG. 4A is a schematic diagram of a rendering tree according to an embodiment of the present application;

[0070] FIG. 4B is another schematic diagram of a rendering tree according to an embodiment of the present application;

[0071] FIG. 5 is a schematic diagram of dividing a rendering task according to an embodiment of the present application;

[0072] FIG. 6 is another schematic diagram of dividing a rendering task according to an embodiment of the present application;

[0073] FIG. 7 is a schematic diagram of a rendering queue according to an embodiment of the present application;

[0074] FIG. 8 is a schematic diagram of an electronic device according to an embodiment of the present application;

[0075] FIG. 9 is another schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] In the following, some terms in the embodiments of the present application are explained to facilitate the understanding of the skilled in the art.

[0077] At least one of the embodiments of the present application includes one or more, wherein the more means greater than or equal to two. In addition, it should be understood that in the description of the present application, the terms "first", "second", "third", etc. are only used for distinguishing description purposes, and cannot be understood as indicating or implying relative importance, nor can be understood as indicating or implying sequence. For example, the first interface and the second interface do not represent the importance or sequence of the two, but are only used for distinguishing description. In the embodiments of the present application, "and / or" is only used to describe the association relationship, which means that there are three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0078] The orientation terms mentioned in the embodiments of the present application, such as "up", "down", "left", "right", "in", "out", etc. are only the direction of the drawings, therefore, the orientation terms are used to better and more clearly illustrate and understand the embodiments of the present application, and are not intended to indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, therefore, it cannot be understood as a limitation of the embodiments of the present application.

[0079] Reference to "one embodiment", "some examples" or "one specific embodiment" etc., in the present application description means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases "in some examples", "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments", etc., in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. The terms "comprising", "comprises", "including", "includes" or "having" etc., mean "including but not limited to" unless otherwise specifically indicated.

[0080] The rendering method provided by the embodiments of the present application can be applied to an electronic device. For example, the electronic device can be a mobile terminal. For example, the mobile terminal can be a mobile phone, a tablet computer, a notebook computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. Alternatively, the mobile terminal can be a wearable device, or a vehicle-mounted display device, or a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, and the like. In this embodiment of the present application, the specific type of the mobile terminal is not limited. For example, the wearable device can include a wrist-worn device, a head-worn device, and the like. For example, the wrist-worn device can include a watch, a bracelet, and the like. For example, the head-worn device can include glasses, a helmet, and the like.

[0081] In the embodiments of the present application, the electronic device has the functions of graphic rendering and display. In order to facilitate understanding, the following briefly describes the two processes of graphic rendering and graphic display.

[0082] Graphic rendering can be understood as converting the content to be displayed from computer language into graphics that can be visually seen by the user. In the embodiments of the present application, the electronic device can render according to a certain rendering frame rate. The rendering frame rate can be understood as a rendering frequency, which is used to indicate the number of renderings per unit time. Through the unit time and the number of renderings per unit time, the duration of each rendering can be determined. The following describes the two parameters, i.e., the number of renderings per unit time and the duration of each rendering.

[0083] (1) the number of rendering times per unit time.

[0084] The unit time can be per second, per millisecond, per microsecond, and can also be per minute, per hour, etc. without limitation. Taking the unit time as per second as an example, the number of rendering times per second can be 60 times, 90 times, 120 times, etc. In this paper, the number of rendering times per unit time is referred to as the frame rate of graphic rendering, and in this paper, the unit time is taken as per second, so the unit of the frame rate of image rendering can be hertz (Hz), which is used to indicate the number of rendering times per second. For example, the frame rate of graphic rendering can be 60 Hz, 90 Hz, 120 Hz, etc. It can be understood that the higher the frame rate of graphic rendering, the faster the rendering speed, the lower the probability of interface lag on the display screen, and the better the user experience.

[0085] (2) the duration of each rendering.

[0086] The duration of each rendering can be derived according to the unit time and the number of rendering times per unit time. For example, the duration of each rendering = unit time / number of rendering times per unit time. Taking the unit time as per second as an example, and assuming that the number of rendering times per unit time is 120 times, then the duration of each rendering = 1s / 120 ≈ 0.00833s ≈ 8ms, that is, the duration of each rendering is 8ms. In this paper, the duration of each rendering is referred to as a rendering frame, for example, a rendering frame is 8ms. Moreover, in this paper, the content rendered by the electronic device in one rendering frame (for example, 8ms) is referred to as one frame of content, for example, 120 frames of content are rendered in 1 second.

[0087] Continuing to take the example of rendering 120 times per second and the duration of each rendering being 8ms, one possible case is that the two adjacent renderings are consecutive, for example, the starting time of the second rendering is equal to the ending time of the first rendering. Another possible case is that the two adjacent renderings are spaced, for example, the starting time of the second rendering can be later than the ending time of the first rendering. This is because, as described in the foregoing, the duration of each rendering of 8ms is the duration obtained after rounding off 0.00833s, so there is idle time within 1s in addition to the 120 8ms, so there can be a certain interval between the two adjacent renderings.

[0088] In some embodiments, the electronic device needs to call the rendering process to perform graphic rendering. Therefore, the number of rendering times per unit time can also be understood as the number of times the rendering process is called per unit time. The duration of each rendering can also be understood as the running duration of the rendering process after being called each time. Continuing to take the example of rendering 120 times per second and the duration of each rendering being 8ms, that is, the electronic device calls the rendering process 120 times per second, and the rendering process runs for 8ms after being called each time.

[0089] The graphic display can be understood as displaying the rendered graphics through the display screen of the electronic device. In the embodiments of the present application, the electronic device can display graphics at a certain refresh frame rate, which can be understood as a refresh frequency, indicating the number of refreshes per unit time. Through the unit time and the number of refreshes per unit time, the duration of each refresh can be determined. The two parameters, i.e., the number of refreshes per unit time and the duration of each refresh, are described below.

[0090] (1) The number of refreshes per unit time.

[0091] The unit time can be per second, per millisecond, per microsecond, and can also be per minute, per hour, etc. without limitation. Taking the unit time as per second as an example, the number of refreshes per second can be 60 times, 90 times, 120 times, etc. The number of refreshes per unit time is referred to as the display screen refresh frame rate in this paper, and the unit time is taken as per second in this paper, so the unit of the display screen refresh frame rate can be hertz (Hz), indicating the number of refreshes per second. For example, the display screen refresh frame rate can be 60 Hz, 90 Hz, 120 Hz, etc. It can be understood that the higher the display screen refresh frame rate, the faster the graphics on the display screen are refreshed, the lower the probability of interface lag, and the better the user experience. In the embodiments of the present application, there can be a certain correlation between the graphic rendering frame rate and the display screen refresh frame rate. For example, the graphic rendering frame rate can be greater than or equal to the display screen refresh frame rate, which can be understood as the graphic rendering speed being higher than the graphic refresh speed on the display screen, so as to ensure that there is enough graphics on the display screen to be refreshed, avoiding lag.

[0092] (2) The duration of each refresh.

[0093] The duration of each refresh can be derived according to the unit time and the number of refreshes per unit time. For example, the duration of each refresh = unit time / number of refreshes per unit time. Taking the unit time as per second as an example, and assuming that the number of refreshes per unit time is 120 times, then the duration of each refresh = 1s / 120≈0.00833s≈8ms, that is, the duration of each refresh is 8ms.

[0094] The process of graphic rendering and display of the electronic device is described below in conjunction with the drawings.

[0095] For example, please refer to FIG. 1A, which is a schematic diagram of a display system provided by an embodiment of the present application. The display system can be applied to an electronic device, which can be various types of electronic devices listed above. As shown in FIG. 1A, the display system includes a main chip and a display screen. It should be noted that only the parts of the electronic device related to the embodiments of the present application are shown in FIG. 1A, and in fact, the electronic device can include more devices than FIG. 1A, which are not listed one by one.

[0096] The main chip can be understood as the nerve center and command center of the electronic device, responsible for various task processing of the electronic device. For example, the main chip can be a system on chip (SOC) of the electronic device. For example, the main chip can include one or more processors, such as one or more of a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), etc. In the embodiments of the present application, the main chip can be used for graphics rendering. For example, the main chip integrates a GPU, which can be used for graphics rendering. For graphics rendering, please refer to the description above. After the main chip renders the graphics, it can provide the rendered graphics to the display screen for display.

[0097] The display screen is used to display graphics. Optionally, the display screen can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In other words, the display screen is not limited in the embodiments of the present application. In the embodiments of the present application, the display screen can receive the rendered graphics from the main chip and display the graphics.

[0098] Continuing with FIG. 1A as an example, in the embodiments of the present application, the main chip can have two rendering mechanisms, which are described below.

[0099] The first rendering mechanism is single-frame rendering.

[0100] Single-frame rendering can be understood as rendering a complete interface within one rendering frame. The rendering frame has been described above. Taking a graphics rendering frame rate of 120 Hz as an example, the rendering frame is 8 ms, that is, a complete interface is rendered within 8 ms. For example, as shown in FIG. 1B, the main chip renders a complete interface 1 within the rendering frame 1 (8 ms), renders a complete interface 2 within the rendering frame 2 (the next 8 ms), and renders a complete interface 3 within the rendering frame 3 (the next 8 ms). The interface 1, the interface 2, and the interface 3 can be the same interface or different interfaces, which are not limited. The main chip sends the rendered interface to the display screen. As described above, the graphics rendering frame rate can be greater than or equal to the refresh frame rate of the display screen. Taking the graphics rendering frame rate equal to the refresh frame rate of the display screen as an example, for example, both are 120 Hz, in other words, the main chip renders one frame every 8 ms, and the display screen refreshes one frame every 8 ms. Therefore, the main chip can send one frame after rendering one frame. For example, as shown in FIG. 1C, the main chip renders one frame after another. The display screen refreshes one frame after another. It can be understood that the main chip needs to render the next frame in advance before the display screen refreshes the next frame. For example, as shown in FIG. 1C, the main chip needs to render the first frame in advance before the display screen refreshes the first frame (for example, the interface 1), and the main chip needs to render the second frame in advance before the display screen refreshes the second frame (for example, the interface 2).

[0101] It should be noted that in this single-frame rendering mechanism, for a complex interface, it is difficult to complete rendering in one rendering frame due to the difficulty and time-consuming of rendering, and therefore the rendering time of the complex interface can exceed one rendering frame. For example, interface 3 is a complex interface, and it takes 10 ms to complete the rendering of interface 3, for example. In this case, it can affect the interface refresh of the display screen. For example, as shown in FIG. 1C, after the display screen refreshes the second frame interface, it should refresh the third frame interface (for example, interface 3), but due to the difficulty of the third frame interface, it cannot be rendered within 8 ms, that is, when the display screen needs to refresh the third frame interface, the third frame interface has not been rendered, causing the display screen to stay on the second frame interface. This phenomenon is called "frame loss". On the one hand, it can cause the interface on the display screen to stay, and users with sensitive vision can perceive that the interface is not smooth; on the other hand, due to the incomplete rendering of the third frame interface, the subsequent frame interfaces cannot be rendered in time, causing rendering tasks to accumulate, which can cause the system to crash in severe cases.

[0102] The second rendering mechanism is frame-by-frame rendering.

[0103] Frame-by-frame rendering can be understood as rendering an interface in multiple rendering frames. Taking a graphics rendering frame rate of 120 Hz as an example, the rendering frame is 8 ms, and an interface is rendered in multiple rendering frames (for example, 3 x 8 ms, i.e. 24 ms). For example, as shown in FIG. 2A, the main chip renders part of interface 1 (for example, the part outside the dashed box) in rendering frame 1 (8 ms); renders another part of interface 1 (for example, the square part) in rendering frame 2 (the next 8 ms); and renders the remaining part of interface 1 (for example, the circular part) in rendering frame 3 (the next 8 ms). That is, interface 1 is rendered in three rendering frames. The main chip can send a frame to the display screen after rendering one frame. Similarly, the main chip needs to render the next frame in advance before the display screen refreshes the next frame. For example, as shown in FIG. 2B, the main chip needs to render the first frame (for example, part of interface 1) in advance before the display screen refreshes the first frame; the main chip needs to render the second frame (for example, another part of interface 1) in advance before the display screen refreshes the second frame; and the main chip needs to render the third frame (for example, the remaining part of interface 1) in advance before the display screen refreshes the third frame.

[0104] As shown in FIG. 2B, in the second rendering mechanism, the interface on the display screen is gradually displayed completely. For example, the display screen displays a part of the interface first, then displays another part of the interface, and so on, until all the contents of the interface are displayed. For example, it takes three rendering frames (i.e. 3*8ms=24ms) to render a complete interface, that is, the display screen needs 24ms to display the complete contents of the interface. Generally, the magnitude of the visual reflection time of a user is greater than milliseconds, so the display of the complete contents of the interface within 24ms will not cause a time delay in the visual experience of the user, and therefore the frame-by-frame rendering mechanism has little effect on the visual experience of the user.

[0105] In the first rendering mechanism (e.g. FIG. 1C), an interface needs to be completed within one rendering frame, and once a complex interface appears, the rendering will not be timely and will cause frame loss. In the second rendering mechanism (e.g. FIG. 2B), an interface can be completed within multiple rendering frames, and if a complex interface appears, it can be rendered in multiple rendering frames, which can to some extent avoid frame loss caused by untimely rendering.

[0106] In the second rendering mechanism, the problems of interface splitting and rendering task allocation are involved. The interface splitting is to split an interface into multiple parts. The rendering task allocation is to allocate the multiple parts obtained by splitting to multiple rendering frames.

[0107] If the interface splitting is improper, frame loss will occur. For example, an interface is split into A part, B part and C part, A part has a large rendering difficulty and a long rendering time, and A part is allocated to rendering frame 1. In rendering frame 1, the rendering of A part cannot be completed. In this case, frame loss will still occur. For example, in FIG. 2B, if the first frame (i.e. the part other than the circle and the square) has a long rendering time, the rendering cannot be completed within one rendering frame, which will cause frame loss.

[0108] If the rendering task allocation is improper, the rendering frame cannot be fully utilized. For example, an interface is split into A part, B part and C part, and the three parts are allocated to three rendering frames, for example, A part is allocated to rendering frame 1, B part is allocated to rendering frame 2, and C part is allocated to rendering frame 3. If the rendering of B part has a short time, after the rendering of B part is completed in rendering frame 2, there is still time left, but there is no other content to be rendered in the remaining time, so the remaining time is idle time, which is wasted. For example, in FIG. 2B, the second frame (i.e. the square) has a short rendering time, and the rendering is completed within 3ms. Therefore, the remaining 5ms is wasted. Similarly, the third frame (i.e. the circle) has a short rendering time, and the rendering is completed within 2ms. Therefore, the remaining 6ms is wasted.

[0109] In order to avoid frame loss and waste of time in rendering frames, in the embodiments of the present application, the electronic device can split the interface into multiple parts when splitting the interface, and the rendering time of each part of the multiple parts does not exceed one rendering frame (for example, 8 ms). Therefore, each part can be completed within one rendering frame, and frame loss does not occur. In addition, when the electronic device allocates rendering tasks, the rendering time of each part split can be allocated, for example, if the total rendering time of two parts is less than one rendering frame, the two parts can be allocated within one rendering frame, so as to improve the time utilization rate within the rendering frame and avoid wasting idle time within the rendering frame.

[0110] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the drawings.

[0111] An interface can be displayed on the display screen of the electronic device, and the interface can be an application interface of a currently running application of the electronic device. The currently running application can be a system application or a third-party application, which is not limited. It can be understood that the interface can be updated. The updating method can include automatic updating and manual updating. Taking automatic updating as an example, for example, when the electronic device receives an incoming call or a new message, the interface is automatically updated, and of course there are other scenarios of automatic updating of the interface, which are not listed one by one. Taking manual updating as an example, when the electronic device receives a user operation, the interface is updated, and the user operation can be various types of operations such as clicking, sliding, and long pressing. Regardless of which way, when the electronic device updates the interface, the interface to be displayed (or referred to as the interface to be updated) can be determined first, and then the interface to be displayed is rendered, and finally the rendered interface is displayed. Among them, the interface to be displayed (that is, what kind of interface to be displayed) can be determined by the application. One possible way is that the program package of the application contains display elements in each interface of all interfaces of the application, and the electronic device can determine the interface to be displayed and the display elements in the interface to be displayed through the program package of the application, and render the interface to be displayed through the display elements. It should be noted that the "display element" in the text can also have other names, such as "display component", "component", etc.

[0112] As described above, the electronic device needs to determine the display elements in the interface to be displayed. It is considered that in actual applications, there is a scenario that the display positions of two display elements in the interface to be displayed overlap. For example, display element 1 is located in region 1, and display element 2 is located in region 2. Region 1 is a part of region 2. From the perspective of the user, it is easy to be mistaken that display element 1 belongs to a part of display element 2, and only one display element 2 is considered, but actually there are two display elements. In order to avoid missing display elements, one possible way is that for a display element, if the display element can accommodate other display elements, the accommodated other display elements are separated from the display element as separate display elements. Continuing to take the example that display element 1 is inside display element 2, display element 1 can be separated from display element 2 as two independent display elements. It should be noted that in this way, the electronic device needs to know which display elements accommodate other display elements. As an example, the electronic device can determine whether the display element accommodates other display elements according to the attribute of the display element, and if so, the accommodated display elements are separated therefrom. The attribute of the display element can be indicated in the application package. For example, the attribute of the display element can include container, row, column, etc., which will be described below.

[0113] If the attribute of a display element is a container, it means that the display element can accommodate other display elements. For example, in (a) of FIG. 3, the attribute of display element Q is a container, and display element Q can accommodate other display elements, such as display elements A, B, and C. In this case, the electronic device can separate all display elements accommodated in the display element with the attribute of the container. For example, in (a) of FIG. 3, display elements A, B, and C are separated from display element Q, as shown in (b) of FIG. 3. It should be noted that after display elements A, B, and C are separated from display element Q, display element Q can be a complete display element or can not be a complete display element. Taking the example that display element Q is a complete display element, one possible scenario is that display element Q is a complete image, and display elements A, B, and C cover the upper layer of the image and block part of the image. After display elements A, B, and C are separated, display element Q is no longer blocked and remains complete. Taking the example that display element Q is not a complete display element, one possible scenario is that display element Q is an incomplete image, and the image includes three empty positions for filling display elements A, B, and C. After display elements A, B, and C are separated, the three empty positions are not filled, resulting in an incomplete display element Q.

[0114] If a display element has a property of row, it means that the display element can contain one or more rows of other display elements. Among them, the display elements in the same row can be horizontally aligned. The horizontal alignment can be understood as alignment in the horizontal direction. For example, the center points of the display elements in the same row have the same height, and the height can be the distance between the center point and the top edge or bottom edge of the display screen. For example, in (a) of FIG. 3, the display element C has a property of row. The display element C includes display elements C1, C2 and C3, and the display elements C1, C2 and C3 are horizontally aligned. In this case, the electronic device can separate all the display elements contained in the display element with the property of row. For example, in (a) of FIG. 3, the display elements C1, C2 and C3 are separated from the display element C. It should be noted that after the display elements C1, C2 and C3 are separated from the display element C, the display element C can be a complete display element or can not be a complete display element, and the principle has been described above and will not be repeated here.

[0115] If a display element has a property of column, it means that the display element can contain one or more columns of other display elements. Among them, the display elements in the same column can be vertically aligned. The vertical alignment can be understood as alignment in the vertical direction. For example, the center points of the display elements in the same column have the same distance from the left edge (or right edge) of the display screen. For example, in (a) of FIG. 3, the display elements C1, C2 and C3 all have a property of column. Taking the display element C1 as an example, the display element C1 includes display elements C1a and C1b, and the display elements C1a and C1b are vertically aligned. In this case, the electronic device can separate all the display elements contained in the display element with the property of column. For example, in (a) of FIG. 3, the display elements C1a and C1b are separated from the display element C1. It should be noted that after the display elements C1a and C1b are separated from the display element C1, the display element C1 can be a complete display element or can not be a complete display element, and the principle has been described above and will not be repeated here.

[0116] If a display element does not have any property, for example, is not a container, a row, or a column, it means that the display element cannot contain other display elements. For example, in (a) of FIG. 3, the display elements C1a, C1b, C2a, C2b, C3a and C3b do not have any property, that is, these display elements do not contain other display elements. In this case, the display elements C1a, C1b, C2a, C2b, C3a and C3b can be directly used as independent display elements.

[0117] Through the above process, the electronic device determines all display elements in the to-be-displayed interface, and for a display element capable of containing other display elements, all display elements contained by the display element can be separated out. In this way, there is no (or no other display elements are contained in) any other display elements in each display element. Continuing to take the to-be-displayed interface in (a) of FIG. 3 as an example, the to-be-displayed interface includes a total of 13 display elements, such as display elements Q, A, B, C, C1, C2, C3, C1a, C1b, C2a, C2b, C3a, and C3b in (b) of FIG. 3.

[0118] In some embodiments, after the electronic device determines all display elements in the to-be-displayed interface, the rendering time consumption of each display element can be evaluated to ensure that the rendering time consumption of each display element is less than one rendering frame (for example, 8 ms). There are various ways to evaluate the rendering time consumption, including but not limited to at least one of the following.

[0119] Method A: The rendering time consumption of a display element is estimated according to the type of the display element. The type of the display element can include an image, text, and the like. In one possible way, the electronic device pre-stores a corresponding relationship between the type of the display element and the rendering time consumption, and based on the corresponding relationship, the rendering time consumption of the display element can be determined. For example, the corresponding relationship can be Table 1 as follows.

[0120] Table 1:

[0121] Therefore, the electronic device can determine the rendering time consumption of the display element according to the type of the display element and Table 1. It can be understood that the rendering time consumption of the display element can be a specific time length or a time length range. If it is a time length range, the electronic device can determine a time length (for example, the average value) in the time length range as the rendering time consumption of the display element.

[0122] Method B: The rendering time consumption of a display element is estimated according to the size of the display element. The size of the display element can be understood as the data amount (or file amount) size of the display element, and the unit can be B, KB, and the like. In one possible way, the electronic device pre-stores a corresponding relationship between the size of the display element and the rendering time consumption, and based on the corresponding relationship, the rendering time consumption of the display element can be determined. For example, the corresponding relationship can be Table 2 as follows.

[0123] Table 2:

[0124] Therefore, the electronic device can determine the rendering time consumption of the display element according to the size of the display element and the above table 2. It can be understood that the rendering time consumption of the display element can be a specific time length or a time length range. If it is a time length range, the electronic device can determine a time length (for example, the average) in the time length range as the rendering time consumption of the display element.

[0125] In a manner C, the rendering time consumption of the display element is estimated according to the style of the display element. The style of the display element can include the shape of the display element or the like. The shape can include a rectangle, a circle, a pentagram, etc. In one possible manner, the electronic device pre-stores a corresponding relationship between the style of the display element and the rendering time consumption, and the rendering time consumption of the display element can be determined based on the corresponding relationship. For example, the corresponding relationship can be the following table 3.

[0126] Table 3:

[0127] Therefore, the electronic device can determine the rendering time consumption of the display element according to the style of the display element and the above table 3. It can be understood that the rendering time consumption of the display element can be a specific time length or a time length range. If it is a time length range, the electronic device can determine a time length (for example, the average) in the time length range as the rendering time consumption of the display element.

[0128] The above lists several manners of evaluating the rendering time consumption of the display element. It should be noted that in addition to the above three manners, other manners can also be used to evaluate the rendering time consumption of the display element, which are not limited. In addition, it should be noted that the above three manners can be used alone or in combination. For example, the manner A and the manner B are used in combination. The electronic device determines that the rendering time consumption of the display element is a time length range by using the manner A, and determines that the rendering time consumption of the display element is another time length range by using the manner B. The electronic device can determine a time length in the overlapping range of the two time length ranges as the rendering time consumption of the display element.

[0129] After the electronic device evaluates the rendering time consumption of each display element in the to-be-displayed interface, in one possible case, the rendering time consumption of each display element does not exceed one rendering frame. In another possible case, the rendering time consumption of one or more display elements is greater than one rendering frame. For such display elements, they can be further split, for example, the display element with the rendering time consumption greater than one rendering frame is split into multiple parts, and the rendering time consumption of each part does not exceed one rendering frame. Optionally, the further splitting of the display element can be that the display element is cropped to be divided into multiple parts, and of course there can be other splitting manners, which are not limited. In order to facilitate understanding, the following mainly takes the case that the rendering time consumption of each display element does not exceed one rendering frame as an example for description.

[0130] In the embodiments of the present application, after the electronic device determines all the display elements of the interface to be displayed, the rendering tree can be generated according to all the display elements. The rendering tree can be various types of trees, such as a multi-way tree, an ordered tree, a balanced tree, etc., wherein the multi-way tree can include a binary tree, a quad tree, an octree, a hexadecatree, etc., in general, the type of the rendering tree is not limited in the embodiments of the present application.

[0131] One possible way is that one node on the rendering tree corresponds to one display element. Taking (b) in FIG. 3 as an example, the interface to be displayed has a total of 13 display elements, and the rendering tree has 13 nodes. For example, as shown in FIG. 4A, the rendering tree includes a root node. The root node can correspond to the display element Q. Since the display element Q is used to accommodate the display elements A, B and C, the root node includes three nodes, i.e., node A, node B and node C. Among them, node A corresponds to the display element A, node B corresponds to the display element B, and node C corresponds to the display element C. Since the display element C is used to accommodate the display elements C1, C2 and C3, node C as a parent node can include three child nodes, such as child node C1, child node C2 and child node C3. The child nodes C1, C2 and C3 can respectively include one or more sub-nodes as parent nodes. For example, the child node C1 includes sub-nodes C1a and C1b. The child node C2 includes sub-nodes C2a and C2b. The child node C3 includes sub-nodes C3a and C3b.

[0132] Continuing to take FIG. 4A as an example, the rendering tree presents a row-by-row structure, and each row can include one or more nodes.

[0133] In some embodiments, one line on the render tree can correspond to one layer. To facilitate understanding, first briefly describe the layers. The interface displayed by the electronic device is often formed by superimposing multiple layers, and each layer can include a part of the content, so that multiple layers are superimposed to form the final interface. A possible implementation is that the application program specifies the layers included in each application interface of the application program and the layer superimposition order in advance. For example, in (b) of FIG. 3, the display element Q is located in layer 1, the display elements A, B and C are located in layer 2, the display elements C1, C2 and C3 are located in layer 3, and the display elements C1a, C1b, C2a, C2b, C3a and C3b are located in layer 4. The interface shown in (a) of FIG. 3 is formed by superimposing the four layers. Among them, layer 1 is located at the bottom layer, layer 2 is superimposed on layer 1, layer 3 is superimposed on layer 2, and layer 4 is superimposed on layer 3. One line on the render tree corresponds to one layer, for example, compare FIG. 4A with (b) of FIG. 3. The render tree in FIG. 4A includes four lines, and (b) of FIG. 3 includes four layers, that is, the number of lines of the render tree is equal to the number of layers. Moreover, the first line on the render tree in FIG. 4A corresponds to layer 1 in (b) of FIG. 3, the second line corresponds to layer 2, the third line corresponds to layer 3, and the fourth line corresponds to layer 4.

[0134] In the embodiments of the present application, when multiple layers are superimposed, the layer located at the top is referred to as the "front layer" or the "upper layer", and the layer located at the bottom is referred to as the "rear layer" or the "lower layer". For example, in (b) of FIG. 3, layer 3 and layer 4 can be referred to as the "front layer" or the "upper layer", and layer 1 and layer 2 can be referred to as the "rear layer" or the "lower layer". Optionally, the "front layer" can include a transparent part which does not block the content of the "rear layer". Optionally, the "rear layer" can include a display element for accommodating other display elements, such as display element C; and the "front layer" can include a display element for accommodating other display elements, such as display elements C1, C2 and C3. When the "front layer" is superimposed on the "rear layer", the visual effect that the display elements C1, C2 and C3 are accommodated in the display element C can be realized. It can be understood that the display screen of the electronic device can be used to refresh each layer in turn to realize the visual effect of "superimposing multiple layers". Taking (b) of FIG. 3 as an example, assuming that the multiple layers are refreshed in turn according to the layer superimposition order, for example, the content of layer 1 is refreshed on the display screen at T1, the content of layer 1 is retained on the display screen at T2, and the content of layer 2 is further refreshed, and so on. After refreshing the four layers, the visual effect of superimposing multiple layers is realized.

[0135] In some embodiments, each row on the render tree can include one or more nodes. If multiple nodes are included in a row, the multiple nodes have a certain arrangement order. In one possible implementation, the arrangement order can be related to the display position of the display element corresponding to the node. For example, the arrangement order of the nodes in the same row from left to right is the display order of the display elements corresponding to the nodes in the interface to be displayed from top to bottom. For example, please compare FIG. 4A and FIG. 3. In FIG. 3, the display elements A, B and C are located in the same layer, i.e., layer 2, and are arranged in the order of display element A -> display element B -> display element C from top to bottom according to the display position. Therefore, in FIG. 4A, the nodes A, B and C are located in the same row, i.e., the second row, and are arranged in the order of node A -> node B -> node C from left to right. In another possible implementation, the arrangement order can also be related to the number of display elements contained in the display element corresponding to the node. For example, the arrangement order of the nodes in the same row from left to right is the order of the number of display elements contained in the display elements corresponding to the nodes from small to large. For example, in FIG. 3, the display elements A, B and C are located in the same layer, i.e., layer 2, and are arranged in the order of display element A and display element B -> display element C from small to large according to the number of display elements contained. The number of display elements contained in display element A and display element B is the same, i.e., 0, so display element A and display element B are arranged in the same position. Therefore, in FIG. 4A, the nodes A, B and C are located in the same row, i.e., the second row, and are arranged in the order of node A and node B -> node C from left to right. The order of node A and node B is not limited, but node A and node B are both arranged to the left of node C. It should be noted that the arrangement order of the nodes in the same row can also have other implementations, which are not listed one by one.

[0136] The above is the description of the render tree. In order to distinguish, each node in the render tree can have an identifier. For example, the identifier can be a data group, which can be in the form of (X, Y) for example. Wherein, X represents the row where the node is located, X is a positive integer, Y represents the column where the node is located, i.e., represents the order of the node in the row, Y is a positive integer. Taking node A in FIG. 4A as an example, node A is located in the first position in the second row, so the identifier of node A is (2, 1); taking node C3 as an example, node C3 is located in the third position in the third row, so the identifier of node C3 is (3, 3).

[0137] As described above, a row on the render tree corresponds to a layer, so the value of X of the node identifier represents the row where the node is located, and can also represent the layer where the display element corresponding to the node is located. For example, please compare FIG. 4A and (b) in FIG. 3. The value of X of the root node is 1, i.e., the display element Q corresponding to the root node is located in layer 1, and the value of X of node A is 2, i.e., the display element A corresponding to node A is located in layer 2.

[0138] As mentioned above, if a line on the render tree includes multiple nodes, the multiple nodes have a certain arrangement order, for example, the arrangement order is related to the display position of the display elements corresponding to the nodes. Therefore, the Y value of the node identifier represents the column where the node is located, and can also represent the display position of the display element corresponding to the node. For example, please compare FIG. 4A and (b) in FIG. 3. In FIG. 4A, the Y values of the nodes A, B and C in the second row represent the display positions of the display elements corresponding to the nodes A, B and C in the layer 2. For example, the larger the Y value is, the lower the display position is. The Y value of the node C is the largest, so the display position of the display element corresponding to the node C is the lowest.

[0139] Another possible way is that one node on the render tree corresponds to multiple display elements. For example, (b) in FIG. 3, the total number of display elements of the interface to be displayed is 13, and the 13 display elements can be divided into N sets, so the render tree corresponds to N nodes. It should be noted that the total rendering time of the display elements in each set in the N sets does not exceed one rendering frame. In this way, the electronic device needs to determine which display elements are a set (this process can also be referred to as a display element aggregation process). One possible way is that the electronic device can determine which display elements are a set according to the display positions of the display elements. For example, multiple display elements close to each other can be a set. Alternatively, for display elements with attributes of container, row or column, the display element and one or more display elements contained in the display element can be a set. Another possible way is that the electronic device can determine which display elements are a set according to the layers where the display elements are located. For example, multiple display elements located in the same layer can be a set. Of course, there can be other aggregation methods, which are not listed one by one.

[0140] For example, as shown in FIG. 4B, display elements A and B correspond to one node, which is denoted as node A+B. It should be noted that node A+B is one node, but corresponds to two display elements, i.e., display element A and display element B. Continuing with FIG. 3 as an example, display elements C1 and C2 are located in the same layer and are relatively close in position, so display elements C1 and C2 are taken as a set, corresponding to one node on the rendering tree, such as node C1+C2 in FIG. 4B. Similarly, node C1+C2 is one node, but corresponds to two display elements, i.e., display element C1 and display element C2. Continuing with FIG. 3 as an example, display elements C1a and C1b are located in the same layer and are relatively close in position, so they are taken as a set, corresponding to one node on the rendering tree, such as node C1a+C1b in FIG. 4B. Display elements C2a and C2b are located in the same layer and are relatively close in position, so they are taken as a set, corresponding to one node on the rendering tree, such as node C2a+C2b in FIG. 4B. Display elements C3a and C3b are located in the same layer and are relatively close in position, so they are taken as a set, corresponding to one node on the rendering tree, such as node C3a+C3b in FIG. 4B.

[0141] Continuing with FIG. 4B as an example, the rendering tree presents a row-by-row structure, and each row can include one or more nodes.

[0142] In some embodiments, one row on the rendering tree in FIG. 4B can correspond to one layer. For example, the rendering tree in FIG. 4B includes four rows, and the (b) in FIG. 3 includes four layers, i.e., the number of rows of the rendering tree is equal to the number of layers. Moreover, the first row on the rendering tree in FIG. 4B corresponds to layer 1 in the (b) in FIG. 3, the second row corresponds to layer 2, the third row corresponds to layer 3, and the fourth row corresponds to layer 4.

[0143] In some embodiments, each row on the rendering tree in FIG. 4B can include one or more nodes. If multiple nodes are included in a row, the multiple nodes have a certain arrangement order. In one possible implementation, the arrangement order can be related to the display position of the display element set corresponding to the nodes. For example, the arrangement order of the nodes in a row from left to right is the display order of the display element set corresponding to the nodes in the interface to be displayed from top to bottom. For example, please compare FIG. 4B with FIG. 3. In FIG. 3, display elements A and B are in a set (denoted as display elements A+B hereinafter) and display element C are in the same layer, i.e., layer 2, and are arranged in the display position from top to bottom as display elements A+B—> display element C. Therefore, in FIG. 4B, nodes A+B and node C are in the same row, i.e., the second row, and the nodes are arranged from left to right as node A+B—> node C. In another possible implementation, the arrangement order can also be related to the number of display elements contained in the display element set corresponding to the nodes. For example, the arrangement order of the nodes in a row from left to right is the order of the number of display elements contained in the display element set corresponding to the nodes from small to large. For example, in FIG. 3, display elements A+B and display element C are in the same layer, i.e., layer 2, and are arranged in the number of display elements contained from small to large as display elements A+B—> display element C. Therefore, in FIG. 4B, nodes A+B and node C are in the same row, i.e., the second row, and the nodes are arranged from left to right as node A+B—> node C. It should be noted that the arrangement order of the nodes in a row can also have other implementations, which are not listed one by one.

[0144] Each node in the rendering tree in FIG. 4B can also correspond to an identifier. For example, the identifier can be a data group, which can be in the form of (X, Y), for example. Wherein X represents the row where the node is located, X is a positive integer, and Y represents the column where the node is located, i.e., represents the order of the node in the row, Y is a positive integer. Taking node A+B in FIG. 4B as an example, node A+B is located in the first column in the second row, and the identifier of node A+B is (2, 1).

[0145] As described above, a row on the rendering tree in FIG. 4B corresponds to a layer, so the value of X in the node identifier represents the row where the node is located, and can also represent the layer where the display element set corresponding to the node is located. For example, please compare (b) in FIG. 4B with FIG. 3. The value of X of node A+B is 2, i.e., the set of display elements A and B is located in layer 2.

[0146] As described above, one line in the rendering tree in FIG. 4B can include multiple nodes, which have a certain arrangement order, for example, the arrangement order is related to the display position of the display element set corresponding to the nodes. Therefore, the Y value of the node identifier represents the column where the node is located, and can also represent the display position of the display element set corresponding to the node. For example, please compare FIG. 4B with (b) in FIG. 3, in FIG. 4B, the Y values of the nodes A+B and C in the second line represent the display positions of the display element set composed of the display elements A and B and the display element C in layer 2. For example, the larger the Y value is, the lower the display position is. The Y value of the node C is the largest, so the display position of the display element C is the lowest.

[0147] The difference between the two modes is that in the first mode, one display element corresponds to one node of the rendering tree, and if the number of display elements is large, the number of nodes of the rendering tree is large, and the rendering tree is relatively complex. In the second mode, multiple display elements correspond to one node of the rendering tree, so the number of nodes of the rendering tree is relatively small, and the rendering tree is relatively simplified. The electronic device can use any of the two modes, which is not limited. No matter which mode is used, the rendering time of each node on the rendering tree does not exceed one rendering frame. In order to facilitate understanding, the first mode is taken as an example in the following description, that is, the rendering tree shown in FIG. 4A is taken as an example.

[0148] After the electronic device generates the rendering tree (for example, the rendering tree shown in FIG. 4A), the rendering can be performed according to the rendering tree, and the rendering mode can include multiple modes.

[0149] The first rendering mode is to render only one node in one rendering frame.

[0150] As described above, the rendering time of each node on the rendering tree does not exceed one rendering frame, so if only one node is rendered in one rendering frame, there will be no frame loss. As mentioned earlier, one node can correspond to one display element or a group of display elements. If one node corresponds to one display element, one display element is rendered in one rendering frame. If one node corresponds to a group of display elements, a group of display elements is rendered in one rendering frame. It can be understood that if the number of nodes on the rendering tree is large, the electronic device can render each node in a certain order.

[0151] The first order is to sort according to the X value of each node. As described above, the identification of each node in the rendering tree is (X, Y), X represents the row where the node is located, and Y represents the column where the node is located. This rendering order is to render according to the X value. For example, the order can be sorted according to the X value from small to large, that is, rendering from the first row to the end of the last row; or, it can also be sorted according to the X value from large to small, that is, rendering from the last row to the end of the first row; of course, it can also be rendered from a certain row in the middle to the end of all rows.

[0152] It can be understood that there can be multiple nodes with the same X value, for example, multiple nodes located in the same row, and the X values of these nodes are the same. In this case, the electronic device can render according to the Y value of each node. For example, it can be sorted according to the Y value from small to large, that is, rendering from the first column to the end of the last column (i.e. from left to right); or, it can also be sorted according to the Y value from large to small, that is, rendering from the last column to the end of the first column (i.e. from right to left); of course, it can also be rendered from a certain column in the middle to the end of all columns.

[0153] Continuing to take FIG. 4A as an example, the first row is the root node, including display element Q; the second row is the child node of the root node, including display elements A, B and C; the third row is the child node of display element C, including display elements C1, C2 and C3; the fourth row is the end node, including display elements C1a, C1b, C2a, C2b, C3a and C3b. Assuming that the electronic device renders from the first row to the end of the last row (i.e. the fourth row), and if a row includes multiple nodes, it renders in the order from left to right, then the rendering order is: display element Q -> A -> B -> C -> C1 -> C2 -> C3 -> C1a -> C1b -> C2a -> C2b -> C3a -> C3b.

[0154] The second order is to sort according to the Y value of each node. As described above, the identification of each node in the rendering tree is (X, Y), X represents the row where the node is located, and Y represents the column where the node is located. This rendering order is to render according to the Y value. For example, the order can be sorted according to the Y value from small to large, that is, rendering from the first column to the end of the last column; or, it can also be sorted according to the Y value from large to small, that is, rendering from the last column to the end of the first column; of course, it can also be rendered from a certain column in the middle to the end of all columns.

[0155] It can be understood that there can be multiple nodes with the same Y value, for example, multiple nodes are located in the same column. In this case, the electronic device can render according to the size of the X value of each node. For example, it can be sorted in ascending order of X value, that is, rendering from the first row to the last row (that is, from top to bottom); or it can also be sorted in descending order of X value, that is, rendering from the last row to the first row (that is, from bottom to top); of course, it can also be rendered from a certain row in the middle to the end of all rows.

[0156] Continuing to take FIG. 4A as an example, the first column includes the root node, node A, C1, C1a; the second column includes node B, C2, C1b; the third column includes node C, C3, C2a; the fourth column includes node C2b; the fifth column includes node C3a; and the sixth column includes node C3b. Assuming that the electronic device renders from the first column to the end of the last column, and if a column includes multiple nodes, it is rendered in order from top to bottom, then the rendering order is: display element Q -> A -> C1 -> C1a -> B -> C2 -> C1b -> C -> C3 -> C2a -> C2b -> C3a -> C3b.

[0157] The above lists two rendering orders in the first rendering mode. It can be understood that in addition to the above two orders, there can be other rendering orders, which are not limited by the embodiments of the application.

[0158] The second rendering mode can render multiple nodes in one rendering frame.

[0159] The difference from the first rendering mode described above is that: in the first rendering mode, only one node is rendered in each rendering frame (for example, 8ms), and if the number of nodes is large, many rendering frames are needed to render a complete interface; in the second rendering mode, multiple nodes can be rendered in each rendering frame, so that the number of rendering frames needed to render a complete interface can be reduced.

[0160] In the second rendering mode, the electronic device can render multiple nodes in one rendering frame. One possible way is that the electronic device groups all the nodes on the rendering tree to obtain multiple groups of nodes, each group of nodes can include one or more nodes, and renders one group of nodes in one rendering frame. In order to avoid frame loss, when grouping the nodes, the total rendering time of all the nodes in each group of nodes does not exceed one rendering frame (for example, 8 ms). For example, one possible grouping way is that the electronic device determines the rendering time of each node, and then groups the nodes according to the rendering time of each node to ensure that the total rendering time of all the nodes in each group of nodes does not exceed one rendering frame. As mentioned earlier, one node can correspond to one display element, or can correspond to a group of display elements. If one node corresponds to one display element, the rendering time of the node is the rendering time of the corresponding display element, and the evaluation method of the rendering time of each display element has been described above and will not be repeated. If one node corresponds to a group of display elements, the rendering time of the node is equal to the sum of the rendering time of each display element in the group of display elements, and the evaluation method of the rendering time of each display element has been described above and will not be repeated. In summary, the electronic device can determine the rendering time of each node on the rendering tree.

[0161] The following describes the process of grouping the nodes according to the rendering time of each node. It can be understood that the nodes need to be traversed to each node, and the following describes two traversal methods.

[0162] The first traversal method is to traverse according to the size of the X value of each node.

[0163] As described above, the identification of each node in the rendering tree is (X, Y), X represents the row where the node is located, and Y represents the column where the node is located. This traversal method is to traverse according to the size of the X value. For example, the X value is traversed from small to large, that is, from the first row to the last row; or, the X value can also be traversed from large to small, that is, from the last row to the first row; of course, it can also be from a certain row in the middle to traverse all the rows.

[0164] If the X values of multiple nodes are the same, for example, multiple nodes are located in the same row. In this case, the Y value of each node can be traversed according to the size of the Y value. For example, the Y value can be traversed from small to large, that is, from the first column to the last column (from left to right); or, the Y value can also be traversed from large to small, that is, from the last column to the first column (from right to left); of course, it can also be from a certain column in the middle to traverse all the columns.

[0165] For example, as shown in (a) of FIG. 5, the rendering time consumption of each node of the rendering tree is marked on the node. Taking the electronic device traversing from the first row as an example, and if a plurality of nodes are included in a row, the traversal is in the order from left to right, and the grouping process includes:

[0166] I. The first row

[0167] For example, as shown in (a) of FIG. 5, the first row includes a root node. Assuming that the rendering time consumption of the root node is 4 ms, and taking 8 ms as one rendering frame, it is illustrated that the root node is rendered in one rendering frame, and 4 ms is left. The electronic device can traverse the next row to determine whether there is a node with rendering time consumption less than or equal to 4 ms in the next row.

[0168] II. The second row

[0169] When traversing the second row, the traversal can be in the order from left to right. As shown in (a) of FIG. 5, the first node in the second row is node A, and the rendering time consumption of node A is 2 ms, so node A and the root node can be divided into a group, and the total rendering time consumption of the two nodes in the group is 6 ms, that is, 2 ms is left, and the next node in the second row, that is, node B, can be continuously traversed. Since the rendering time consumption of node B is 2 ms, node B, node A and the root node can be divided into a group, and the total rendering time consumption of the three nodes in the group is 8 ms, which occupies one rendering frame, so the nodes in the group are determined. For example, as shown in (b) of FIG. 5, the root node, node A and node B are divided into a group, and the group is denoted as rendering task 1 (task 1).

[0170] In the above example, node A and node B in the second row have been divided into Task 1, and node C is left, so the traversal is continued from node C. Since the rendering time consumption of node C is 3 ms, it is illustrated that node C is rendered in one rendering frame, and 5 ms is left. The electronic device can traverse the next row to determine whether there is a node with rendering time consumption less than or equal to 5 ms in the next row.

[0171] III. The third row

[0172] When traversing the third row, the traversal can be performed in the order from left to right. As shown in (a) of FIG. 5, the first node in the third row is node C1, and the rendering time of node C1 is 2 ms. Therefore, node C1 and node C can be divided into a group, and the total rendering time of the two nodes in the group is 5 ms, i.e., there is still 3 ms left, and the next node in the third row, i.e., node C2, can be further traversed. Since the rendering time of node C2 is 2 ms, node C2, node C1 and node C can be divided into a group, and the total rendering time of the three nodes in the group is 7 ms, and there is still 1 ms left, and the next node in the third row, i.e., node C3, can be further traversed. Since the rendering time of node C3 is 4 ms, node C3 cannot be divided into a group with node C2, C1 and C. At this time, the third row has been traversed, but there is still 1 ms left, and there are two ways to handle it.

[0173] In the first way, the next row (i.e., the fourth row) is traversed to find whether there is a node with a rendering time less than or equal to 1 ms in the next row, and if there is, the node is divided into a group with node C, C1 and C2. This way can make full use of a rendering frame and ensure that there is no idle time in a rendering frame as much as possible. When traversing the next row, i.e., the fourth row, the traversal can be performed in the order from left to right until a node with a rendering time less than or equal to 1 ms is found. Since the fourth row is the last row, if there is no node with a rendering time less than or equal to 1 ms in the fourth row, node C, C1 and C2 are only taken as a group. Of course, if the fourth row is not the last row, the next row can be further traversed, and the principle is the same. Therefore, through this way, the total rendering time of each group can be as close to a rendering frame (e.g., 8 ms) as possible.

[0174] In the second way, the next row (i.e., the fourth row) is not traversed, and node C2, C1 and C are only taken as a group. That is, even if the next row includes a node with a rendering time less than or equal to 1 ms, the node will not be divided into a group with node C2, C1 and C. This is because the nodes in the next row are child nodes of node C3, and the child nodes will not participate in grouping before the parent node in the case that node C3 has not been grouped as a parent node. Through this way, the display elements corresponding to the parent nodes can be rendered preferentially, and then the display elements corresponding to the child nodes can be rendered.

[0175] Therefore, before using the second mode, the electronic device needs to determine that there is no child node of the node C, C1 or C2 in the next row (i.e., the fourth row). Of course, if there is a child node of the node C, C1 or C2 in the next row, the first mode can be used to find a node with a rendering time less than or equal to 1 ms in the child node. That is, if a node is to be added in the group consisting of the nodes C, C1 and C2, a child node of the node C, C1 or C2 in the next row needs to be added, and it needs to be ensured that the total rendering time of the nodes in the group does not exceed 8 ms. It can be understood that, since the display position of the display element corresponding to the child node is located in the display area of the display element corresponding to the parent node, in this way, the display element corresponding to the parent node can be preferentially rendered, and then the display element corresponding to the child node can be rendered.

[0176] The electronic device can select the first mode or the second mode. One selection mode is that the electronic device can determine whether to use the first mode or the second mode according to the length of the remaining time. For example, when the remaining time is relatively long (for example, the remaining time is greater than a first threshold), the first mode can be used; when the remaining time is relatively short (for example, the remaining time is less than or equal to the first threshold), the second mode can be used. Optionally, the first threshold can be preconfigured or set by a user, which is not limited. Optionally, the first threshold can be less than or equal to the rendering time of the smallest display element in the interface to be displayed. The smallest display element can be understood as a display element with the shortest rendering time. The evaluation process of the rendering time of each display element has been described above, and is not repeated here. Taking the rendering time of the smallest display element as 1 ms as an example, if the remaining time of a group is greater than or equal to 1 ms, it indicates that the smallest display element can still be accommodated in the group, and then the first mode can be used to continue traversal to find the smallest display element; if the remaining time of a group is less than 1 ms, so the group cannot accommodate even the smallest display element, and then the second mode does not need to be used.

[0177] Taking the second mode used by the electronic device as an example, only the nodes C2, C1 and C are taken as a group. As shown in (b) of FIG. 5, the nodes C, C1 and C2 are divided into a group, which is denoted as a rendering task 2 (task 2).

[0178] In the above example, the nodes C1 and C2 in the third row have been divided into the task 2, and there is still a node C3, so the traversal continues from the node C3. Since the rendering time of the node C3 is 4 ms, it indicates that the node C3 can be rendered in one rendering frame, and there is still 4 ms left. The electronic device can traverse the next row to determine whether there is a node with a rendering time less than or equal to 4 ms in the next row.

[0179] Four, the fourth row

[0180] When traversing the fourth row, the traversal can be performed in a left-to-right order. As shown in (a) of FIG. 5, the first node in the fourth row is node C1a, and the rendering time of node C1a is 1 ms. Node C3 and node C1a can be divided into a group, and the total rendering time of the two nodes in the group is 5 ms, i.e., there is still 3 ms left, and the next node in the fourth row, i.e., node C1b, can be traversed. Since the rendering time of node C1b is 2 ms, node C3, C1a, and C1b can be divided into a group, and the total rendering time of the three nodes in the group is 7 ms, and there is still 1 ms left, and the next node in the fourth row, i.e., node C2a, can be traversed. Since the rendering time of node C2a is 2 ms, node C2a cannot be divided into a group with node C3, C1a, and C1b. The electronic device can continue to traverse the other nodes in the fourth row. Assuming that the rendering time of the other nodes is 2 ms, the electronic device divides node C3, C1a, and C1b into a group. For example, as shown in (b) of FIG. 5, node C3, C1a, and C1b are divided into a group, which is denoted as rendering task 3 (task 3).

[0181] In the above example, nodes C1a and C1b in the fourth row have been divided into task 3, and there are still four nodes, i.e., nodes C2a, C2b, C3a, and C3b, so the traversal can start from node C2a. Since the rendering time of node C2a is 2 ms, it means that node C2a is rendered in one rendering frame, and there is still 6 ms left. The next node in the fourth row, i.e., node C2b, can be traversed. Since the rendering time of node C2b is 2 ms, node C2b can be divided into a group with node C2a, and the total rendering time of the two nodes in the group is 4 ms, and there is still 4 ms left, and the next node in the fourth row, i.e., node C3a, can be traversed. Since the rendering time of node C3a is 2 ms, node C3a can be divided into a group with node C2b and C2a, and the total rendering time of the three nodes in the group is 6 ms, and there is still 2 ms left, and the next node in the fourth row, i.e., node C3b, can be traversed. Since the rendering time of node C3b is 2 ms, node C3b can be divided into a group with node C3a, C2b, and C2a, and the total rendering time of the four nodes in the group is 8 ms, which occupies one rendering frame, and the traversal ends. For example, as shown in (b) of FIG. 5, nodes C2a, C2b, C3a, and C3b are divided into a group, which is denoted as rendering task 4 (task 4).

[0182] The second traversal manner is to traverse according to the Y value of each node.

[0183] As described above, the identification of each node in the render tree is (X, Y), X represents the row where the node is located, and Y represents the column where the node is located. This traversal manner is to traverse according to the value of Y. For example, the value of Y is traversed from small to large, that is, starting from the first column to the last column; or, it can also be traversed from large to small, that is, starting from the last column to the first column; of course, it can also start from a certain column in the middle and traverse all columns.

[0184] If the values of Y of multiple nodes are the same, for example, multiple nodes are located in the same column. In this case, each node can be traversed according to the value of X. For example, the value of X can be traversed from small to large, that is, starting from the first row to the last row (that is, from top to bottom); or, it can also be traversed from large to small, that is, starting from the last row to the first row (that is, from bottom to top); of course, it can also start from a certain row in the middle and traverse all rows.

[0185] For example, as shown in (a) of FIG. 6, the rendering time consumption of each node in the render tree is marked. Taking the example that the electronic device starts to traverse from the first column, and if a column includes multiple nodes, the nodes are traversed in the order from top to bottom, then the grouping process includes:

[0186] I. The first column

[0187] For example, as shown in (a) of FIG. 6, the first column includes a root node, a node A, a node C1 and a node C1a. It is assumed that the rendering time consumption of the root node is 4 ms, which means that the root node is rendered in a rendering frame, and there are still 4 ms left. The electronic device can traverse other nodes in the first column to find nodes with rendering time consumption less than 4 ms. For example, the rendering time consumption of node A is 2 ms, so node A can be divided into a group with the root node, and the total rendering time consumption of the two nodes in the group is 6 ms, and there are still 2 ms left, so the electronic device can continue to traverse other nodes in the first column, for example, the rendering time consumption of node C1 is 2 ms, so node C1 can be divided into a group with the root node and node A, and the total rendering time consumption of the three nodes in the group is 8 ms, which occupies a rendering frame, so as shown in (b) of FIG. 6, the root node, node A and C1 are divided into a group, and the group is denoted as rendering task 1 (Task 1).

[0188] In the above example, the first column includes four nodes: a root node, a node A, a node C1 and a node C1a. Among them, the root node, node A and C1 are divided into a group, and there is still node C1a left, so the traversal continues from node C1a. Since the rendering time consumption of node C1a is 1 ms, it means that node C1a is rendered in a rendering frame, and there are still 7 ms left, at this time, since the first column has been traversed, the next column can be traversed.

[0189] Second column

[0190] As shown in (a) of FIG. 6, the second column includes node B, node C2 and node C1b. When traversing the second column, it can be traversed in order from top to bottom. For example, as shown in (a) of FIG. 6, the first node of the second column is node B, and the rendering time consumption of node B is 2 ms, so node B and node C1a can be divided into a group, and the total rendering time consumption of the two nodes in the group is 3 ms, i.e. there are still 5 ms left, and the next node in the second column, i.e. node C2, can be continued to be traversed. Since the rendering time consumption of node C2 is 2 ms, node C2, node B and node C1a can be divided into a group, and the total rendering time consumption of the three nodes in the group is 5 ms, and there are still 3 ms left, and the next node in the second column, i.e. node C1b, can be continued to be traversed. Since the rendering time consumption of node C1b is 2 ms, node C1b and node C2, B and C1a can be divided into a group, and the total rendering time consumption of the four nodes in the group is 7 ms, and there is still 1 ms left, and at this time there are two ways since the second column has been traversed.

[0191] The first way is to traverse the next column (i.e. the third column) to find whether there is a node with rendering time consumption less than or equal to 1 ms in the next column, and if there is, it is divided into a group with node C1b, C2, B and C1a. This way can make full use of a rendering frame and try to ensure that there is no idle time in a rendering frame. When traversing the next column, i.e. the third column, it can be traversed in order from top to bottom until a node with rendering time consumption less than or equal to 1 ms is found. If there is no node with rendering time consumption less than or equal to 1 ms in the third column, the next column is continued to be traversed, and the same is true for all columns. If no node with rendering time consumption less than 1 ms is found after traversing all columns, only node C1b, C2, B and C1a are divided into a group.

[0192] The second way is not to traverse other columns, but only node C1b, C2, B and C1a are divided into a group. That is, even if the next column includes a node with rendering time consumption less than or equal to 1 ms, it will not be divided into a group with node C1b, C2, B and C1a.

[0193] Taking the second way as an example, i.e. node C1b, C2, B and C1a are divided into a group. As shown in (b) of FIG. 6, node C1b and node C2, B and C1a are divided into a group, which is denoted as rendering task 2 (task 2).

[0194] In the above example, the second column has been traversed, and the next column, i.e. the third column, can be traversed.

[0195] Third column

[0196] As shown in (a) of FIG. 6, the third column includes nodes C, C3 and C2a. When traversing the third column, the nodes can be traversed in order from top to bottom. As shown in (a) of FIG. 6, the first node of the third column is node C, and the rendering time consumption of node C is 3 ms, which indicates that node C is rendered within one rendering frame, and there are 5 ms left. The electronic device can continue to traverse the next node of the third column, i.e., node C3. Since the rendering time consumption of node C3 is 4 ms, node C3 and node C can be divided into a group, and the total rendering time consumption of the two nodes in the group is 7 ms, and there is 1 ms left. The electronic device can continue to traverse the next node of the third column, i.e., node C2a. Since the rendering time consumption of node C2a is 2 ms, node C2a cannot be divided into a group with node C and node C3. At this time, the third column has been traversed, but there is 1 ms left, and there are two ways to handle it.

[0197] In the first way, the next column (i.e., the fourth column) is traversed to find whether there is a node with rendering time consumption less than or equal to 1 ms in the next column. If there is, the node is divided into a group with node C and node C3. This way can make full use of one rendering frame and ensure that there is no idle time within one rendering frame as much as possible. When traversing the next column, i.e., the fourth column, the nodes can be traversed in order from top to bottom until a node with rendering time consumption less than or equal to 1 ms is found. If there is no node with rendering time consumption less than or equal to 1 ms in the fourth column, the next column is continued to be traversed, and the same is true for all columns. If no node with rendering time consumption less than 1 ms is found after traversing all columns, only node C and node C3 are taken as a group.

[0198] In the second way, only node C and node C3 are taken as a group without traversing other columns. That is, even if the next column includes a node with rendering time consumption less than or equal to 1 ms, the node will not be divided into a group with node C and node C3.

[0199] Taking the second way as an example, node C and node C3 are taken as a group. As shown in (b) of FIG. 6, node C and node C3 are divided into a group, which is denoted as rendering task 3 (task 3).

[0200] In the above example, node C and node C3 in the third column have been divided into Task 3, and node C2a is left. Node C2a can be traversed from. Since the rendering time consumption of node C2a is 2 ms, i.e., there are 6 ms left within one rendering frame, the electronic device can traverse the next column, i.e., the fourth column, to determine whether there is a node with rendering time consumption less than 2 ms in the fourth column.

[0201] IV. The fourth column

[0202] As shown in (a) of FIG. 6, the fourth column only includes node C2b. Since the rendering time consumption of node C2b is 2 ms, node C2b can be grouped with node C2a as a group, the total rendering time consumption of the two nodes in the group is 4 ms, 4 ms is left, and the next column can be further traversed.

[0203] V. Fifth Column

[0204] As shown in (a) of FIG. 6, the fifth column only includes node C3a. Since the rendering time consumption of node C3a is 2 ms, node C3a can be grouped with nodes C2a and C2b as a group, the total rendering time consumption of the three nodes in the group is 6 ms, 2 ms is left, and the next column can be further traversed.

[0205] VI. Sixth Column

[0206] As shown in (a) of FIG. 6, the sixth column only includes node C3b. Since the rendering time consumption of node C3b is 2 ms, node C3b can be grouped with nodes C3a, C2a and C2b as a group, the total rendering time consumption of the four nodes in the group is 8 ms, which fills a rendering frame, and the traversal of the rendering tree is completed. For example, as shown in (b) of FIG. 6, nodes C3b, C3a, C2a and C2b are divided into a group, which is denoted as rendering task 4 (Task4).

[0207] The above illustrates two traversal modes of the rendering tree, and there can be other traversal modes, which are not illustrated one by one. In order to facilitate understanding, the first traversal mode is mainly taken as an example for illustration in the following embodiments of the present disclosure, that is, the nodes on the rendering tree are divided into four rendering tasks, that is, four Tasks, as shown in (b) of FIG. 5.

[0208] It should be noted that, as known from the foregoing, there is no repeated node in any two of the four Tasks, so that repeated rendering of the nodes can be avoided. As mentioned above, a node can correspond to one display element or a group of display elements. There is no repeated node in any two of the Tasks, that is, there is no repeated display element in any two of the Tasks.

[0209] Continuing with the example of (b) in FIG. 5, after the nodes on the render tree are divided into four Tasks, the electronic device can render the four Tasks in turn, and since each Task can include multiple nodes, for each Task, the electronic device can render the nodes in the Task in turn. Therefore, in the embodiments of the present application, the rendering order can include: 1) the rendering order of the nodes in a Task; 2) the rendering order between multiple Tasks. It can be understood that the nodes or Tasks that are rendered first will be displayed first on the display screen and will be seen first by the user. In order to ensure the user experience, the nodes in a Task can be sorted, and the nodes in a Task are rendered according to the sorting. Similarly, multiple Tasks can also be sorted, and the multiple Tasks are rendered according to the sorting.

[0210] In some embodiments, the electronic device can sort the nodes in a Task. As described above, a Task can include multiple nodes, and the multiple nodes can be sorted according to a certain order. The sorting method can include at least one of the following.

[0211] The first sorting method is to sort according to the traversal time of the nodes. As described above, the electronic device traverses each node on the render tree, thereby grouping the nodes. During the traversal, each time a node is traversed, the traversal time of the node is recorded. For example, taking Task1 in (b) in FIG. 5 as an example, the traversal times of the nodes in Task1 can be seen in Table 4 below.

[0212] Table 4:

[0213] Therefore, the electronic device can sort the nodes in a Task according to the traversal time of the nodes. For example, the nodes can be sorted according to the traversal time from early to late, assuming that T1 is earlier than T2, and T2 is earlier than T3, then the sorting of the nodes is root node -> node A -> node B. Alternatively, the nodes can be sorted according to the traversal time from late to early, assuming that T1 is earlier than T2, and T2 is earlier than T3, then the sorting of the nodes is node B -> node A -> root node.

[0214] The second sorting method is to sort according to the X value of the nodes. For example, the X values can be sorted from large to small, or the X values can be sorted from small to large. It can be understood that there can be multiple nodes with the same X value. For the nodes with the same X value, the Y values can be sorted, for example, the Y values can be sorted from large to small, or the Y values can be sorted from small to large.

[0215] For example, taking Task 1 in (b) in FIG. 5 as an example, Task 1 includes a root node, a node A and a node B. Taking the X value in ascending order as an example, and for the nodes with the same X value, taking the Y value in ascending order as an example, the order of the nodes in Task 1 is: the root node -> the node A -> the node B, that is, the rendering order of the nodes in Task 1 is: the root node -> the node A -> the node B. It can be understood that Task 1 includes multiple nodes in the same row, for example, the node A and the node B. As described above, the display elements corresponding to the nodes in the same row are located in the same layer, that is, the display element A corresponding to the node A and the display element B corresponding to the node B are located in the same layer. Since the rendering order of the node A and the node B in Task 1 is: the node A -> the node B. Therefore, the order of the nodes in the same row in Task 1 is related to the display positions of the display elements corresponding to the nodes. For example, the display position of the display element A corresponding to the node A is at the top (the Y value is small), and the display position of the display element B corresponding to the node B is at the bottom (the Y value is large), so the node A is rendered first and the node B is rendered later, that is, the display element A is rendered first and the display element B is rendered later, which can reflect the process of rendering from top to bottom.

[0216] The third sorting manner is to sort according to the Y value of the node. For example, the Y value can be sorted in descending order, or the Y value can be sorted in ascending order. It can be understood that the Y values of multiple nodes can be the same. For the nodes with the same Y value, the X value can be sorted, for example, the X value can be sorted in descending order, or the X value can be sorted in ascending order.

[0217] For example, taking Task 1 in (b) in FIG. 6 as an example, Task 1 includes a root node, a node A and a node C1. Taking the Y value in ascending order as an example, and for the nodes with the same Y value, taking the X value in ascending order as an example, the order of the nodes in Task 1 is: root node -> node A -> node C1. That is, the rendering order of the nodes in Task 1 is: root node -> node A -> node C1. It can be understood that Task 1 includes multiple nodes in the same column, for example, the root node, the node A and the node C1 are in the same column. As described above, the display elements corresponding to the nodes in the same column are located in different layers, that is, the display elements corresponding to the root node, the node A and the node C1 are located in different layers. For example, the display element Q corresponding to the root node is located in layer 1, the display element A corresponding to the node A is located in layer 2, and the display element C1 corresponding to the node C1 is located in layer 3. Since the rendering order of the nodes in Task 1 is: root node -> node A -> node C1, that is, the order of the nodes in the same column in Task 1 is related to the layer order of the display elements corresponding to the nodes, that is, the display elements in layer 1 are rendered first, the display elements in layer 2 are rendered second, and the display elements in layer 3 are rendered last.

[0218] In some other embodiments, after the electronic device obtains multiple Tasks, the electronic device can sort the multiple Tasks. The sorting manner can include at least one of the following.

[0219] The first sorting manner is to sort according to the generation time of each Task. As described above, the electronic device traverses each node on the rendering tree, thereby grouping the nodes. During the grouping process, the generation time of each group (i.e., a Task) is recorded. For example, the generation times of the four Tasks in (b) in FIG. 5 can be seen from Table 5 below.

[0220] Table 5:

[0221] Therefore, the electronic device can sort the Tasks according to the generation time of the Tasks. For example, the Tasks are sorted in the order from early to late according to the generation time, assuming that T1 is earlier than T2, T2 is earlier than T3, and T3 is earlier than T4, then the order of the Tasks is Task 1 -> Task 2 -> Task 3 -> Task 4. Or, the Tasks are sorted in the order from late to early according to the generation time, assuming that T1 is earlier than T2, T2 is earlier than T3, and T3 is earlier than T4, then the order of the Tasks is Task 4 -> Task 3 -> Task 2 -> Task 1.

[0222] The second sorting manner is to sort the Tasks according to the average value of X corresponding to each Task. As described above, a Task can include multiple nodes, and the electronic device can calculate the average value of X of all nodes in each Task. Taking Task 1 in (b) in FIG. 5 as an example, Task 1 includes a root node, node A, and node B, and the electronic device can calculate the average value of X of these three nodes to obtain the average value of X corresponding to Task 1. Similarly, the electronic device can also obtain the average values of X corresponding to the other three Tasks. For example, the average values of X corresponding to the four Tasks in (b) in FIG. 5 can be seen from Table 6 below.

[0223] Table 6:

[0224] Therefore, the electronic device can sort the Tasks according to the average values of X of the Tasks. For example, the Tasks can be sorted in descending order of the average values of X, assuming that X1>X2>X3>X4, the order of the Tasks is Task 1—>Task 2—>Task 3—>Task 4. Alternatively, the Tasks can be sorted in ascending order of the average values of X, assuming that X1>X2>X3>X4, the order of the Tasks is Task 4—>Task 3—>Task 2—>Task 1.

[0225] The second sorting manner is to sort the Tasks according to the average value of Y corresponding to each Task. As described above, a Task can include multiple nodes, and the electronic device can calculate the average value of Y of all nodes in each Task. Taking Task 1 in (b) in FIG. 5 as an example, Task 1 includes a root node, node A, and node B, and the electronic device can calculate the average value of Y of these three nodes to obtain the average value of Y corresponding to Task 1. Similarly, the electronic device can also obtain the average values of Y corresponding to the other three Tasks. For example, the average values of Y corresponding to the four Tasks in (b) in FIG. 5 can be seen from Table 7 below.

[0226] Table 7:

[0227] Therefore, the electronic device can sort the Tasks according to the average values of Y of the Tasks. For example, the Tasks can be sorted in descending order of the average values of Y, assuming that Y1>Y2>Y3>Y4, the order of the Tasks is Task 1—>Task 2—>Task 3—>Task 4. Alternatively, the Tasks can be sorted in ascending order of the average values of Y, assuming that Y1>Y2>Y3>Y4, the order of the Tasks is Task 4—>Task 3—>Task 2—>Task 1.

[0228] Continuing with the example of the four Tasks in (b) of FIG. 5, assume that the electronic device determines the order of the four Tasks to be: Task 1 -> Task 2 -> Task 3 -> Task 4, and stores the four Tasks in the rendering queue according to the order. For example, as shown in FIG. 7, the rendering queue includes Task 1, Task 2, Task 3, and Task 4. Task 1 is at the head of the queue, followed by Task 2, Task 3, and Task 4 in that order. It can be appreciated that the rendering queue satisfies the first-in-first-out principle, so the order in which the Tasks are dequeued from the rendering queue, i.e., the rendering order of the Tasks, is: Task 1 -> Task 2 -> Task 3 -> Task 4. Thus, the rendering order of the four Tasks is: Task 1 -> Task 2 -> Task 3 -> Task 4.

[0229] Continuing with the example of FIG. 7, after obtaining the rendering queue, the electronic device can render the Tasks in the rendering queue one by one. For example, one Task is rendered in one rendering frame. For example, Task 1 is rendered in rendering frame 1, Task 2 is rendered in rendering frame 2, Task 3 is rendered in rendering frame 3, and Task 4 is rendered in rendering frame 4. Thus, the electronic device needs four rendering frames to complete the rendering of one interface, and since the total time for rendering each Task does not exceed one rendering frame, there is no frame loss, and each Task occupies as much of a rendering frame as possible, so that the time utilization of the rendering frame is maximized.

[0230] Continuing with the example of FIG. 7, since the rendering order is: Task 1 -> Task 2 -> Task 3 -> Task 4, i.e., Task 1 is rendered first and Task 4 is rendered last. Task 1 includes the root node and nodes A, B. As mentioned above, the root node and nodes A, B correspond to the display elements in layer 1 and layer 2. Task 4 includes nodes C2a, C2b, C3a, C3b, which correspond to the display elements in layer 4. Relative to layer 1 and layer 2, layer 4 is a front layer. That is, the display elements in the back layer are rendered first, and the display elements in the front layer are rendered last. Moreover, the display elements in the back layer can be used to accommodate the display elements in the front layer, in other words, the display elements in the front layer can be displayed in the display area of the display elements in the back layer.

[0231] Continuing with FIG. 7 as an example, Task 1 is rendered first, and the rendering order of the nodes in Task 1 is: root node -> node A -> node B. That is, the root node is rendered first, which, as described above, corresponds to the display element Q and can be understood as the interface background. Thus, rendering Task 1 first enables the interface background to be rendered first. After the root node is rendered, the display elements in layer 1 are rendered, and then the display elements in layer 2 are rendered, such as for layer 2, the display element A corresponding to node A is rendered first, and then the display element B corresponding to node B is rendered, completing the rendering of Task 1.

[0232] Continuing with FIG. 7 as an example, after Task 1 is rendered, Task 2 is rendered. The rendering order of the nodes in Task 2 is: node C -> node C1 -> node C2. As described above, after Task 1 is rendered, only the display element C in layer 2 has not been rendered, so node C is rendered first when Task 2 is rendered. It should be noted that the display element C corresponding to node C can be understood as the background of a display area for accommodating other display elements, and rendering node C first enables the background of the area for accommodating other display elements to be rendered first. After the background of the area is rendered, the display elements accommodated in the area, i.e., the display elements C1 and C2 corresponding to nodes C1 and C2, are rendered.

[0233] Continuing with FIG. 7 as an example, after Task 2 is rendered, Task 3 is rendered. The rendering order of the nodes in Task 3 is: node C3 -> node C1a -> node C1b. As described above, after Task 2 is rendered, only the display element C3 in layer 3 has not been rendered, so node C3 is rendered first when Task 2 is rendered. After layer 3 is rendered, layer 4 is rendered, such as the display element C1a corresponding to node C1a is rendered first, and then the display element C1b corresponding to node C1b is rendered.

[0234] Continuing with FIG. 7 as an example, Task 4 is finally rendered, and the rendering order of the nodes in Task 4 is: node C2a -> node C2b -> node C3a -> node C3b. As described above, only the display elements C1a and C1b are rendered in layer 4, so other display elements in layer 4 are rendered, such as the display elements corresponding to nodes C2a, C2b, C3a, and C3b are rendered in turn. It should be noted that the display elements corresponding to nodes C2a, C2b, C3a, and C3b are display elements accommodated in other display elements and can be understood as interface details. Thus, rendering Task 4 last enables the interface details to be rendered last.

[0235] Therefore, in the embodiments of the present application, the electronic device can preferentially render the interface background, and then render the interface details, thereby embodying the effect of gradually refining the interface.

[0236] In the foregoing, the graphic rendering frame rate is taken as an example of 120 Hz, and one rendering frame is taken as an example of 8 ms. Alternatively, the graphic rendering frame rate of the electronic device can be fixed or can also be dynamically changed. If the graphic rendering frame rate is dynamically changed, the rendering frame is also dynamically changed, for example, the graphic rendering frame rate changes from 120 Hz to 60 Hz, and the rendering frame changes from 8 ms to 16 ms. Taking the dynamic change of the graphic rendering frame rate as an example, one possible way is that the electronic device can adjust the graphic rendering frame rate. Alternatively, the adjustment manner can include automatic adjustment and manual adjustment. Taking manual adjustment as an example, the electronic device adjusts the graphic rendering frame rate when receiving a user adjustment operation. For example, the electronic device can provide an entry for adjusting the graphic rendering frame rate. The display position of the entry is not limited, for example, it can be located in a certain application (for example, a settings application). Taking automatic adjustment as an example, the electronic device can automatically adjust the graphic rendering frame rate according to the device running state. Alternatively, the device running state can include at least one of the running load and the remaining power.

[0237] Taking the electronic device automatically adjusting the graphic rendering frame rate according to the running load as an example. One possible way is that the electronic device stores a corresponding relationship between the running load and the graphic rendering frame rate, and can automatically adjust the graphic rendering frame rate based on the corresponding relationship. Exemplarily, the corresponding relationship can be as shown in Table 8.

[0238] Table 8:

[0239] Therefore, the electronic device can determine the graphic rendering frame rate according to the running load and Table 8, and then automatically adjust the graphic rendering frame rate. It should be noted that the running load can be described by one or more of the CPU running speed, the memory occupancy rate, and the number of currently running applications. Taking the CPU running speed as an example, the slower the CPU running speed, the higher the running load. Taking the memory occupancy rate as an example, the higher the memory occupancy rate, the higher the running load. Taking the number of currently running applications as an example, the more the number of currently running applications, the higher the running load. It should be noted that the running load is described by the CPU running speed in Table 8.

[0240] Taking the electronic device automatically adjusting the graphic rendering frame rate according to the remaining power as an example. One possible way is that the electronic device stores a corresponding relationship between the remaining power and the graphic rendering frame rate, and can automatically adjust the graphic rendering frame rate based on the corresponding relationship. Exemplarily, the corresponding relationship can be as shown in Table 9.

[0241] Table 9:

[0242] Therefore, the electronic device can determine the graphical rendering frame rate according to the remaining power and the above Table 9, and then automatically adjust the graphical rendering frame rate.

[0243] After the electronic device adjusts the graphical rendering frame rate, the rendering frame is adjusted accordingly, for example, the rendering frame is changed from 8ms to 16ms. In this case, the electronic device processes the to-be-displayed interface according to the adjusted rendering frame (for example, 16ms). For example, the electronic device determines all display elements within the to-be-displayed element, and ensures that the rendering time consumption of each display element does not exceed 16ms. After determining all display elements within the to-be-displayed interface, a rendering tree is generated, and the rendering time consumption of each node on the rendering tree does not exceed 16ms. When traversing the rendering tree to group nodes, grouping is performed according to the rendering time consumption of each node to ensure that the total rendering time consumption of each group of nodes does not exceed 16ms. Details of these processes have been described in the foregoing (the foregoing takes 8ms as an example), and will not be repeated here.

[0244] Referring to FIG. 8, it is a structural schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device can be various electronic devices listed in the foregoing. As shown in FIG. 8, the electronic device can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0245] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors. The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions. The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can directly call from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0246] In some embodiments, the processor 110 can perform the rendering method provided in the embodiments of the present application. For example, the graphics rendering frame rate of the processor 110 is a first frame rate, and the first frame rate is used to indicate the number of renderings of the processor 110 within a first time length. After determining the first interface to be displayed, the processor 110 determines the rendering time length as a second time length according to the first frame rate, and then determines N rendering tasks of the first interface according to the second time length, the N rendering tasks are used to render all contents of the first interface, and the rendering time length of each rendering task in the N rendering tasks is less than or equal to the second time length. After that, the processor 110 completes the N rendering tasks and displays the first interface within the N second time lengths.

[0247] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0248] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can include multiple sets of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces, respectively. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface, and realize the touch function of the electronic device 100.

[0249] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple sets of I2S buses. The processor 110 can be coupled to the audio module 170 through the I2S bus, and realize the communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the I2S interface, and realize the function of answering the phone through the Bluetooth headset.

[0250] The PCM interface can also be used for audio communication, which samples, quantizes and encodes analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled through the PCM bus interface. In some embodiments, the audio module 170 can also deliver audio signals to the wireless communication module 160 through the PCM interface, and realize the function of answering the phone through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0251] The UART interface is a universal serial bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to realize the function of playing music through the Bluetooth headset.

[0252] The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display screen serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to realize the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to realize the display function of the electronic device 100.

[0253] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0254] The USB interface 130 is an interface that meets the USB standard specification, which can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0255] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0256] The wireless communication function of the electronic device can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc. The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0257] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit the processed signals to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and radiate the signals as electromagnetic waves through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 can be disposed in the same device.

[0258] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device. The wireless communication module 160 can be one or more devices integrated with at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, perform frequency modulation and amplification on the signals, and radiate the signals as electromagnetic waves through the antenna 2.

[0259] In some embodiments, the antenna 1 and the mobile communication module 150 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device can communicate with a network and other devices through wireless communication technology.

[0260] The display screen 194 is configured to display a display interface of an application, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device can include one or N display screens 194, where N is a positive integer greater than 1.

[0261] The electronic device 100 can implement a photographing function through an ISP, the camera 193, a video codec, a GPU, the display screen 194, and an application processor, etc. The ISP is configured to process data fed back by the camera 193.

[0262] The internal memory 121 can be configured to store computer executable program codes including instructions. The processor 110 executes various function applications and data processing of the electronic device by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, software codes of at least one application program, etc. The data storage area can store data (e.g., images, videos, etc.) generated during use of the electronic device, etc. In addition, the internal memory 121 can include a high-speed random access memory, and can further include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage, etc.

[0263] The external memory interface 120 can be configured to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as saving pictures, videos, etc. in the external memory card.

[0264] The electronic device can implement an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, etc.

[0265] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and is also configured to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or part of the functions of the audio module 170 can be disposed in the processor 110.

[0266] The speaker 170A, also called a "loudspeaker," converts an electrical audio signal into a sound signal. The electronic device 100 can listen to music or listen to an external speaker such as a hands-free call through one or more speakers 170A.

[0267] The receiver 170B, also called a "earpiece," can be one or more, and converts an electrical audio signal into a sound signal. When the electronic device 100 receives a call or a voice message, the receiver 170B can be held close to a person's ear to listen to the voice.

[0268] The microphone 170C, also called a "microphone," "mic," or "voice microphone," converts a sound signal into an electrical signal.

[0269] The earphone interface 170D is used to connect a wired earphone.

[0270] The pressure sensor 180A senses a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194.

[0271] The gyro sensor 180B can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for shooting a shake reduction.

[0272] The barometric sensor 180C measures air pressure. In some embodiments, the electronic device calculates altitude, assists positioning and navigation using the air pressure value measured by the barometric sensor 180C.

[0273] The magnetic sensor 180D includes a Hall sensor. The electronic device can detect the opening and closing of a flip cover using the magnetic sensor 180D.

[0274] The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device in various directions (typically, three axes). When the electronic device is stationary, the acceleration sensor 180E can detect the magnitude and direction of gravity.

[0275] The distance sensor 180F measures a distance. The electronic device can measure a distance using infrared or laser.

[0276] The proximity light sensor 180G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode can be an infrared light emitting diode. The electronic device emits infrared light outwardly through the light emitting diode. The electronic device detects infrared reflected light from a nearby object using the photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device. When insufficient reflected light is detected, the electronic device can determine that there is no object near the electronic device.

[0277] The ambient light sensor 180L is configured to sense ambient light brightness. The electronic device can adaptively adjust the display screen 194 brightness according to the sensed ambient light brightness.

[0278] The fingerprint sensor 180H is configured to collect a fingerprint.

[0279] The temperature sensor 180J is configured to detect temperature.

[0280] The touch sensor 180K, also referred to as a “touch panel”. The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 together form a touch screen, also referred to as a “touch panel”. The touch sensor 180K is configured to detect a touch operation acting on or near the touch sensor 180K. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the touch event type.

[0281] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a human body sound part vibration bone block.

[0282] The keys 190 include a power-on key, a volume key, and the like. The keys 190 can be mechanical keys. Alternatively, the keys 190 can be touch keys. The electronic device can receive key input, and generate key signal input related to user settings and function control of the electronic device. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, and can be used to indicate a charging state, a power change, and can also be used to indicate a message, a missed call, a notification, and the like. The SIM card interface 195 is configured to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device.

[0283] It can be understood that the components shown in FIG. 8 do not constitute a specific limitation on the electronic device. The electronic device in the embodiments of the present application can include more or fewer components than those in FIG. 8. In addition, the combination / connection relationship between the components in FIG. 8 can also be adjusted and modified.

[0284] FIG. 9 is a structural schematic diagram of an electronic device 900 according to an embodiment of the present application. The electronic device 900 can be any of the electronic devices listed above (e.g., a mobile phone or a watch). As shown in FIG. 9, the electronic device 900 can include one or more processors 901, one or more memories 902, a communication interface 903, and one or more computer programs 904, which can be connected through one or more communication buses 905. The one or more computer programs 904 are stored in the memory 902 and configured to be executed by the one or more processors 901, and the one or more computer programs 904 include instructions. The communication interface 903 is configured to realize communication between the electronic device 900 and other devices, and the communication interface can be a transceiver, for example.

[0285] In the above embodiments of the present application, the method provided by the embodiments of the present application is described from the perspective of the electronic device (e.g., a mobile phone or a watch) as an execution subject. In order to implement each function in the above method provided by the embodiments of the present application, the electronic device can include a hardware structure and / or a software module, and each function is implemented in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether a certain function in the above is implemented in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on specific application of the technical solution and design constraints.

[0286] In the above embodiments, the method can be implemented by software, hardware, firmware, or any combination thereof, in whole or in part. When implemented by software, the method can be implemented in the form of a computer program product, in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the above method or function is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center through wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)), etc. The solutions of the above embodiments can be combined without conflict.

[0287] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, various elements are implemented in hardware, software, or a combination of both hardware and software. In a software embodiment, the software implementation can include but is not limited to a bit stream, a machine executable file, a program, or a software application. In a software embodiment, the various elements are implemented as software programs to provide the functionality described herein. It will be apparent to those skilled in the art that substantial equivalents of the structures described herein can be substituted for the structures described herein and that suitable equivalents of the functions described herein can be substituted for the functions described herein. Therefore, it is contemplated to cover any and all adaptations of this application with the scope of the claims.

[0288] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems) and computer program products according to this application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.

[0289] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks.

[0290] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.

[0291] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A rendering method, characterized by, The method is applied to an electronic device, a frame rate of a graphic rendering of the electronic device is a first frame rate, the first frame rate is used to indicate a rendering frequency of the electronic device within a first time length, and the method comprises the following steps: determining a first interface to be displayed; determining a rendering time length as a second time length according to the first frame rate; determining N rendering tasks of the first interface according to the second time length, the N rendering tasks are used to render all contents of the first interface, a rendering time length of each rendering task in the N rendering tasks is less than or equal to the second time length, and N is a positive integer; completing the N rendering tasks within N second time lengths; displaying the first interface.

2. The method of claim 1, wherein, A difference between the rendering time length of each rendering task and the second time length is less than or equal to a first threshold value.

3. The method of claim 2, wherein, The first threshold value is less than or equal to a rendering time length of a smallest display element in the first interface.

4. The method according to any one of claims 1 to 3, characterized in that, Each rendering task in the N rendering tasks is used to render P display elements, P is a positive integer, and display elements rendered by any two rendering tasks in the N rendering tasks are not repeated.

5. The method of claim 4, wherein, The rendering order of the N rendering tasks is related to at least one of the following: a layer where the display elements corresponding to the N rendering tasks are located; display positions of the display elements corresponding to the N rendering tasks.

6. The method of claim 5, wherein, The rendering order of the N rendering tasks is related to the layer where the display elements corresponding to the N rendering tasks are located, and the rendering order of the N rendering tasks comprises: the rendering order of the N rendering tasks is that the N rendering tasks are rendered in an order from back to front according to the layers where the corresponding display elements are located.

7. The method of claim 5, wherein, The rendering order of the N rendering tasks is related to the display positions of the display elements corresponding to the N rendering tasks, and the rendering order of the N rendering tasks comprises: the rendering order of the N rendering tasks is that the N rendering tasks are rendered in an order from top to bottom or from left to right according to the display positions of the corresponding display elements.

8. The method of claim 5, wherein: a plurality of rendering tasks in the N rendering tasks correspond to display elements located in a same layer, and the plurality of rendering tasks are rendered in an order according to display positions of the corresponding display elements; or a plurality of rendering tasks in the N rendering tasks correspond to display elements located in a same position, and the plurality of rendering tasks are rendered in an order according to layers where the corresponding display elements are located. The rendering order of the P display elements is related to at least one of the following:

9. The method of claim 4, wherein, layers where the P display elements are located; display positions of the P display elements. The rendering order of the P display elements is related to the layers where the P display elements are located, and the rendering order of the P display elements comprises:

10. The method of claim 9, wherein, the rendering order of the P display elements is that the P display elements are rendered in an order from back to front according to the layers where the P display elements are located. The rendering order of the P display elements is related to the display positions of the P display elements, and the rendering order of the P display elements comprises:

11. The method of claim 9, wherein, the rendering order of the P display elements is that the P display elements are rendered in an order from top to bottom or from left to right according to the display positions of the P display elements.

12. The method of claim 9, wherein: a plurality of display elements in the P display elements are located in a same layer, and the plurality of display elements are rendered in an order according to display positions of the plurality of display elements; or a plurality of display elements in the P display elements are located in a same position, and the plurality of display elements are rendered in an order according to layers where the plurality of display elements are located. The P display elements are located at the same position, and the P display elements are sorted and rendered according to the layers.

13. The method according to any one of claims 1 to 12, characterized in that, The first rendering task in the N rendering tasks comprises a first sub-task for rendering an interface background in the first interface.

14. The method according to any one of claims 1 to 13, characterized in that, The last rendering task in the N rendering tasks comprises a second sub-task for rendering interface details in the first interface.

15. The method according to any one of claims 4-12, characterized in that, Each of the P display elements does not contain other display elements.

16. The method according to any one of claims 1 to 15, characterized in that, According to the second time length, determining N rendering tasks of the first interface, comprising: determining all display elements in the first interface; According to the second time length and the rendering time consumption of each display element in the all display elements, dividing the all display elements into N sets, the rendering time consumption of the display elements in each set of the N sets is less than or equal to the second time length, and each set corresponds to a rendering task.

17. The method of claim 16, wherein, The all display elements are divided into N display element sets, comprising: traversing according to a first traversal order to determine a first display element, the rendering time consumption of the first display element is less than the second time length, and the first traversal order is a traversal order of the all display elements; According to the rendering time consumption of the first display element, continue to traverse according to the first traversal order to determine a second display element, the sum of the rendering time consumption of the first display element and the second display element is less than or equal to the second time length; determining that the first display element and the second display element belong to the same rendering task.

18. The method of any one of claim 17, wherein, The first display element and the second display element are located in the same layer or different layers.

19. The method of claim 18, wherein, The first display element and the second display element are located in different layers, and the first display element is used to accommodate the second display element.

20. The method according to any one of claims 17-19, characterized by, The first traversal order comprises: traversing according to a layer order, or traversing according to a display order.

21. The method of claim 20, wherein: traversing according to a layer order comprises: traversing all display elements in each layer in turn according to a layer order from back to front; traversing according to a display order comprises: traversing all display elements in turn according to a display order from top to bottom or from left to right.

22. The method of claim 16, wherein, The method further comprises: determining the rendering time consumption of each display element according to at least one of a type, a size and a style of the each display element.

23. The method of any one of claims 1-22, wherein, The method further comprises: adjusting the graphic rendering frame rate to a second frame rate, the second frame rate being used to indicate a rendering number of the electronic device within a third time length; determining a second interface to be displayed; determining a rendering time length as a fourth time length according to the second frame rate; determining M rendering tasks of the second interface according to the fourth time length, the M rendering tasks being used to render all contents of the second interface, and a rendering time length of each rendering task in the M rendering tasks being less than or equal to the fourth time length, M being a positive integer; completing the M rendering tasks within M fourth time lengths; displaying the second interface.

24. The method of claim 23, wherein, The adjusting the graphic rendering frame rate to a second frame rate comprises: According to a device running state or a user operation of the electronic device, the graphical rendering frame rate is adjusted to a second frame rate.

25. An electronic device, comprising: Comprising: a processor, a memory, and one or more programs; wherein the one or more programs are stored in the memory and, when executed by the processor, cause the electronic device to perform the steps of the methods of any of claims 1-24.

26. A computer-readable storage medium, characterized in that, The computer readable storage medium for storing a computer program which, when executed on a computer, causes the computer to perform the method of any of claims 1 to 24.

27. A computer program product, characterised in that, A computer program which, when executed on a computer, causes the computer to perform the method of any of claims 1 to 24.

Images