Rendering method and related equipment

CN119096273APending Publication Date: 2024-12-06HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202480000463.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-03-08
Filing Date
2024-03-06
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When user interface rendering tasks are processed using a serial process in electronic devices, as the screen size and the number of interface elements increase, the rendering efficiency decreases, resulting in low processing efficiency.

Method used

By determining the grouping results of frames to be displayed, the drawing tasks of the application interface are divided into drawing task groups, and multi-threading is used to process these task groups to improve rendering efficiency and ensure the correctness of the rendering results. Specific methods include splitting the rendering tree of the user interface to generate non-overlapping spatial trees, and using multi-core processors to optimize the distribution and processing of drawing tasks.

Benefits of technology

It significantly improves the rendering efficiency of frames to be displayed, ensures the correctness of the rendering results of the user interface, and effectively utilizes the capabilities of multi-core processors to avoid the problem of uneven rendering loads.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119096273A_ABST
    Figure CN119096273A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a rendering method and related equipment, and relates to the field of electronic equipment. In the method, a grouping result of a to-be-displayed frame is determined firstly, and the grouping result can enable the drawing duration of the to-be-displayed frame to be smaller than or equal to the sum of the drawing durations of at least one application interface, namely the drawing duration required by the to-be-displayed frame when grouping is not performed. The method comprises the steps of obtaining a grouping result of a to-be-displayed frame, dividing drawing tasks of the to-be-displayed frame according to the grouping result to obtain at least one drawing task group, processing the drawing tasks of the at least one drawing task group by at least one drawing thread to obtain at least one drawing result, and synthesizing the at least one drawing result to obtain a total drawing result of the to-be-displayed frame. And rendering the to-be-displayed frame. Therefore, by using the rendering method of the scheme, the grouping result of the to-be-displayed frame is determined in advance so as to realize optimal rendering of the to-be-displayed frame, and the rendering efficiency of the to-be-displayed frame can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Rendering method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 8, 2023, with application number 202310267611.1 and application name “Rendering Method and Related Equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of electronic devices, and in particular to a rendering method and related devices. Background Art

[0003] Electronic devices can provide a user interface (UI) for interacting with users. The UI includes interface elements such as icons, windows, and controls. Controls can include visual interface elements such as images, buttons, menus, tabs, text boxes, dialog boxes, status bars, and navigation bars.

[0004] User interface rendering tasks are typically handled serially. However, as rendering becomes more demanding, such as with increasing screen sizes, the number of UI elements that can be displayed increases, leading to increased rendering content. This inefficiency leads to a need to improve rendering efficiency.

[0005] Summary of the Invention

[0006] The present application provides a rendering method and related devices, which can improve the rendering efficiency of picture frames to be displayed.

[0007] In a first aspect, a rendering method is provided.

[0008] The method includes the following steps: determining a grouping result for frames to be displayed. The frames to be displayed are used to display at least one application interface. The grouping result is used to group the drawing tasks of the at least one application interface into at least one drawing task group, such that the drawing duration of the frame to be displayed is less than or equal to the sum of the drawing durations of the at least one application interface. Based on the grouping result, the drawing tasks of the at least one application interface are divided into at least one drawing task group. At least one drawing thread is used to process the drawing tasks of the at least one drawing task group to obtain at least one drawing result, and the at least one drawing result is used to synthesize the total drawing result of the frame to be displayed.

[0009] In this solution, the grouping results for the frames to be displayed are first determined. This grouping result ensures that the rendering time of the frames to be displayed is less than or equal to the sum of the rendering times of at least one application interface, that is, the rendering time required for the frames to be displayed without grouping. The rendering tasks for the frames to be displayed are then divided based on this grouping result to obtain at least one rendering task group. Finally, at least one rendering thread processes the drawing tasks of the at least one rendering task group to obtain at least one rendering result. This at least one rendering result is then used to synthesize the total rendering result for the frames to be displayed, thereby rendering the frames to be displayed. This shows that the rendering method of this solution predetermines the grouping results for the frames to be displayed to achieve optimal rendering of the frames to be displayed, effectively improving the rendering efficiency of the frames to be displayed.

[0010] In combination with a possible implementation of the first aspect, the rendering method further includes: sending at least one drawing result.

[0011] It can be seen that in this case, at least one drawing result corresponding to at least one drawing task group is sent to other devices or modules, so that other devices or modules can synthesize the total drawing result of the frame to be displayed and finally complete the rendering of the frame to be displayed.

[0012] In one possible implementation of the first aspect, the rendering method further includes: splitting the rendering tree corresponding to the user interface to obtain at least two spatial trees. The state operation of the drawing nodes in each spatial tree remains unchanged before and after the split, and the state operation is an operation that modifies properties of the canvas.

[0013] The at least two spatial trees are used to render a user interface, wherein the user interface is at least one of at least one application interface.

[0014] It can be seen that in this application, the rendering tree corresponding to the user interface is split into at least two spatial trees, and the state operation of the drawing nodes in each spatial tree is guaranteed to remain unchanged before and after the split. In this way, when the above-mentioned user interface is rendered using at least two spatial trees, correct rendering can be achieved to ensure the correctness of the rendering results of the user interface.

[0015] In a possible implementation of the first aspect, the rendering areas corresponding to the at least two spatial trees do not overlap, whereby the rendering areas do not overlap at all, thereby further ensuring the correctness of the rendering result of the user interface.

[0016] In combination with a possible implementation of the first aspect, the rendering tree corresponding to the user interface is split to obtain at least two spatial trees, including:

[0017] Determine the graphics bounding the rendering area corresponding to each rendering node in the rendering tree. Divide the graphics bounding each rendering node in the rendering tree into at least two sets. The difference between the number of graphics bounding in any two of the at least two sets is less than a first threshold, and / or the difference between the sum of the areas of the graphics bounding in any two of the at least two sets is less than a second threshold. Split the rendering tree according to the at least two sets to obtain at least two spatial trees.

[0018] The spatial tree corresponding to the set includes the drawing nodes corresponding to the external graphics in the set, and the state operation nodes passed when tracing back from the drawing nodes to the root node of the rendering tree.

[0019] Circumscribed graphics refer to regular graphics and / or graphics whose areas can be calculated, such as rectangles, circles, squares, trapezoids, etc. Since the shapes of the rendering areas corresponding to drawing nodes vary, it is impossible to accurately divide the drawing nodes. Therefore, the circumscribed graphics corresponding to the drawing nodes can be used to accurately divide the drawing nodes. In addition, when dividing the nodes in the rendering tree, the drawing nodes are divided first, and then the state operation nodes passed through during the backtracking process are grouped into the same set as the above-mentioned drawing nodes. In this way, the state operation of the drawing nodes before and after the division can be ensured to remain unchanged, ensuring that the spatial tree obtained after the split can be rendered correctly.

[0020] In a possible implementation of the first aspect, the rendering areas corresponding to the at least two sets do not overlap, thereby further ensuring the correctness of the rendering result of the user interface.

[0021] In one possible implementation of the first aspect, the external graphic of the rendering area corresponding to the drawing node is a bounding box of the rendering area corresponding to the drawing node. This eliminates the need to recreate the external graphic and directly uses the existing bounding box of the drawing node as the external graphic, thus saving processing time and improving rendering efficiency.

[0022] In one possible implementation of the first aspect, at the same time, one core of a central processing unit (CPU) in the electronic device executes one drawing thread. One core is responsible for one drawing thread, thus ensuring the highest rendering efficiency.

[0023] In combination with a possible implementation of the first aspect, the grouping result is a grouping scheme of at least one application interface whose first value meets a preset condition, the first value of the grouping scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the grouping scheme, and the synthesis time is the product of the number of drawing task groups in the grouping scheme and the synthesis overhead.

[0024] It can be seen that in the present application, by determining a grouping scheme of at least one application interface, when the first value of the grouping scheme meets the preset condition, the grouping scheme is determined to be the above-mentioned grouping result.

[0025] In conjunction with one possible implementation of the first aspect, when the number of drawing task groups in the grouping scheme is two, the grouping scheme for the at least one application interface includes a first grouping scheme, wherein the first grouping scheme divides the application interface sequence into two parts of equal drawing duration, and the application interface sequence is a sequence obtained by arranging the at least one application interface in a preset order. This indicates that the bisection scheme can be used as a possible grouping scheme for the at least one application interface.

[0026] Optionally, the grouping scheme for the at least one application interface further includes a second grouping scheme and / or a third grouping scheme, wherein the second grouping scheme is to move the bisection point to the starting point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface. The bisection point is the position corresponding to the division of the application interface sequence into two parts of equal drawing duration, and the first application interface is one of the at least one application interface.

[0027] The third grouping scheme is to move the bisection point to the end point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface.

[0028] It can be seen that it is also possible to obtain a possible grouping scheme for at least one of the above-mentioned application interfaces by moving the bisection point to the starting point and / or end point of the drawing task of the application interface, thereby avoiding the splitting overhead generated when splitting the application interface and reducing the drawing time of the drawing task group.

[0029] In combination with a possible implementation of the first aspect, when the number of drawing task groups in the grouping scheme is M, M is greater than or equal to three, and the grouping scheme of the above-mentioned at least one application interface includes a fourth grouping scheme, and the fourth grouping scheme is to divide the application interface sequence into M parts with equal drawing time.

[0030] Optionally, the grouping scheme for at least one application interface further includes a fifth grouping scheme, wherein when the first M sub-point is within the drawing task range of the first application interface, the positions of M-2 of the M-1 M sub-points in the application interface sequence are fixed, and the first M sub-point is moved to the starting point of the drawing task of the first application interface. The M-1 M sub-points are positions corresponding to dividing the application interface sequence into M parts of equal drawing duration, and the first M sub-point is any one of the M-1 M sub-points.

[0031] Optionally, the grouping scheme of the above-mentioned at least one application interface also includes a sixth grouping scheme, and the above-mentioned sixth grouping scheme is that when the first M sub-point is located within the drawing task range of the first application interface, M-2 of the M-1 M sub-points in the above-mentioned application interface sequence are fixed, and the above-mentioned first M sub-point is moved to the end point of the drawing task of the first application interface.

[0032] As can be seen, when the number of drawing task groups in the grouping scheme is M, a similar method is used when the number of drawing task groups in the grouping scheme is two, and the M-equal division scheme is used as a possible grouping scheme for the at least one application interface. Furthermore, when the first M-divided point is within the drawing task range of the first application interface, by fixing the positions of M-2 of the M-1 M-divided points in the application interface sequence and moving the first M-divided point to the starting point and / or end point of the drawing task of the first application interface, a possible grouping scheme for the at least one application interface can be obtained.

[0033] In one possible implementation of the first aspect, the drawing duration of the drawing task group is at least one of the following:

[0034] When the drawing task group includes a drawing task of a complete application interface, "include" here means only including, and the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0035] When the drawing task group includes the drawing task of an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the first drawing duration, the first drawing duration is the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the sum of the split durations, and the split duration is the product of the number of incomplete application interfaces and the splitting overhead.

[0036] When the drawing task group includes drawing tasks for a complete application interface and drawing tasks for an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the second drawing duration, the second drawing duration is the sum of the first drawing duration and the third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0037] In a possible implementation of the first aspect, the drawing time of a complete application interface is any one of the following:

[0038] The weighted average drawing time of the application interface. The weighted average drawing time is the weighted calculation value of the fourth drawing time, the fifth drawing time, and their respective weights. The fourth drawing time is the actual measured drawing time of the application interface in the second frame before the frame to be displayed. The fifth drawing time is the actual measured drawing time of the application interface in the frame before the frame to be displayed.

[0039] The total number of rendering instructions of the application interface is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

[0040] In this application, the weighted average drawing time of the application interface or the total number of rendering instructions is used as the drawing time of the application interface, which can accurately estimate the drawing load size of the application interface.

[0041] In a second aspect, the present application further provides a rendering method, the method comprising the following steps:

[0042] The rendering tree corresponding to the user interface is split to obtain at least two spatial trees. The state operations of the rendering nodes in each spatial tree remain unchanged before and after the split, and the state operations are operations that modify the properties of the canvas. The at least two spatial trees are used to render the user interface.

[0043] In this application, a user interface is at least one application interface presented to a user on a screen. The rendering tree corresponding to the user interface is split into at least two spatial trees, and the state and operation of the drawing nodes in each spatial tree are guaranteed to remain unchanged before and after the split. In this way, when the user interface is rendered using at least two spatial trees, the correctness of the rendering result of the user interface can be ensured.

[0044] In a possible implementation of the second aspect, the rendering areas corresponding to the at least two spatial trees do not overlap, wherein the rendering areas do not overlap at all, thereby further ensuring the correctness of the rendering result of the user interface.

[0045] In combination with a possible implementation of the second aspect, the rendering tree corresponding to the user interface is split to obtain at least two spatial trees, including:

[0046] Determine the bounding box of the rendering area corresponding to each drawing node in the rendering tree. Divide the bounding box corresponding to the drawing nodes in the rendering tree into at least two sets. The difference between the number of bounding boxes in any two of the at least two sets is less than a first threshold, and / or the difference between the sum of the areas of the bounding boxes in any two of the at least two sets is less than a second threshold. Split the rendering tree according to the at least two sets to obtain at least two spatial trees. The spatial trees corresponding to the sets include the drawing nodes corresponding to the bounding boxes in the sets and the state operation nodes traversed when tracing back from the drawing nodes to the root node of the rendering tree.

[0047] Because the rendering areas corresponding to drawing nodes vary in shape, it's impossible to precisely divide drawing nodes. Therefore, in this application, the external graphics corresponding to the drawing nodes can be used to accurately divide drawing nodes. Furthermore, when dividing nodes in the rendering tree, the drawing nodes are divided first, and then traced back from the drawing nodes to the root node of the rendering tree. The state operation nodes passed during the tracing back are grouped together with the drawing nodes. This ensures that the state operation of the drawing nodes before and after the division remains unchanged, ensuring that the resulting spatial tree can be rendered correctly.

[0048] In a possible implementation of the second aspect, the rendering areas corresponding to the at least two sets do not overlap, thereby further ensuring the correctness of the rendering result of the user interface.

[0049] In one possible implementation of the second aspect, the external graphic of the rendering area corresponding to the drawing node is a bounding box of the rendering area corresponding to the drawing node. This eliminates the need to recreate the external graphic and directly uses the existing bounding box of the drawing node as the external graphic, thus saving processing time and improving rendering efficiency.

[0050] In conjunction with one possible implementation of the second aspect, before splitting the rendering tree corresponding to the user interface, the rendering method further includes: determining a grouping result for frames to be displayed. The frames to be displayed are used to display the at least one application interface. The grouping result is used to organize the drawing tasks of the at least one application interface into at least one drawing task group, such that the drawing time of the frames to be displayed is less than or equal to the sum of the drawing times of the at least one application interface.

[0051] In this solution, a grouping result for the frames to be displayed is first determined. This grouping result ensures that the rendering time of the frames to be displayed is less than or equal to the sum of the rendering times of at least one application interface, that is, the rendering time required for the frames to be displayed if no grouping is performed. If the grouping result indicates that the user interface needs to be split, the splitting method provided in the second aspect is used to split the user interface.

[0052] In one possible implementation of the second aspect, the rendering method further includes: dividing the drawing tasks of the at least one application interface into at least one drawing task group based on the grouping result; processing the drawing tasks of the at least one drawing task group using at least one drawing thread to obtain at least one drawing result; and synthesizing the at least one drawing result to obtain an overall drawing result of the frame to be displayed.

[0053] As can be seen, the rendering method of this solution predetermines the grouping results of the frames to be displayed to achieve optimal rendering of the frames to be displayed, effectively improving the rendering efficiency of the frames to be displayed. Furthermore, when the grouping results indicate that the user interface needs to be split, after splitting the rendering tree corresponding to the user interface, the drawing tasks of at least one application interface are divided into at least one drawing task group based on the grouping results, and the drawing tasks of the drawing task group are then processed by the drawing thread.

[0054] In combination with a possible implementation of the second aspect, the rendering method further includes: sending at least one drawing result.

[0055] In one possible implementation of the second aspect, at the same time, one core of a central processing unit (CPU) in the electronic device executes one drawing thread. One core is responsible for one drawing thread, thus ensuring the highest rendering efficiency.

[0056] In combination with a possible implementation of the second aspect, the grouping result is a grouping scheme of at least one application interface whose first value meets a preset condition, the first value of the grouping scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the grouping scheme, and the synthesis time is the product of the number of drawing task groups in the grouping scheme and the synthesis overhead.

[0057] In combination with a possible implementation of the second aspect, when the number of drawing task groups in the grouping scheme is two, the grouping scheme of the above-mentioned at least one application interface includes a first grouping scheme, and the first grouping scheme is to divide the application interface sequence into two parts with equal drawing time, and the above-mentioned application interface sequence is a sequence obtained by arranging the above-mentioned at least one application interface in a preset order.

[0058] Optionally, the grouping scheme for the at least one application interface further includes a second grouping scheme and / or a third grouping scheme, wherein the second grouping scheme is to move the bisection point to the starting point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface. The bisection point is the position corresponding to the division of the application interface sequence into two parts of equal drawing duration, and the first application interface is one of the at least one application interface.

[0059] The third grouping scheme is to move the bisection point to the end point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface.

[0060] In combination with a possible implementation of the second aspect, when the number of drawing task groups in the grouping scheme is M, M is greater than or equal to three, and the grouping scheme of the above-mentioned at least one application interface includes a fourth grouping scheme, and the fourth grouping scheme is to divide the application interface sequence into M parts with equal drawing time.

[0061] Optionally, the grouping scheme for at least one application interface further includes a fifth grouping scheme, wherein when the first M sub-point is within the drawing task range of the first application interface, the positions of M-2 of the M-1 M sub-points in the application interface sequence are fixed, and the first M sub-point is moved to the starting point of the drawing task of the first application interface. The M-1 M sub-points are positions corresponding to dividing the application interface sequence into M parts of equal drawing duration, and the first M sub-point is any one of the M-1 M sub-points.

[0062] Optionally, the grouping scheme of the above-mentioned at least one application interface also includes a sixth grouping scheme, and the above-mentioned sixth grouping scheme is that when the first M sub-point is located within the drawing task range of the first application interface, M-2 of the M-1 M sub-points in the above-mentioned application interface sequence are fixed, and the above-mentioned first M sub-point is moved to the end point of the drawing task of the first application interface.

[0063] In one possible implementation of the second aspect, the drawing duration of the drawing task group is at least one of the following:

[0064] When the drawing task group includes a drawing task of a complete application interface, "include" here means only including, and the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0065] When the drawing task group includes the drawing task of an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the first drawing duration, the first drawing duration is the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the sum of the split durations, and the split duration is the product of the number of incomplete application interfaces and the splitting overhead.

[0066] When the drawing task group includes drawing tasks for a complete application interface and drawing tasks for an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the second drawing duration, the second drawing duration is the sum of the first drawing duration and the third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0067] In a possible implementation of the second aspect, the drawing time of the complete application interface is any one of the following:

[0068] The weighted average drawing time of the application interface. The weighted average drawing time is the weighted calculation value of the fourth drawing time, the fifth drawing time, and their respective weights. The fourth drawing time is the actual measured drawing time of the application interface in the second frame before the frame to be displayed. The fifth drawing time is the actual measured drawing time of the application interface in the frame before the frame to be displayed.

[0069] The total number of rendering instructions of the application interface is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

[0070] In a third aspect, the present application further provides a rendering device, which includes a determination module, a division module, and a processing module, wherein:

[0071] A determination module is configured to determine grouping results corresponding to frames to be displayed. The frames to be displayed are used to display at least one application interface, and the grouping results are used to organize the drawing tasks of the at least one application interface into at least one drawing task group, such that the drawing duration of the frames to be displayed is less than or equal to the sum of the drawing durations of the at least one application interface.

[0072] The division module is used to divide the drawing task of at least one application interface into at least one drawing task group according to the grouping result.

[0073] The processing module is configured to process at least one drawing task group using at least one drawing thread to obtain at least one drawing result.

[0074] In combination with a possible implementation of the third aspect, the rendering device further includes a sending module, wherein:

[0075] The sending module is used to send at least one drawing result.

[0076] In combination with a possible implementation of the third aspect, the rendering device further includes a splitting module, wherein:

[0077] A splitting module is configured to split the rendering tree corresponding to the user interface into at least two spatial trees. The state operations of the rendering nodes in each spatial tree remain unchanged before and after the split, and the state operations are operations that modify canvas properties. The at least two spatial trees are used to render the user interface. The user interface is at least one of at least one application interface.

[0078] In combination with a possible implementation of the third aspect, rendering areas corresponding to the at least two spatial trees do not overlap.

[0079] In conjunction with a possible implementation of the third aspect, the splitting module is specifically configured to:

[0080] Determine the bounding box of the rendering area corresponding to each drawing node in the rendering tree. Divide the bounding box corresponding to each drawing node in the rendering tree into at least two sets. The difference between the number of bounding boxes in any two of the at least two sets is less than a first threshold, and / or the difference between the sum of the areas of the bounding boxes in any two of the at least two sets is less than a second threshold. Split the rendering tree according to the at least two sets to obtain at least two spatial trees. The spatial trees corresponding to the sets include the drawing nodes corresponding to the bounding boxes in the sets and the state operation nodes traversed when tracing back from the drawing nodes to the root node of the rendering tree.

[0081] In combination with a possible implementation of the third aspect, the rendering areas corresponding to the at least two sets do not overlap.

[0082] The external graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

[0083] In combination with a possible implementation of the third aspect, at the same time, one core of a central processing unit (CPU) in the electronic device executes one of the drawing threads.

[0084] In combination with a possible implementation of the third aspect, the above-mentioned grouping result is a grouping scheme of at least one application interface whose first value meets the preset conditions, and the first value of the grouping scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the grouping scheme.

[0085] In combination with a possible implementation of the third aspect, when the number of drawing task groups in the grouping scheme is two, the grouping scheme of the above-mentioned at least one application interface includes a first grouping scheme, and the first grouping scheme is to divide the application interface sequence into two parts with equal drawing time, and the above-mentioned application interface sequence is a sequence obtained by arranging the above-mentioned at least one application interface in a preset order.

[0086] Optionally, the grouping scheme for the at least one application interface further includes a second grouping scheme and / or a third grouping scheme, wherein the second grouping scheme is to move the bisection point to the starting point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface. The bisection point is the position corresponding to the division of the application interface sequence into two parts of equal drawing duration, and the first application interface is one of the at least one application interface.

[0087] The third grouping scheme is to move the bisection point to the end point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface.

[0088] In combination with a possible implementation of the third aspect, when the number of drawing task groups in the grouping scheme is M, M is greater than or equal to three, and the grouping scheme of the above-mentioned at least one application interface includes a fourth grouping scheme, and the fourth grouping scheme is to divide the application interface sequence into M parts with equal drawing time.

[0089] Optionally, the grouping scheme for at least one application interface further includes a fifth grouping scheme, wherein when the first M sub-point is within the drawing task range of the first application interface, the positions of M-2 of the M-1 M sub-points in the application interface sequence are fixed, and the first M sub-point is moved to the starting point of the drawing task of the first application interface. The M-1 M sub-points are positions corresponding to dividing the application interface sequence into M parts of equal drawing duration, and the first M sub-point is any one of the M-1 M sub-points.

[0090] Optionally, the grouping scheme of the above-mentioned at least one application interface also includes a sixth grouping scheme, and the above-mentioned sixth grouping scheme is that when the first M sub-point is located within the drawing task range of the first application interface, M-2 of the M-1 M sub-points in the above-mentioned application interface sequence are fixed, and the above-mentioned first M sub-point is moved to the end point of the drawing task of the first application interface.

[0091] In conjunction with a possible implementation of the third aspect, the drawing duration of the drawing task group is at least one of the following:

[0092] When the drawing task group includes a drawing task for a complete application interface, the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group;

[0093] When a drawing task group includes drawing tasks for incomplete application interfaces, the drawing duration of the drawing task group is the first drawing duration. The first drawing duration is the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the segmentation duration. The segmentation duration is the product of the number of incomplete application interfaces and the segmentation overhead.

[0094] When the drawing task group includes a drawing task for a complete application interface and a drawing task for an incomplete application interface, the drawing duration of the drawing task group is the second drawing duration, the second drawing duration is the sum of the first drawing duration and the third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0095] In a possible implementation of the third aspect, the drawing time of the complete application interface is any one of the following:

[0096] The weighted average drawing time of the application interface, which is the weighted calculation value of the fourth drawing time, the fifth drawing time, and their respective weights. The fourth drawing time is the actual drawing time of the application interface in the second frame before the frame to be displayed. The fifth drawing time is the actual drawing time of the application interface in the frame before the frame to be displayed.

[0097] The total number of rendering instructions of the application interface is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

[0098] In a fourth aspect, the present application further provides a rendering device, which includes a splitting module, wherein:

[0099] A splitting module is configured to split the rendering tree corresponding to the user interface to obtain at least two spatial trees. The state operations of the drawing nodes in each spatial tree remain unchanged before and after the split, where the state operations are operations that modify canvas properties. The at least two spatial trees are used to render the user interface. In one possible implementation of the fourth aspect, the rendering areas corresponding to the at least two spatial trees do not overlap.

[0100] In conjunction with a possible implementation of the fourth aspect, the splitting module is specifically configured to:

[0101] Determine the bounding box of the rendering area corresponding to each drawing node in the rendering tree. Divide the bounding box corresponding to the drawing nodes in the rendering tree into at least two sets. The difference between the number of bounding boxes in any two of the at least two sets is less than a first threshold, and / or the difference between the sum of the areas of the bounding boxes in any two of the at least two sets is less than a second threshold. Split the rendering tree according to the at least two sets to obtain at least two spatial trees. The spatial trees corresponding to the sets include the drawing nodes corresponding to the bounding boxes in the sets and the state operation nodes traversed when tracing back from the drawing nodes to the root node of the rendering tree.

[0102] In combination with a possible implementation of the fourth aspect, the rendering areas corresponding to the at least two sets do not overlap.

[0103] In combination with a possible implementation of the fourth aspect, the circumscribed graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

[0104] In combination with a possible implementation of the fourth aspect, the rendering device further includes a determination module, wherein:

[0105] A determination module is configured to determine grouping results for frames to be displayed. The frames to be displayed are used to display the at least one application interface. The grouping results are used to organize the drawing tasks for the at least one application interface into at least one drawing task group, such that the drawing duration of the frames to be displayed is less than or equal to the sum of the drawing durations for the at least one application interface.

[0106] In combination with a possible implementation of the fourth aspect, the rendering device further includes a division module and a processing module, wherein:

[0107] The division module is used to divide the drawing task of at least one application interface into at least one drawing task group according to the grouping result.

[0108] The processing module is used to process the drawing tasks of at least one drawing task group using at least one drawing thread to obtain at least one drawing result, and the at least one drawing result is used to synthesize the total drawing result of the frame to be displayed.

[0109] Obviously, when the grouping result indicates that the user interface does not need to be split, the division module divides the drawing tasks of at least one application interface into at least one drawing task group according to the grouping result, and the processing module then uses the drawing thread to process the drawing tasks of the drawing task group.

[0110] Furthermore, when the grouping result indicates that the user interface needs to be split, after the splitting module splits the rendering tree corresponding to the user interface, the division module divides the drawing tasks of at least one application interface into at least one drawing task group according to the grouping result, and the processing module then uses the drawing thread to process the drawing tasks of the drawing task group.

[0111] In combination with a possible implementation of the fourth aspect, the rendering device further includes a sending module, wherein:

[0112] The sending module is used to send at least one drawing result.

[0113] In combination with a possible implementation of the fourth aspect, at the same time, a core of a central processing unit (CPU) in the electronic device executes a drawing thread.

[0114] In combination with a possible implementation of the fourth aspect, the grouping result is a grouping scheme of at least one application interface whose first value meets a preset condition, the first value of the grouping scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the grouping scheme, and the synthesis time is the product of the number of drawing task groups in the grouping scheme and the synthesis overhead.

[0115] In combination with a possible implementation of the fourth aspect, when the number of drawing task groups in the grouping scheme is two, the grouping scheme of the above-mentioned at least one application interface includes a first grouping scheme, and the first grouping scheme is to divide the application interface sequence into two parts with equal drawing time. The above-mentioned application interface sequence is a sequence obtained by arranging the above-mentioned at least one application interface in a preset order.

[0116] Optionally, the grouping scheme for the at least one application interface further includes a second grouping scheme and / or a third grouping scheme. The second grouping scheme is to move the bisection point to the starting point of the first application interface's drawing task when the bisection point is within the drawing task range of the first application interface. The bisection point is the position corresponding to the division of the application interface sequence into two portions of equal drawing duration, and the first application interface is one of the at least one application interface.

[0117] The third grouping scheme is to move the bisection point to the end point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface.

[0118] In combination with a possible implementation of the fourth aspect, when the number of drawing task groups in the grouping scheme is M, M is greater than or equal to three, and the grouping scheme of the above-mentioned at least one application interface includes a fourth grouping scheme, and the fourth grouping scheme is to divide the application interface sequence into M parts with equal drawing time.

[0119] Optionally, the grouping scheme for at least one application interface further includes a fifth grouping scheme, wherein when the first M sub-point is within the drawing task range of the first application interface, the positions of M-2 of the M-1 M sub-points in the application interface sequence are fixed, and the first M sub-point is moved to the starting point of the drawing task of the first application interface. The M-1 M sub-points are positions corresponding to dividing the application interface sequence into M parts of equal drawing duration, and the first M sub-point is any one of the M-1 M sub-points.

[0120] Optionally, the grouping scheme of the above-mentioned at least one application interface also includes a sixth grouping scheme, and the above-mentioned sixth grouping scheme is that when the first M sub-point is located within the drawing task range of the first application interface, M-2 of the M-1 M sub-points in the above-mentioned application interface sequence are fixed, and the above-mentioned first M sub-point is moved to the end point of the drawing task of the first application interface.

[0121] In one possible implementation of the fourth aspect, the drawing duration of the drawing task group is at least one of the following:

[0122] When the drawing task group includes a drawing task of a complete application interface, "include" here means only including, and the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0123] When the drawing task group includes the drawing task of an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the first drawing duration, the first drawing duration is the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the sum of the split durations, and the split duration is the product of the number of incomplete application interfaces and the splitting overhead.

[0124] When the drawing task group includes drawing tasks for a complete application interface and drawing tasks for an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the second drawing duration, the second drawing duration is the sum of the first drawing duration and the third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0125] In one possible implementation of the fourth aspect, the drawing time of the complete application interface is any one of the following:

[0126] The weighted average drawing time of the application interface. The weighted average drawing time is the weighted calculation value of the fourth drawing time, the fifth drawing time, and their respective weights. The fourth drawing time is the actual measured drawing time of the application interface in the second frame before the frame to be displayed. The fifth drawing time is the actual measured drawing time of the application interface in the frame before the frame to be displayed.

[0127] The total number of rendering instructions of the application interface is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

[0128] In a fifth aspect, the present application also provides an electronic device comprising a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute the rendering method as described in any one of the first aspect or the second aspect.

[0129] In a sixth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the rendering method as described in any one of the first aspect or the second aspect.

[0130] In a seventh aspect, the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the rendering method described in any one of the first aspect or the second aspect.

[0131] In an eighth aspect, the present application further provides a chip, comprising a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface and executes the method described in the first aspect or the second aspect.

[0132] Optionally, as an implementation method, the chip may further include a memory, in which instructions are stored, and the processor is used to execute the instructions stored on the memory. When the instructions are executed, the processor is used to execute the method described in the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] The following is an introduction to the drawings used in the embodiments of this application.

[0134] FIG1 is a schematic diagram of an application architecture of a rendering method provided in an embodiment of the present application;

[0135] FIG2 is a schematic diagram of a flow chart of a rendering method provided in an embodiment of the present application;

[0136] FIG3A is a schematic diagram of a process for determining a segmentation scheme provided in an embodiment of the present application;

[0137] FIG3B is a schematic diagram of another process for determining a segmentation scheme provided in an embodiment of the present application;

[0138] FIG3C is a schematic diagram of calculating the duration of a drawing task group provided in an embodiment of the present application;

[0139] FIG4A is a schematic diagram of a flow chart of a rendering method provided in an embodiment of the present application;

[0140] FIG4B is a schematic diagram of a single-threaded single-frame drawing method provided by an embodiment of the present application;

[0141] FIG4C is a schematic diagram of a multi-threaded drawing method provided by an embodiment of the present application;

[0142] FIG5 is a schematic diagram of a flow chart of another rendering method provided in an embodiment of the present application;

[0143] FIG6A is a schematic diagram of a rendering tree provided in an embodiment of the present application;

[0144] FIG6B is a schematic diagram of a bounding box of a drawing node provided in an embodiment of the present application;

[0145] FIG6C is a schematic diagram of a rectangle division method provided in an embodiment of the present application;

[0146] FIG6D is a schematic diagram of marking a rendering tree provided in an embodiment of the present application;

[0147] FIG6E is a schematic diagram of a spatial tree and its drawing result provided in an embodiment of the present application;

[0148] FIG6F is a schematic diagram of thread allocation of an application interface provided in an embodiment of the present application;

[0149] FIG6G is a schematic diagram of thread allocation of another application interface provided in an embodiment of the present application;

[0150] FIG7 is a schematic diagram of a process of a rendering device provided in an embodiment of the present application;

[0151] FIG8 is a schematic structural diagram of another rendering device provided in an embodiment of the present application;

[0152] FIG9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0153] The technical solution in this application will be described below with reference to the accompanying drawings.

[0154] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0155] The “at least one” mentioned in the embodiments of this application refers to one or more, and “plurality” refers to two or more. “At least one of the following items” or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, c can be single or multiple. “And / or” describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character “ / ” generally indicates that the related objects before and after are in an “or” relationship. The serial numbers of the steps in the embodiments of this application (such as step S1, step S21, etc.) are only for distinguishing different steps and do not limit the order of execution between the steps.

[0156] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, timing, priority, or importance of multiple objects. For example, the first device and the second device are only for ease of description and do not indicate differences in structure, importance, etc. between the first and second devices. In some embodiments, the first device and the second device can also be the same device.

[0157] As used in the embodiments of this application, the term "when" may be interpreted to mean "if...", "after...", "in response to determining...", or "in response to detecting...", depending on the context. The above descriptions are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc. made within the concepts and principles of this application shall be included within the scope of protection of this application.

[0158] Due to the applications involved in the embodiments of the present application, in order to facilitate understanding, the relevant terms and other related concepts involved in the embodiments of the present application are first introduced below.

[0159] (1) Rendering pipeline

[0160] The rendering pipeline, also known as the rendering production line, is the various operation processes that graphics images must go through step by step from data to the final output image. It generally includes: vertex data input, vertex shader, surface subdivision process, geometry shader, primitive assembly, clipping and culling, rasterization, fragment shader and mixed test and other processes.

[0161] (2) Rendering frame rate

[0162] The rendering frame rate refers to the frame rate per second (Frame Pre Second, FPS), which is the number of times (or rate) that bitmap images appear continuously on the display per second. In simple terms, it is the number of frames displayed on the screen in one second.

[0163] (3) Canvas

[0164] The canvas is the core class provided by the rendering engine. It encapsulates all drawing operations on the device and records the drawing status of the management device. By calling the rendering canvas interface, the rendering texture can be drawn or the status can be changed.

[0165] (4) Rendering tree

[0166] User Interface (UI) refers to at least one application interface presented to the user on the screen. The rendering tree is used for the layout and drawing of the user interface, is responsible for the actual rendering, and saves information such as the size and layout of the elements. Among them, the rendering tree includes multiple rendering nodes, each rendering node contains the layout information of the node and the recorded drawing operation information (DrawOp). The layout information includes the position, size (such as width, height) and bounding box of the rendering node. The bounding box refers to the circumscribed rectangle of the drawing area (or rendering area) corresponding to the rendering node. The drawing operation information refers to the content that the rendering node needs to draw, including but not limited to at least one of the following: drawing graphics, drawing text characters, drawing pictures, canvas clipping (Clip), canvas rotation (Transform), canvas movement (Move), and canvas transparency modification.

[0167] The rendering node is used to perform drawing operations and / or state operations, wherein the drawing operation can be understood as a rendering operation, including but not limited to at least one of the following: drawing graphics, drawing text characters, and drawing pictures.

[0168] State operations can be understood as operations that modify the properties of the canvas (such as the size, position, transparency, etc. of the canvas), including but not limited to at least one of the following: canvas cropping, canvas rotation, canvas movement, and canvas transparency modification.

[0169] In the embodiments of the present application, a drawing node refers to a rendering node in a rendering tree used to perform drawing operations, and a state operation node refers to a rendering node in a rendering tree used to perform state operations. For example, the drawing node and the state operation node can be the same rendering node, i.e., the rendering node is used to perform both drawing operations and state operations.

[0170] (5) Buffer Queue

[0171] Buffer queues are used to connect the party that generates graphics data buffers (producers) to the party that receives data for display or further processing (consumers).

[0172] (6) UI controls

[0173] UI controls are components that users can interact with or operate, such as buttons and input boxes.

[0174] (7) Rendering space tree (referred to as space tree)

[0175] A spatial tree is a tree structure composed of rendering nodes (i.e., drawing nodes and / or state operation nodes) corresponding to UI controls within a rendering area. It primarily represents the spatial relationship between UI controls. Optionally, different spatial trees are spatially disjoint, meaning that the rendering areas corresponding to different spatial trees do not overlap at all.

[0176] (8) Window

[0177] A window is a graphical interface element that displays the content of an application for the user to view and interact with. A window is typically a rectangular area that can be resized and often edited according to the capabilities and restrictions imposed on it by the application that provides the window.

[0178] (9) Z order

[0179] The Z-order refers to the Z-axis order of windows. A window's Z-order indicates its position in the list of stacked windows, and can also be understood as the Z-order of the application interface. The direction of the window stacking is along an imaginary axis, namely the Z-axis, extending outward from the screen.

[0180] User interface rendering tasks are typically handled serially. However, as the rendering workload increases, this process becomes inefficient. For example, as the number of application screens within a user interface increases, the rendering workload increases, making serial processing inefficient. Therefore, improving rendering efficiency is crucial.

[0181] To this end, the present application proposes a rendering method that first determines the grouping results of the screen frames to be displayed (i.e., the frames to be displayed), then processes the drawing tasks of the frames to be displayed based on the grouping results to obtain at least one drawing task group, and finally uses at least one drawing thread to process the drawing tasks of the at least one drawing task group to obtain at least one drawing result. The at least one drawing result is synthesized to obtain the total drawing result of the frames to be displayed. Among them, the frames to be displayed are used to display at least one application interface. Since the above grouping results can make the drawing time of the frames to be displayed less than the sum of the drawing times of at least one application interface, that is, the drawing time required for the frames to be displayed when no grouping is performed, the rendering method of the present application can effectively improve the rendering efficiency of the frames to be displayed.

[0182] In addition, when the grouping result indicates that the user interface needs to be split, another rendering method provided by the present application can be used to split the user interface, and the above-mentioned user interface is at least one of at least one application interface. Of course, the above-mentioned user interface can also be understood as at least one application interface presented to the user on the screen, and the rendering method of the present application can also be applied. Among them, in the embodiment of the present application, when the user interface is split, the rendering tree corresponding to the user interface is split into at least two spatial trees, and the state operation of the drawing node in each spatial tree before and after the split is guaranteed to remain unchanged. In this way, when the above-mentioned user interface is obtained by rendering using at least two spatial trees, the correctness of the rendering result of the user interface can be ensured.

[0183] The following is an introduction to the application architecture of the rendering method of this application.

[0184] Refer to Figure 1, which is a schematic diagram of the application architecture of a rendering method provided by an embodiment of the present application. The numbers in Figure 1 represent the following information:

[0185] ① Update message for drawing content;

[0186] ② is the display list;

[0187] ③ Drawing task group;

[0188] ④ Drawing instructions for the Graphics Processing Unit (GPU);

[0189] ⑤ is the cache (Buffer) of the total drawing results.

[0190] The rendering method in the embodiment of the present application can be applied to an electronic device, wherein, referring to Figure 1, the electronic device includes a central processing unit (CPU) and a GPU, wherein the rendering method of the present application is mainly executed on the CPU side, the CPU is responsible for preparing data, and the GPU performs rendering.

[0191] The following describes the rendering process using the application architecture in Figure 1:

[0192] 1. When the content drawn on the screen of the electronic device needs to be updated, a drawing content update message is sent to the CPU to trigger the UI thread.

[0193] 2. The UI thread completes the measurement and layout of controls and constructs a render tree based on the control information. The render tree consists of several render nodes, each of which contains a display list, which stores information about the drawing operations that need to be processed. The UI thread then sends a Sync signal to the main rendering thread, triggering state synchronization between the main rendering thread and the UI thread, including synchronizing the display list. The UI thread also synchronizes node attribute information (such as node size and position) with the main rendering thread.

[0194] 3. The main rendering thread requests a drawing buffer for the current frame to be displayed, then determines the grouping results for the drawing tasks for the frame to be displayed (see Figures 2 through 4C for details), and then processes the drawing tasks for the frame to be displayed based on the grouping results. If the grouping results for the frame to be displayed indicate that the user interface needs to be split, the user interface is split (see Figures 5 through 6G for details). Based on the grouping results, the drawing tasks for the frame to be displayed are combined to form at least one drawing task group, and finally, the at least one drawing task group is sent to the worker thread.

[0195] 4. After receiving at least one drawing task group, the working thread will create the required resources for each drawing thread in the at least one drawing thread, including applying for a drawing cache. Each drawing thread has an independent GPU rendering pipeline (such as "shared context", "create rendering target", "execute drawing operation", and "submit drawing instructions" in Figure 1, where creating context and rendering target does not need to be re-operated every frame, and only the content can be cleared). The working thread assigns the drawing tasks of at least one drawing task group to at least one drawing thread for processing to obtain at least one drawing result, i.e., a GPU instruction. Among them, one drawing thread processes the drawing tasks of one drawing task group, and each drawing thread converts CPU instructions into GPU instructions, and then submits the GPU instructions to the GPU for processing.

[0196] 5. After receiving the GPU instruction, the GPU starts to perform rendering work, that is, calculating the correct color for the corresponding pixel point. Since each drawing thread is submitted to a separate drawing buffer, the GPU also needs to synthesize the drawing results of each drawing buffer to obtain the total drawing result, and finally submit the cache of the total drawing result to the hardware synthesizer.

[0197] 6. After the hardware compositor waits for the GPU rendering to complete, it sends the total drawing result to the screen for display.

[0198] The rendering method provided in the embodiment of the present application is described in detail below.

[0199] Referring to FIG. 2 , FIG. 2 is a flow chart of a rendering method provided in an embodiment of the present application; the rendering method includes the following steps:

[0200] 201. Determine a grouping result of frames to be displayed.

[0201] Specifically, a pending frame is a frame waiting to be displayed. The pending frame is used to display at least one application interface. The grouping result is used to organize the drawing tasks of the at least one application interface into at least one drawing task group, such that the drawing duration of the pending frame is less than or equal to the sum of the drawing durations of the at least one application interface.

[0202] 202. Divide the drawing task of at least one application interface into at least one drawing task group according to the grouping result.

[0203] Specifically, when the grouping result indicates that the user interface needs to be split, the above-mentioned user interface is an interface composed of at least one application interface among the above-mentioned at least one application interface. The method shown in Figure 5 below can be used to split the user interface, which will not be described in detail here. Then, according to the grouping result, the drawing tasks corresponding to the spatial tree obtained after the split are divided into corresponding drawing task groups. In addition, when the grouping result indicates that the application interface does not need to be split, the drawing tasks of at least one application interface can be combined according to the grouping result. When the grouping result indicates that the drawing tasks of at least one application interface do not need to be split, the drawing tasks of at least one application interface are treated as a drawing task group.

[0204] 203. Process the drawing tasks of the at least one drawing task group using at least one drawing thread to obtain at least one drawing result. The at least one drawing result is used to synthesize and obtain a total drawing result of the frame to be displayed.

[0205] In an embodiment of the present application, the grouping result of the frames to be displayed is first determined. The grouping result can make the drawing time of the frames to be displayed less than the sum of the drawing time of at least one application interface, that is, the drawing time required for the frames to be displayed when no grouping is performed. The drawing tasks of the frames to be displayed are then divided according to the grouping result to obtain at least one drawing task group. Finally, at least one drawing thread processes the drawing tasks of the at least one drawing task group to obtain at least one drawing result. The at least one drawing result is used to synthesize the total drawing result of the frames to be displayed to render the frames to be displayed. It can be seen that by using the rendering method of the embodiment of the present application, the grouping result of the frames to be displayed is determined in advance to achieve optimal rendering of the frames to be displayed, which can effectively improve the rendering efficiency of the frames to be displayed.

[0206] For example, the embodiments of the present application determine the rendering tasks of the frames to be displayed in different situations to determine the corresponding grouping results, so as to optimize the rendering efficiency of the frames to be displayed. For example, when the rendering tasks are heavy, the grouping results can divide the rendering tasks into two or more groups; while when the rendering tasks are light, the rendering tasks may not be grouped. The actual grouping results are determined based on the actual situation of the rendering tasks of the frames to be displayed.

[0207] In a possible implementation, referring to FIG2 , the rendering method further includes:

[0208] 204. Send at least one drawing result.

[0209] In this embodiment of the present application, at least one drawing result corresponding to at least one drawing task group is sent to other devices or modules, allowing the other devices or modules to synthesize the total drawing result of the frame to be displayed, ultimately completing the rendering of the frame to be displayed. Exemplarily, step 204 is performed by the CPU, where the CPU sends the at least one drawing result to the GPU, so that the GPU synthesizes the at least one drawing result to obtain the total drawing result of the frame to be displayed.

[0210] In a possible implementation, in the embodiment of the present application, at the same time, one core of the CPU of the electronic device executes the at least one drawing thread.

[0211] In one possible implementation, in an embodiment of the present application, at the same time, one core of the CPU in the electronic device executes one drawing thread. With one core responsible for one drawing thread, the rendering efficiency of the frames to be displayed can be maximized. Furthermore, when an electronic device has two or more cores, the method of an embodiment of the present application can be used to fully utilize the multi-core capabilities of the electronic device by rationally grouping the drawing content of the frames to be displayed, enabling the hardware to achieve maximum energy efficiency and avoiding uneven rendering load issues.

[0212] For example, when the electronic device has two or more cores, in an embodiment of the present application, at the same time, the number of drawing threads executing tasks is less than or equal to the number of cores of the CPU of the electronic device, and can be set according to actual conditions.

[0213] In one possible implementation, the grouping result is a grouping scheme of at least one application interface whose first value satisfies a preset condition, the first value of the grouping scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the grouping scheme, and the synthesis time is the product of the number of drawing task groups in the grouping scheme and the synthesis overhead.

[0214] The grouping scheme is a scheme for grouping the drawing tasks of at least one application interface into at least one drawing task group. In an embodiment of the present application, by determining the grouping scheme of at least one application interface, when the first value of the grouping scheme meets a preset condition, the grouping scheme is determined to be the above-mentioned grouping result.

[0215] For example, at least one grouping scheme for at least one application interface can be first determined, and then the grouping scheme whose first value satisfies a preset condition in the at least one grouping scheme can be determined as the grouping result for the at least one application interface. In the embodiment of the present application, the corresponding preset condition can be set according to the actual situation, and there is no special limitation on this. For example, the preset condition is to minimize the first value so as to minimize the drawing time of a single frame.

[0216] Specifically, the drawing time of each frame is mainly affected by the window load, that is, the drawing time of each application interface. The more application interfaces that need to be drawn, the greater the total load of all windows, and the longer the single-frame drawing time will be. In addition, the drawing time of each frame is also mainly affected by the number of GPU synthesis times (that is, the number M of drawing task groups of frames to be displayed). Each synthesis causes a certain synthesis overhead α, α is an empirical value, which is generally related to the size of the synthesis buffer and the computing power of the device and can be obtained through actual testing. Therefore, this implementation example needs to calculate a suitable number of GPU synthesis times. Multiple drawing task groups are drawn in parallel, so the drawing time of each frame is mainly affected by the drawing task group with the longest (drawing time). Further, when the drawing time of the longest drawing task group is the shortest, the drawing time of a single frame is also the shortest. That is, the embodiment of the present application determines the grouping of drawing tasks through time budget, synthesis number constraints, and the maximum number of threads of the device.

[0217] As an optimization, we want to maximize the number of drawing task groups that can be executed simultaneously. Because each CPU core can independently execute the drawing operation of a drawing task group at the same time within a drawing cycle, the maximum number of GPU synthesis times is set to be equal to the number of CPU cores m of the electronic device to maximize the parallel execution of drawing operations. When the longest drawing task group has the shortest duration, the drawing duration of a single frame is minimized and the drawing frame rate is maximized. j Indicates the drawing duration of the j-th drawing task group.

[0218] In the embodiment of the present application, the first value D is set to: D=T max +M*α (1)

[0219] Among them, T max =max(T),T j ∈T,1≤j≤m,T max Indicates the drawing duration of the longest drawing task group in the grouping scheme.

[0220] In an embodiment of the present application, at least one possible grouping scheme for at least one application interface can be obtained first, and then the first value of each grouping scheme can be calculated according to formula (1), and the grouping scheme with the smallest first value can be used as the grouping result of at least one application interface, so that the drawing time of the frame to be displayed is less than the sum of the drawing time of at least one application interface.

[0221] In one possible implementation, when the number of drawing task groups in the grouping scheme is two, the grouping scheme for the at least one application interface includes a first grouping scheme, wherein the first grouping scheme divides the application interface sequence into two parts of equal drawing duration, and the application interface sequence is a sequence obtained by arranging the at least one application interface in a preset order. In this embodiment of the present application, a bisection scheme can be used as a possible grouping scheme for the at least one application interface.

[0222] The preset order can be determined based on actual conditions. For example, the application interface sequence can be obtained by arranging them in Z order from small to large. The preset order can also be in Z order from large to small, and there is no special limitation on the order of the application interface sequence. In addition, since the drawing order of the application interface must be in Z order from small to large, during the GPU synthesis stage, it is necessary to synthesize the application interface in Z order from small to large in order to ensure the correctness of the drawing result.

[0223] Optionally, the grouping scheme for the at least one application interface further includes a second grouping scheme and / or a third grouping scheme. The second grouping scheme is to move the bisection point to the starting point of the first application interface's drawing task when the bisection point is within the drawing task range of the first application interface. The bisection point is the position corresponding to the division of the application interface sequence into two portions of equal drawing duration, and the first application interface is one of the at least one application interface.

[0224] The third grouping scheme is to move the bisection point to the end point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface.

[0225] In an embodiment of the present application, it is also possible to obtain a possible grouping scheme for at least one of the above-mentioned application interfaces by moving the bisection point to the starting point and / or end point of the drawing task of the application interface, thereby avoiding the splitting overhead generated when splitting the application interface and reducing the drawing time of the drawing task group.

[0226] In one possible implementation, when the number of drawing task groups in the grouping scheme is M, M is greater than or equal to three, and the grouping scheme of the above-mentioned at least one application interface includes a fourth grouping scheme, and the fourth grouping scheme is to divide the application interface sequence into M parts with equal drawing time.

[0227] Optionally, the grouping scheme for at least one application interface further includes a fifth grouping scheme, wherein when the first M sub-point is within the drawing task range of the first application interface, the positions of M-2 of the M-1 M sub-points in the application interface sequence are fixed, and the first M sub-point is moved to the starting point of the drawing task of the first application interface. The M-1 M sub-points are positions corresponding to dividing the application interface sequence into M parts of equal drawing duration, and the first M sub-point is any one of the M-1 M sub-points.

[0228] Optionally, the grouping scheme of the above-mentioned at least one application interface also includes a sixth grouping scheme, and the above-mentioned sixth grouping scheme is that when the first M sub-point is located within the drawing task range of the first application interface, M-2 of the M-1 M sub-points in the above-mentioned application interface sequence are fixed, and the above-mentioned first M sub-point is moved to the end point of the drawing task of the first application interface.

[0229] In an embodiment of the present application, when the number of drawing task groups in the grouping scheme is M, a method similar to the method used when the number of drawing task groups in the grouping scheme is two is used to divide the at least one application interface into M equal parts as a possible grouping scheme. Furthermore, when the first M-divided point is within the drawing task range of the first application interface, a possible grouping scheme for the at least one application interface can be obtained by fixing the positions of M-2 of the M-1 M-divided points in the application interface sequence and moving the first M-divided point to the starting point and / or end point of the drawing task of the first application interface.

[0230] Exemplarily, a possible grouping scheme for the at least one application interface may also be determined by methods such as manual grouping, and the grouping method for the at least one application interface is not limited.

[0231] The following is a detailed description of a method for obtaining at least one possible grouping scheme for the at least one application interface:

[0232] First, the number M of the drawing task groups of the frames to be displayed is determined. The number M is selected in the range of 1 to m, where m is the number of cores of the central processing unit (CPU) of the electronic device, that is, the maximum number of GPU synthesis times supported by the electronic device.

[0233] Determine at least one segmentation scheme corresponding to each number M of application interface sequences. This yields m segmentation schemes, each corresponding to the number M, with Y being greater than or equal to m. Each segmentation scheme divides the application interface sequence into at least one portion, each corresponding to a drawing task group. A segmentation scheme under the number M represents a possible grouping scheme for the at least one application interface.

[0234] Determine the first value corresponding to each segmentation scheme according to formula (1). The segmentation scheme with the smallest first value among the Y segmentation schemes and the corresponding number M are determined as the grouping result. The first value of the segmentation scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the segmentation scheme. The synthesis time is the product of the number M and the synthesis cost.

[0235] In an embodiment of the present application, by determining at least one segmentation scheme corresponding to each number M, and then determining a segmentation scheme with the smallest first value from the Y segmentation schemes, the segmentation scheme with the smallest first value and the corresponding number M are used as the grouping results, the accuracy of the grouping results can be guaranteed.

[0236] For example, assuming that at least one segmentation scheme corresponding to each number M is determined, the following provides a possible solution for determining the above three segmentation schemes. First, take the case where the number M is 2 to determine the corresponding three segmentation schemes as an example. Referring to Figure 3A, Figure 3A is a schematic diagram of the process of determining a segmentation scheme provided by an embodiment of the present application. The application interface sequence 304 is divided into the order of background application, Douyin application, Taobao application, status bar application and navigation bar application as an example. First, the bisection point 302 of the application interface sequence 304 is determined. The bisection point 302 is located within the drawing task range of the Taobao application. The segmentation scheme corresponding to the bisection point 302 is used as a segmentation scheme of the application interface sequence 304. Then, the bisection point 302 is moved to the left to the starting point 301 of the drawing task of the Taobao application, and the segmentation scheme corresponding to the starting point 301 is used as a segmentation scheme of the application interface sequence 304. Similarly, the bisection point 302 is moved to the right to the end point 303 of the drawing task of the Taobao application, and the segmentation scheme corresponding to the end point 303 is used as a segmentation scheme of the application interface sequence 304. Thus, it is determined that three segmentation schemes corresponding to the application interface sequence 304 are determined when the number M is 2. In addition, when the bisection point 302 overlaps with the starting point 301 or the end point 303, there are two segmentation schemes corresponding to the number M being 2.

[0237] Referring to Figure 3B, Figure 3B is a schematic diagram of another process for determining a segmentation scheme provided by an embodiment of the present application. Continuing with the example of the number M being 3, similarly, first determine the three-point points, which are 305 and 306 respectively, and use the segmentation schemes corresponding to the three-point points 305 and 306 as a segmentation scheme for the application interface sequence 304. Then fix the position of the three-point point 306, and move the part from the starting point of the application interface sequence 304 to the three-point point 306 to the left and right respectively according to the method of Figure 3A to reach the starting point and end point of the drawing task of the vibrato application, and use the segmentation scheme corresponding to the three-point point 306 and the starting point of the drawing task of the vibrato application as a segmentation scheme for the application interface sequence 304, and use the segmentation scheme corresponding to the three-point point 306 and the end point of the drawing task of the vibrato application as a segmentation scheme for the application interface sequence 304. Similarly, the position of the third point 305 is fixed, and the portion from the third point 305 to the end point of the application interface sequence 304 is respectively moved left and right by the third point 306 according to the method in FIG3A to reach the starting point and end point of the drawing task of the status bar application. The segmentation scheme corresponding to the third point 305 and the starting point of the drawing task of the status bar application is used as a segmentation scheme of the application interface sequence 304, and the segmentation scheme corresponding to the third point 305 and the end point of the drawing task of the status bar application is used as a segmentation scheme of the application interface sequence 304.

[0238] When the number M is greater than 3, the method for determining the segmentation scheme is the same as when the number M is 3, and will not be repeated here. The difference is that the M points need to be determined, and then the positions of M-2 M points are fixed. The remaining M points are moved left and right in sequence to the boundary points of the application interface (i.e., the starting and ending points of the application interface drawing task).

[0239] In a possible implementation, when the drawing task group includes a complete application interface drawing task, "include" here means only including, and the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0240] Specifically, when the positions of the front and back cutting points of a drawing task group are both located on the boundary of the application interface, refer to Figure 3C, which is a schematic diagram of the duration calculation of a drawing task group provided by an embodiment of the present application. In this case, the drawing task group H1 includes the drawing tasks of two complete application interfaces, namely the background application and the Douyin application, taking the application interface sequence 304 as an example, and the drawing task group H1 includes the drawing tasks of the background application and the Douyin application. In this case, the drawing duration of the drawing task group is determined by formula (2):

[0241] Among them, t p to t qIndicates the drawing time required for the complete application interface included in the drawing task group.

[0242] Exemplarily, referring to FIG3C , the drawing duration of the drawing task group H1 is the sum of the drawing duration required by the background application and the drawing duration required by the TikTok application.

[0243] In a possible implementation, when the above-mentioned drawing task group includes the drawing task of an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the first drawing duration, the first drawing duration is the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the sum of the split durations, and the split duration is the product of the number of incomplete application interfaces and the splitting overhead.

[0244] Specifically, when the front and back cut points of a rendering task group are both located within an application interface and the rendering task group does not include the complete application interface, one possible scenario is that the rendering task group only includes a rendering task for an incomplete application interface. Referring to Figure 3C , rendering task group H2 includes a portion of the Taobao application's rendering tasks. In this case, the rendering duration of the rendering task group is the rendering duration of the incomplete application interface plus a splitting cost β. The splitting cost β is typically related to the number of nodes in the render tree of the split application interface. Since this value is generally small, it can be simplified to a fixed splitting cost measured by the system. Alternatively, a splitting cost β can be measured for each application interface. The rendering duration of the incomplete application interface is the product of the splitting ratio and the rendering duration of the complete application interface. The splitting ratio is the proportion of the rendering tasks of the rendering task group to the entire application interface. Taking Figure 3C as an example, the rendering duration of rendering task group H2 is the product of the Taobao application's rendering duration t5 and k, where k is the proportion of the rendering task group H2 to the Taobao application's rendering task group.

[0245] Another possible scenario is that the drawing task group includes two incomplete application interface drawing tasks. Referring to Figure 3C , drawing task group H3 includes a portion of the Taobao application drawing tasks and a portion of the status bar application drawing tasks. In this case, the drawing duration of the drawing task group is the drawing duration of the two incomplete application interfaces plus two splitting overheads β. The portion of the Taobao application drawing tasks included in drawing task group H3 accounts for a proportion u of the Taobao application's drawing tasks, where the sum of k and u is one. The portion of the status bar application drawing tasks included in drawing task group H3 accounts for a proportion e of the status bar application's drawing tasks.

[0246] In a possible implementation, when the above-mentioned drawing task group includes a drawing task of a complete application interface and a drawing task of an incomplete application interface, "include" here means only including, the drawing time of the drawing task group is the second drawing time, the second drawing time is the sum of the first drawing time and the third drawing time, and the third drawing time is the sum of the drawing times of the complete application interfaces included in the drawing task group.

[0247] Referring to Figure 3C , drawing task group H4 includes a portion of the drawing tasks for the status bar application and a portion of the drawing tasks for the navigation bar application. The proportion of the drawing tasks for the status bar application in drawing task group H4 to the total drawing tasks for the status bar application is n, and the sum of e and n is 1. As can be seen, Figure 3C only includes one incomplete application interface, so the first drawing duration includes one segmentation cost β. If the drawing task group includes two incomplete application interface drawing tasks, the first drawing duration includes two segmentation costs β.

[0248] In one possible implementation, the drawing time of the complete application interface may be a weighted average drawing time of the application interface. The weighted average drawing time is a weighted calculation value of the fourth drawing time, the fifth drawing time, and their respective weights. The fourth drawing time is the actual drawing time of the application interface in the second frame before the frame to be displayed, and the fifth drawing time is the actual drawing time of the application interface in the frame before the frame to be displayed.

[0249] In an embodiment of the present application, the historical running time of each application interface is used to fit the load of each application interface, and the weighted average drawing time of the application interface is used as the drawing time of the application interface, so as to accurately estimate the drawing load size of the application interface.

[0250] In an embodiment of the present application, the load of a window is defined as the duration of the CPU processing the drawing operation of the window. Due to the operating system scheduling, the CPU core frequency changes, that is, the number of instructions executed by the CPU per unit time changes. Therefore, under the same drawing operation of the same application, the difference in the drawing duration of multiple frames is also obvious. The embodiment of the present application uses the weighted average drawing duration of multiple frames to relatively accurately estimate the window drawing load. The calculation method of estimating the weighted average drawing duration used in the embodiment of the present application can minimize the impact of severe load jitter caused by system scheduling.

[0251] In the embodiment of the present application, the weighted average drawing time The calculation method of is shown in equations (3) and (4).

[0252] Where n≥1,

[0253] a n-1=max(0.1,0.9×a n-2 ), where n≥2, a1=1 (4)

[0254] in, The estimated weighted average drawing time of the nth frame, that is, the drawing load of the nth frame. is the weighted average load of the first n-2 frames. n-1 is the measured load of the n-1th frame. n-1 is the weighted average factor, and its value is calculated by formula (4). In order to ensure the accuracy of the weighted average drawing time of the current frame, the weighted average factor of the weighted average drawing time of the historical frames has a greater impact on the weighted average drawing time of the current frame. Therefore, in the embodiment of the present application, the weight of the measured load of the n-1th frame is finally determined at an empirical value, as shown in formula (4), and the empirical value can be 0.1 or 0.2. When the window is drawn for the first time, that is, n is equal to 1, the definition of formula (3) is: = 0, and define the average load factor a1 of the first frame in formula (4) to be 1, then the weighted average load of the first frame is The actual load is equal to t1. When the second to nth frames are subsequently drawn, the weighted average factor is first calculated using formula (4), and then the weighted average drawing time of the current frame is calculated using formula (3).

[0255] In a possible implementation, the drawing time of the complete application interface may be the total number of rendering instructions for the application interface, where the total number of rendering instructions is a weighted calculation value of different types of rendering instructions, the number of rendering instructions, and their corresponding weights.

[0256] In the embodiment of the present application, the total number of rendering instructions of the application interface is used as the drawing duration of the application interface, which can accurately estimate the drawing load size of the application interface.

[0257] Specifically, first, determine the weight corresponding to each rendering instruction. The specific value of the weight can be set according to the actual situation. For example, the execution time required for a rendering instruction is actually measured on different devices, and then an average time is calculated as the final execution time of the rendering instruction. The rendering instruction with a longer execution time has a greater weight. Next, count the number of each rendering instruction required to draw the application interface, calculate the product of the weight of different rendering instructions and the corresponding number of instructions, and then sum the different products to obtain the total number of rendering instructions for the application interface.

[0258] The following example specifically illustrates the rendering method of the embodiment of the present application:

[0259] Refer to Figure 4A, which is a schematic diagram of a process flow of a rendering method provided by an embodiment of the present application. The specific process flow is as follows:

[0260] When the graphics rendering module of an electronic device starts, it first initializes the drawing thread pool that performs graphics rendering tasks. This involves creating drawing resources for each drawing thread. Drawing resources include two-dimensional (2D) engine instance resources and drawing backend resources. For example, if OpenGL is used as the rendering backend, to share drawing resources among multiple threads, the drawing threads create shared context resources based on the main thread's context.

[0261] The number of threads in the thread pool can be determined based on the CPU's hardware capabilities. Furthermore, for multi-core CPUs, the number of drawing threads is equal to the number of CPU cores, allowing each drawing thread to run on a separate CPU core at the same time, fully utilizing the CPU's multi-core capabilities. The number of drawing threads can also be less than the number of CPU cores, and there is no specific limit on this.

[0262] After initializing the thread pool, the drawing process for each frame begins. Each frame first checks whether a drawing signal has been received; if not, the thread pool suspends. When a frame drawing signal is received, the render tree information is first updated, including drawing operation information and render node attribute information. After completing this information update, the grouping results are determined.

[0263] Based on formula (1), the first value satisfies the preset condition to determine the grouping result of the frame to be displayed, wherein the grouping result includes how many parts each application interface is split into, the ratio of each part, and how at least one application interface is grouped. According to the grouping result, the drawing task of at least one application interface is processed to obtain at least one drawing task group, and the at least one drawing task group is added to the task queue. If there is no application interface that needs to be split, all drawing operation information contained in the frame to be displayed is packaged into one task. If there is an application interface that needs to be split, the application interface is processed according to the method shown in Figure 5 below.

[0264] Next, determine whether the task group to be drawn in the task queue is empty. If so, synthesize all the drawing results and send them to the display. Otherwise, search for an idle thread in the thread pool to handle the drawing task of the drawing task group, determine whether there is an idle thread in the thread pool, and if there is no idle thread, wait. If there is an idle thread, assign the drawing task group to be drawn to the idle thread, and let the idle thread perform the drawing task. After each task of the drawing task group is completed, save the drawing result of the drawing task group, mark the thread executing the drawing task of the drawing task group as idle, and delete the drawing task group from the task queue. Repeat the above process until all tasks in the task queue are completed, and then synthesize the drawing results of all tasks and send them to the display.

[0265] The following is an introduction to the advantages of the rendering method of the embodiment of the present application using a specific example. Taking an electronic device displaying multiple foreground applications as an example, refer to Figure 4B, which is a single-threaded single-frame drawing schematic provided by an embodiment of the present application; taking the frame to be displayed as an example for displaying five application interfaces: background application, vibrato application, Taobao application, status bar application and navigation bar application, wherein, in split-screen mode, one application (such as Taobao and vibrato) is displayed in the left and right areas of the screen, that is, two applications are displayed side by side. The status bar application is used to display time, battery power and network signal, etc. The navigation bar application is used to switch windows, etc. Both Taobao application and vibrato application are free windows, and the z-order of their windows can be changed. Assume that the z-order of Taobao application is large and the z-order of vibrato application is small. Then, according to the z-order of each application interface from small to large, the drawing order is background, vibrato, Taobao, status bar and navigation bar. When a single thread is used to process the drawing tasks of the frames to be displayed, rendering is performed according to the Z order of the application interface. It can be determined that the drawing time of a single frame is the sum of the drawing times of the five applications, that is, t1+t2+t3+t4+t5, for a total of 21ms. Among them, t1 is the drawing time of the background application, t2 is the drawing time of the Douyin application, t3 is the drawing time of the Taobao application, t4 is the drawing time of the status bar application, and t5 is the drawing time of the navigation bar application. t1 = 3ms, t2 = 10ms, t3 = 6ms, t4 = 1ms, and t5 = 1ms.

[0266] Referring to Figure 4C, Figure 4C is a schematic diagram of a multi-threaded rendering method provided by an embodiment of the present application. When the rendering method of the embodiment of the present application is used to process the rendering task of the above-mentioned frame to be displayed, assuming that the CPU of the electronic device has three cores, it can be divided into a maximum of three rendering task groups, and the maximum value of M is 3. Assume that the final grouping result determined by the method shown in Figure 2 is: the first rendering task group includes the background application and the Douyin application at a ratio of 0.4. The second group of rendering tasks includes the Douyin application at a ratio of 0.6 and the Taobao application at a ratio of 0.4. The third group of rendering tasks includes the Taobao application at a ratio of 0.6, the status bar, and the navigation bar. Because the grouping result determined by the rendering method of the present application can make the rendering time of each rendering task group equal or close, the single-frame rendering time is the rendering time of any rendering task group. Taking the first group of rendering tasks as an example, the single-frame rendering time is t1+t2-1+β+M*α=3+4+0.5+3*0.5=9ms. Compared with the single-threaded case, the rendering efficiency can be improved by 57.1%. In addition, in an embodiment of the present application, the rendering of five application interfaces can be completed in one process, so cross-window animation can be supported.

[0267] The following is a detailed description of the method for splitting the user interface provided in the embodiment of the present application.

[0268] Refer to Figure 5, which is a flowchart of another rendering method provided by an embodiment of the present application. The rendering method includes the following steps:

[0269] 501. Split the rendering tree corresponding to the user interface to obtain at least two spatial trees.

[0270] The user interface is at least one application interface. The state operation of the drawing node in each spatial tree remains unchanged before and after the split, and the state operation is an operation to modify the properties of the canvas. The at least two spatial trees are used to render the user interface.

[0271] Specifically, since the design composition logic of the controls of the application interface is a parent-child relationship, for example, a button control, its parent node is the background of the button, and the child node is the text on the button, etc. This type of parent-child relationship constitutes a complete rendering tree. Each node on the rendering tree includes layout information and recorded drawing operation information. Since the organizational relationship of the controls may have a mezzanine structure, that is, the rendering nodes may be spatially coupled, the drawing of two-dimensional controls is a series of stateful and sequential operations. If you want to split the drawing tasks of the application interface, you must ensure that the state and order of each drawing task after the split are consistent with the original serial drawing. Therefore, when splitting the rendering tree, the embodiment of the present application splits the drawing nodes corresponding to the user interface, and ensures that the state operations of the drawing nodes before and after the split remain unchanged, so as to split and obtain at least two spatial trees. Among them, the specific number of spatial trees can be determined according to actual conditions.

[0272] In an embodiment of the present application, the rendering tree corresponding to the user interface is split into at least two spatial trees, and the state operation of the drawing nodes in each spatial tree is guaranteed to remain unchanged before and after the split. In this way, when the above-mentioned user interface is rendered using at least two spatial trees, the correctness of the rendering result of the user interface can be ensured. In addition, the rendering task of the user interface is split into at least two spatial trees, and when rendering is performed using at least two spatial trees, correct parallel rendering can be supported.

[0273] In a possible implementation, the rendering areas corresponding to the at least two spatial trees overlap, which can reduce the time required for splitting the rendering tree and improve the efficiency of splitting the rendering tree.

[0274] In one possible implementation, when the rendering areas overlap, the rendering results of the overlapping parts may be erroneous. Therefore, in an embodiment of the present application, the rendering areas corresponding to the at least two spatial trees do not overlap. The non-overlapping rendering areas refer to completely non-overlapping rendering areas, which further ensures the correctness of the rendering results of the user interface.

[0275] In a possible implementation, step 501 includes:

[0276] 511. Determine the external graphics of the rendering area corresponding to each rendering node in the rendering tree.

[0277] In the embodiments of the present application, a drawing node refers to a rendering node in a rendering tree used to perform drawing operations, and a state operation node refers to a rendering node in a rendering tree used to perform state operations. For example, the drawing node and the state operation node can be the same rendering node, i.e., the rendering node is used to perform both drawing operations and state operations.

[0278] The circumscribed figure refers to a regular figure and / or a figure whose area can be calculated, such as a rectangle, circle, square, trapezoid, etc. For example, the circumscribed figure in the embodiment of the present application can be a circumscribed figure in the usual sense, that is, the circumscribed figure intersects with each vertex of the rendering area. For another example, the circumscribed figure in the embodiment of the present application can be a figure that includes the rendering area, that is, each vertex of the rendering area does not intersect with the circumscribed figure, but is at a certain distance. The specific value of the distance can be set according to actual needs and is not particularly limited here.

[0279] As a further example, in embodiments of the present application, the external graphics of a drawing node can be determined based on the layout information of the drawing node (including the position and shape of the rendering area, etc.). As another example, the bounding box of the drawing node can be directly used as the external graphics of the drawing node, eliminating the need to recreate the external graphics, thus reducing processing time and improving rendering efficiency.

[0280] In the embodiment of the present application, an external graphic corresponding to a rendering area of ​​a drawing node is determined, so as to approximately estimate the drawing load size of the drawing node based on the external graphic.

[0281] 512. Divide the external graphics corresponding to the drawing nodes in the rendering tree into at least two sets.

[0282] Specifically, the difference in the number of circumscribed graphics in any two of the at least two sets is less than a first threshold, and / or the difference in the sum of the areas of the circumscribed graphics in any two of the at least two sets is less than a second threshold. The specific values ​​of the first and second thresholds can be set based on actual conditions and are not particularly limited, so long as the number of circumscribed graphics in any two of the at least two sets is not significantly different, and / or the sum of the areas of the circumscribed graphics in any two of the at least two sets is not significantly different. This division ensures that the drawing loads of any two of the at least two spatial trees obtained after the split are similar, thus achieving a balanced split of the drawing load.

[0283] Exemplarily, if the difference in the number of external graphics in any two sets of at least two sets is less than a first threshold, the user interface can be abstracted into a two-dimensional array, and obtaining the external graphics of the drawn node is equivalent to marking in the two-dimensional array. At this time, it can be abstracted into a general problem: how to divide the areas of the two groups of arrays so that the number of external graphics in each area is as equal as possible. The layer-by-layer scanning line algorithm or the segment tree algorithm can be used to solve the above general problem.

[0284] Exemplarily, the rendering areas corresponding to the at least two sets overlap. This can reduce the time required for external graphics segmentation. Furthermore, exemplary, the rendering areas corresponding to the at least two sets do not overlap, which can further ensure the accuracy of the user interface rendering result.

[0285] As another example, in order to improve the accuracy of the division of the external graphics, the external graphics can be roughly divided into several small sets, and then the several small sets can be combined according to the first threshold and / or the second threshold to obtain the above-mentioned at least two sets. Each small set can be composed of a preset number of external graphics, or each small set can be composed of a preset number of adjacent external graphics. The specific value of the above-mentioned preset number can be set according to actual conditions and is not particularly limited. For example, the preset number is 2, 3, 4 or 5. Optionally, any two small sets also meet the conditions corresponding to the above-mentioned first threshold and / or second threshold. Similarly, a layer-by-layer scanning line algorithm or a segment tree algorithm can be used to divide and obtain several small sets.

[0286] 513. Split the rendering tree according to at least two sets to obtain at least two spatial trees.

[0287] The spatial tree corresponding to the set includes the drawing nodes corresponding to the external graphics in the set, and the state operation nodes passed when tracing back from the drawing nodes to the root node of the rendering tree.

[0288] Specifically, because the rendering areas corresponding to drawing nodes vary in shape, precise division of drawing nodes is impossible. Therefore, in the embodiments of the present application, the drawing nodes can be accurately divided using the external graphics corresponding to the drawing nodes. Furthermore, when dividing nodes in the rendering tree, the drawing nodes are divided first, and then the state operation nodes passed through during the backtracking process are grouped together with the drawing nodes. This ensures that the state operation of the drawing nodes before and after the division remains unchanged, ensuring that the resulting spatial tree can be rendered correctly.

[0289] In addition, when an external graphic is divided, when an external graphic is divided into at least two sets (taking two sets as an example, in this case, a part of the external graphic belongs to a set A, and another part of the external graphic belongs to another set B), when calculating the number of external graphics included in the set, the above-mentioned at least two sets need to include the external graphic. And the position of the rendering area of ​​the drawing node corresponding to the external graphic covers the rendering area corresponding to at least two spatial trees. Therefore, when rendering, it is assumed that at least two drawing threads are used to process the drawing tasks corresponding to at least two spatial trees corresponding to the above-mentioned at least two sets. Each drawing thread is responsible for processing the drawing tasks of the external graphic in the set corresponding to the drawing thread. For example, partial area a of the external graphic of the drawing node is located in set A, and the remaining partial area b of the external graphic except partial area a is located in set B. Drawing thread 1 corresponding to the spatial tree corresponding to set A processes the drawing tasks corresponding to partial area a, and drawing thread 2 corresponding to the spatial tree corresponding to set B processes the drawing tasks corresponding to partial area b.

[0290] The following takes the user interface as an application interface as an example to illustrate the specific process of user interface splitting:

[0291] Refer to Figure 6A, which is a schematic diagram of a rendering tree provided by an embodiment of the present application. Figure 6A shows a rendering tree for an application interface, where root is the root node of the rendering tree, nodes A, B, and C are state operation nodes, and nodes D, E, F, G, H, I, J, and K are drawing nodes.

[0292] First, as shown in FIG6B , FIG6B is a schematic diagram of a bounding box of a drawing node provided in an embodiment of the present application. The layout information of the rendering node is obtained from the rendering tree corresponding to the application interface 601. The layout information of the rendering node is then used to abstract the drawing nodes (such as node D, node E, etc.) into rectangles, i.e., bounding boxes.

[0293] Next, mark each node (i.e., drawing node) in the area to be rendered, such as "D," "E," "F," "G," "H," "I," "J," and "K" in Figure 6B . Then, divide the rectangle into at least two sets. For example, taking the example of dividing the rectangle into two sets, the rectangles corresponding to nodes D, E, J, and K are in one set, while the rectangles corresponding to nodes H, I, F, and G are in another set.

[0294] For example, in this embodiment, referring to FIG6C , FIG6C is a schematic diagram of a rectangle division provided by an embodiment of the present application; a layer-by-layer scan line algorithm (such as scan line 602 in FIG6C ) is used to divide the entire area to be rendered into multiple balanced small areas (i.e., the above-mentioned small sets), that is, the number of rectangles contained in each of the multiple divided small areas is approximately equal (the difference in the number of rectangles in each small area is less than the first threshold), and the small areas include a small area composed of a rectangle corresponding to node D and a rectangle corresponding to node E, a small area composed of a rectangle corresponding to node J and a rectangle corresponding to node K, a small area composed of a rectangle corresponding to node H and a rectangle corresponding to node I, and a small area composed of a rectangle corresponding to node F and a rectangle corresponding to node G. Then, according to actual needs, each small area is combined to obtain at least two sets. For example, taking the division of the rectangle to obtain two sets as an example, the small area composed of the rectangle corresponding to node D and the rectangle corresponding to node E, the small area composed of the rectangle corresponding to node J and the rectangle corresponding to node K are formed into one set, and the small area composed of the rectangle corresponding to node H and the rectangle corresponding to node I, and the small area composed of the rectangle corresponding to node F and the rectangle corresponding to node G are formed into another set.

[0295] Furthermore, the rendering tree is traversed based on the at least two sets to construct at least two spatial trees. The rendering nodes corresponding to the rectangles in the set are grouped into the same spatial tree. For each rectangle in the set, the state operation nodes passed through are traced back from the node corresponding to the rectangle to the root node, and are grouped into the spatial tree where the rectangle resides.

[0296] For example, referring to FIG6D , which is a schematic diagram of a rendering tree marking method provided by an embodiment of the present application, when obtaining at least two spatial trees based on at least two sets, binary number marking can be used for division. For example, the rectangles corresponding to nodes D, E, J, and K are set A, and the rectangles corresponding to nodes H, I, F, and G are set B. Since there are two sets, the nodes in the rendering tree can be marked with two binary numbers. When the node belongs to set B, the first binary number is 1; otherwise, it is 0. When the node belongs to set A, the second binary number is 1; otherwise, it is 0. The nodes corresponding to the rectangles and the state operation nodes passed through when tracing back to the root node using the above method can be marked, resulting in at least two spatial trees. For example, node D is marked as "01", indicating that node D belongs to set A, and node F is marked as "10", indicating that node F belongs to set B. The root node is marked as "11", indicating that the root node belongs to both set A and set B. The rendering tree in FIG6D is split into a spatial tree 603 corresponding to set A and a spatial tree 604 corresponding to set B as shown in FIG6E . It can be seen that when the drawing load is measured by the number of rectangles, since the number of rectangles corresponding to spatial tree 603 and spatial tree 604 is equal, their drawing load accounts for 1 / 2.

[0297] Then, based on the division of the spatial tree, the drawing operation information contained in each spatial tree is packaged into a single task. Since the spatial tree is constructed based on the spatial position of the UI control, each spatial tree represents the drawing task of an area. For example, after the rendering tree is split, there is no overlap in the rendering areas corresponding to each spatial tree, and the spatial tree can ensure that its drawing operation information is executed in the order of the guaranteed state from bottom to top, so the tasks corresponding to the spatial tree can also be executed in parallel. For example, as shown in Figure 6E, the rendering tree is split into a spatial tree 603 corresponding to set A and a spatial tree 604 corresponding to set B. Among them, rendering according to the spatial tree 603 can obtain the left half of the application interface 601, and rendering according to the spatial tree 604 can obtain the right half of the application interface 601.

[0298] When the system of an electronic device has only one foreground application, and the load of this application is very large, such as Douyin, the performance of the system depends on the slowest application, and the execution time of Douyin affects the overall performance. Referring to Figure 6F, Figure 6F is a thread allocation diagram of an application interface provided by an embodiment of the present application; wherein, thread 1 is responsible for processing the rendering task of the background application, and the drawing time required to render one frame is t1. Thread 2 is responsible for processing the rendering task of the Douyin application, and the drawing time required to render one frame is t2. Thread 3 is responsible for processing the rendering tasks of the status bar application and the navigation bar application, and the drawing times required to render one frame are t3 and t4 respectively. At this time, the time required to complete the rendering of a single frame depends on the thread with the longest drawing time, that is, the drawing time t2 corresponding to thread 2. According to the solution of the embodiment of the present application, the drawing task of the Douyin application can be split to obtain at least two sub-drawing tasks. At this time, at least two sub-drawing tasks can be assigned to at least two threads for processing, thereby reducing the drawing time required for a single frame. For example, Douyin is split into a sub-drawing task corresponding to Douyin-1 (the required drawing time is t2-1) and a sub-drawing task corresponding to Douyin-2 (the required drawing time is t2-2). Referring to Figure 6G, Figure 6G is a thread allocation diagram of another application interface provided by an embodiment of the present application. Assume that the sub-drawing task corresponding to Douyin-1 is assigned to thread 1 for processing, and the sub-drawing task corresponding to Douyin-2 continues to be assigned to thread 2 for processing. At this time, the drawing time of a single frame is t2-2. It can be seen that t2-2 is less than the drawing time t2 of Douyin. It can be seen that using the method of the present application, correct parallel rendering can be achieved to improve the rendering efficiency of the application interface.

[0299] The device provided by this application is described below.

[0300] Figures 7 and 8 are schematic diagrams of possible rendering devices provided in embodiments of the present application. These rendering devices can be used to implement the functions of the above-described method embodiments, thereby also achieving the beneficial effects of the above-described method embodiments. In embodiments of the present application, the rendering device can be an electronic device or a module (such as a chip) used in an electronic device.

[0301] As shown in Figure 7 , rendering apparatus 700 includes a determination module 701, a division module 702, and a processing module 703. Rendering apparatus 700 is configured to implement the functions of the method embodiment shown in Figure 2 . Alternatively, rendering apparatus 700 may include a module configured to implement any function or operation in the method embodiment shown in Figure 2 . This module may be implemented in whole or in part via software, hardware, firmware, or any combination thereof.

[0302] When the rendering device 700 is used to implement the functions of the method embodiment shown in Figure 2, the determination module 701 is used to determine the grouping results corresponding to the frames to be displayed. The frames to be displayed are used to display at least one application interface, and the grouping results are used to group the drawing tasks of the at least one application interface into at least one drawing task group so that the drawing time of the frames to be displayed is less than or equal to the sum of the drawing times of the at least one application interface. The division module 702 is used to divide the drawing tasks of the at least one application interface into at least one drawing task group according to the grouping results. The processing module 703 is used to process the at least one drawing task group using at least one drawing thread to obtain at least one drawing result.

[0303] In a possible implementation, the rendering device 700 further includes a sending module, wherein:

[0304] The sending module is used to send at least one drawing result.

[0305] In a possible implementation, the rendering device 700 further includes a splitting module, wherein:

[0306] A splitting module is configured to split the rendering tree corresponding to the user interface into at least two spatial trees. The state operations of the rendering nodes in each spatial tree remain unchanged before and after the split, and the state operations are operations that modify canvas properties. The at least two spatial trees are used to render the user interface. The user interface is at least one of at least one application interface.

[0307] In a possible implementation, the rendering areas corresponding to the at least two spatial trees do not overlap.

[0308] In one possible embodiment, the splitting module is specifically configured to: determine the bounding box of the rendering area corresponding to each drawing node in the rendering tree. Divide the bounding box corresponding to each drawing node in the rendering tree into at least two sets. The difference in the number of bounding boxes in any two of the at least two sets is less than a first threshold, and / or the difference in the sum of the areas of the bounding boxes in any two of the at least two sets is less than a second threshold. The rendering tree is split according to the at least two sets to obtain at least two spatial trees. The spatial trees corresponding to the sets include the drawing nodes corresponding to the bounding boxes in the sets, as well as the state operation nodes traversed when tracing back from the drawing nodes to the root node of the rendering tree.

[0309] In a possible implementation, the rendering areas corresponding to the at least two sets do not overlap.

[0310] The external graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

[0311] In a possible implementation, at the same time, one core of a central processing unit (CPU) in the electronic device executes one of the drawing threads.

[0312] In a possible implementation, the grouping result is a grouping scheme of at least one application interface whose first value satisfies a preset condition, and the first value of the grouping scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the grouping scheme.

[0313] In a possible embodiment, when the number of drawing task groups in the grouping scheme is two, the grouping scheme of the above-mentioned at least one application interface includes a first grouping scheme, and the first grouping scheme is to divide the application interface sequence into two parts with equal drawing time. The above-mentioned application interface sequence is a sequence obtained by arranging the above-mentioned at least one application interface in a preset order.

[0314] Optionally, the grouping scheme for the at least one application interface further includes a second grouping scheme and / or a third grouping scheme, wherein the second grouping scheme is to move the bisection point to the starting point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface. The bisection point is the position corresponding to the division of the application interface sequence into two parts of equal drawing duration, and the first application interface is one of the at least one application interface.

[0315] The third grouping scheme is to move the bisection point to the end point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface.

[0316] In one possible implementation, when the number of drawing task groups in the grouping scheme is M, M is greater than or equal to three, and the grouping scheme of the above-mentioned at least one application interface includes a fourth grouping scheme, and the fourth grouping scheme is to divide the application interface sequence into M parts with equal drawing time.

[0317] Optionally, the grouping scheme for at least one application interface further includes a fifth grouping scheme, wherein when the first M sub-point is within the drawing task range of the first application interface, the positions of M-2 of the M-1 M sub-points in the application interface sequence are fixed, and the first M sub-point is moved to the starting point of the drawing task of the first application interface. The M-1 M sub-points are positions corresponding to dividing the application interface sequence into M parts of equal drawing duration, and the first M sub-point is any one of the M-1 M sub-points.

[0318] Optionally, the grouping scheme of the above-mentioned at least one application interface also includes a sixth grouping scheme, and the above-mentioned sixth grouping scheme is that when the first M sub-point is located within the drawing task range of the first application interface, M-2 of the M-1 M sub-points in the above-mentioned application interface sequence are fixed, and the above-mentioned first M sub-point is moved to the end point of the drawing task of the first application interface.

[0319] In a possible implementation, the drawing duration of the drawing task group is at least one of the following:

[0320] When the drawing task group includes a drawing task for a complete application interface, the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group;

[0321] When a drawing task group includes drawing tasks for incomplete application interfaces, the drawing duration of the drawing task group is the first drawing duration. The first drawing duration is the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the segmentation duration. The segmentation duration is the product of the number of incomplete application interfaces and the segmentation overhead.

[0322] When the drawing task group includes a drawing task for a complete application interface and a drawing task for an incomplete application interface, the drawing duration of the drawing task group is the second drawing duration, the second drawing duration is the sum of the first drawing duration and the third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0323] In a possible implementation, the drawing time of the complete application interface is any one of the following:

[0324] The weighted average drawing time of the application interface, which is the weighted calculation value of the fourth drawing time, the fifth drawing time, and their respective weights. The fourth drawing time is the actual drawing time of the application interface in the second frame before the frame to be displayed. The fifth drawing time is the actual drawing time of the application interface in the frame before the frame to be displayed.

[0325] The total number of rendering instructions of the application interface is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

[0326] As shown in FIG8 , a rendering device 800 includes a splitting module 801. The rendering device 800 is configured to implement the functions of the method embodiment shown in FIG5 . Alternatively, the rendering device 800 may include a module configured to implement any function or operation in the method embodiment shown in FIG5 . The module may be implemented in whole or in part via software, hardware, firmware, or any combination thereof.

[0327] When rendering device 800 is used to implement the functions of the method embodiment shown in FIG5 , splitting module 801 is configured to split the rendering tree corresponding to the user interface, resulting in at least two spatial trees. The state operations of the rendering nodes in each spatial tree remain unchanged before and after the split; the state operations are operations that modify the properties of the canvas. These at least two spatial trees are used to render the user interface.

[0328] In a possible implementation, the rendering areas corresponding to the at least two spatial trees do not overlap.

[0329] In a possible implementation, the splitting module 801 is specifically configured to:

[0330] Determine the circumscribed graphics of the rendering area corresponding to each drawing node in the rendering tree. Divide the circumscribed graphics corresponding to the drawing nodes in the rendering tree into at least two sets. The difference between the number of circumscribed graphics in any two of the at least two sets is less than a first threshold, and / or the difference between the sum of the areas of the circumscribed graphics in any two of the at least two sets is less than a second threshold. Split the rendering tree according to the at least two sets to obtain at least two spatial trees. The spatial tree corresponding to a set includes the drawing nodes corresponding to the circumscribed graphics in the set, as well as the state operation nodes passed through when tracing back from the drawing nodes to the root node of the rendering tree. In one possible embodiment, the rendering areas corresponding to the at least two sets do not overlap.

[0331] In a possible implementation, the circumscribed graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

[0332] In a possible implementation, the rendering device further includes a determination module, wherein:

[0333] A determination module is configured to determine grouping results for frames to be displayed. The frames to be displayed are used to display the at least one application interface. The grouping results are used to organize the drawing tasks for the at least one application interface into at least one drawing task group, such that the drawing duration of the frames to be displayed is less than or equal to the sum of the drawing durations for the at least one application interface.

[0334] In a possible implementation, the rendering device further includes a division module and a processing module, wherein:

[0335] The division module is used to divide the drawing task of at least one application interface into at least one drawing task group according to the grouping result.

[0336] The processing module is used to process the drawing tasks of at least one drawing task group using at least one drawing thread to obtain at least one drawing result, and the at least one drawing result is used to synthesize the total drawing result of the frame to be displayed.

[0337] In a possible implementation, the rendering device further includes a sending module, wherein:

[0338] The sending module is used to send at least one drawing result.

[0339] In a possible implementation, at the same time, one core of a central processing unit (CPU) in the electronic device executes one drawing thread.

[0340] In one possible implementation, the grouping result is a grouping scheme of at least one application interface whose first value satisfies a preset condition, the first value of the grouping scheme is the sum of the drawing time and the synthesis time of the longest drawing task group corresponding to the grouping scheme, and the synthesis time is the product of the number of drawing task groups in the grouping scheme and the synthesis overhead.

[0341] In a possible implementation, when the number of drawing task groups in the grouping scheme is two, the grouping scheme of the above-mentioned at least one application interface includes a first grouping scheme, and the first grouping scheme is to divide the application interface sequence into two parts with equal drawing time. The above-mentioned application interface sequence is a sequence obtained by arranging the above-mentioned at least one application interface in a preset order.

[0342] Optionally, the grouping scheme for the at least one application interface further includes a second grouping scheme and / or a third grouping scheme. The second grouping scheme is to move the bisection point to the starting point of the first application interface's drawing task when the bisection point is within the drawing task range of the first application interface. The bisection point is the position corresponding to the division of the application interface sequence into two portions of equal drawing duration, and the first application interface is one of the at least one application interface.

[0343] The third grouping scheme is to move the bisection point to the end point of the drawing task of the first application interface when the bisection point is within the drawing task range of the first application interface.

[0344] In one possible implementation, when the number of drawing task groups in the grouping scheme is M, M is greater than or equal to three, and the grouping scheme of the above-mentioned at least one application interface includes a fourth grouping scheme, and the fourth grouping scheme is to divide the application interface sequence into M parts with equal drawing time.

[0345] Optionally, the grouping scheme for at least one application interface further includes a fifth grouping scheme, wherein when the first M sub-point is within the drawing task range of the first application interface, the positions of M-2 of the M-1 M sub-points in the application interface sequence are fixed, and the first M sub-point is moved to the starting point of the drawing task of the first application interface. The M-1 M sub-points are positions corresponding to dividing the application interface sequence into M parts of equal drawing duration, and the first M sub-point is any one of the M-1 M sub-points.

[0346] Optionally, the grouping scheme of the above-mentioned at least one application interface also includes a sixth grouping scheme, and the above-mentioned sixth grouping scheme is that when the first M sub-point is located within the drawing task range of the first application interface, M-2 of the M-1 M sub-points in the above-mentioned application interface sequence are fixed, and the above-mentioned first M sub-point is moved to the end point of the drawing task of the first application interface.

[0347] In a possible implementation, the drawing duration of the drawing task group is at least one of the following:

[0348] When the drawing task group includes a drawing task of a complete application interface, "include" here means only including, and the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0349] When the drawing task group includes the drawing task of an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the first drawing duration, the first drawing duration is the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the sum of the split durations, and the split duration is the product of the number of incomplete application interfaces and the splitting overhead.

[0350] When the drawing task group includes drawing tasks for a complete application interface and drawing tasks for an incomplete application interface, "include" here means only including, the drawing duration of the drawing task group is the second drawing duration, the second drawing duration is the sum of the first drawing duration and the third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

[0351] In one possible implementation, the drawing time of a complete application interface is any of the following:

[0352] The weighted average drawing time of the application interface. The weighted average drawing time is the weighted calculation value of the fourth drawing time, the fifth drawing time, and their respective weights. The fourth drawing time is the actual measured drawing time of the application interface in the second frame before the frame to be displayed. The fifth drawing time is the actual measured drawing time of the application interface in the frame before the frame to be displayed.

[0353] The total number of rendering instructions of the application interface is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

[0354] The present application also provides an electronic device, and the electronic device 900 shown in FIG9 includes a memory 901, a processor 902, a communication interface 904, and a bus 903. The memory 901, the processor 902, and the communication interface 904 are connected to each other via the bus 903.

[0355] Memory 901 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). Memory 901 can store programs. When the program stored in memory 901 is executed by processor 902, processor 902 and communication interface 904 are used to perform the various steps of the rendering method of the embodiment of the present application.

[0356] The processor 902 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits, which is used to execute relevant programs to implement the functions required to be performed by the units in the rendering device of the embodiment of the present application, or to execute the rendering method of the embodiment of the present application.

[0357] The processor 902 may also be an integrated circuit chip having signal processing capabilities. During implementation, the various steps of the rendering method of the embodiment of the present application may be completed by hardware integrated logic circuits or software instructions in the processor 902. The above-mentioned processor 902 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the rendering method in conjunction with the embodiment of the present application may be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in the memory 901, and the processor 902 reads the information in the memory 901 and combines its hardware to complete the functions required to be performed by the units included in the rendering device of the embodiment of the present application, or executes the rendering method of the embodiment of the present application.

[0358] The communication interface 904 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the electronic device 900 and other devices or a communication network. For example, data can be obtained through the communication interface 904.

[0359] The bus 903 may include a path for transmitting information between various components of the electronic device 900 (eg, the memory 901 , the processor 902 , and the communication interface 904 ).

[0360] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0361] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0362] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0363] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0364] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A rendering method, characterized in that: The method comprises: Determine a grouping result of frames to be displayed, wherein the frames to be displayed are used to display at least one application interface, and the grouping result is used to group the drawing tasks of the at least one application interface into at least one drawing task group, so that the drawing time of the frames to be displayed is less than or equal to the sum of the drawing time of the at least one application interface; Dividing the drawing task of the at least one application interface into the at least one drawing task group according to the grouping result; At least one drawing thread is used to process the drawing tasks of the at least one drawing task group to obtain at least one drawing result, and the at least one drawing result is used to synthesize and obtain the total drawing result of the frame to be displayed.

2. The method according to claim 1, characterized in that The method further comprises: The at least one drawing result is sent.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: The rendering tree corresponding to the user interface is split to obtain at least two spatial trees, and the state operation of the drawing nodes in each of the spatial trees remains unchanged before and after the splitting, and the state operation is an operation of modifying the properties of the canvas; the at least two spatial trees are used to render the user interface, and the user interface is at least one of the at least one application interface.

4. The method according to claim 3, characterized in that The rendering areas corresponding to the at least two spatial trees do not overlap.

5. The method according to claim 3 or 4, characterized in that: The rendering tree corresponding to the user interface is split to obtain at least two space trees, including: Determine an external graphic of a rendering area corresponding to each rendering node in the rendering tree; Dividing the circumscribed graphics corresponding to each rendering node in the rendering tree into at least two sets, wherein a difference between the numbers of the circumscribed graphics in any two of the at least two sets is less than a first threshold, and / or a difference between the sums of the areas of the circumscribed graphics in any two of the at least two sets is less than a second threshold; The rendering tree is split according to the at least two sets to obtain the at least two spatial trees.

6. The method according to claim 5, characterized in that The rendering areas corresponding to the at least two sets do not overlap.

7. The method according to claim 5 or 6, characterized in that: The external graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

8. The method according to any one of claims 1 to 7, characterized in that: At the same time, one core of the central processing unit (CPU) in the electronic device executes one of the drawing threads.

9. The method according to any one of claims 1 to 8, characterized in that: The grouping result is a grouping scheme of the at least one application interface whose first value satisfies a preset condition, and the first value of the grouping scheme is the sum of the drawing duration and the synthesis duration of the longest drawing task group corresponding to the grouping scheme.

10. The method according to claim 9, characterized in that The drawing duration of the drawing task group is at least one of the following: When the drawing task group includes a drawing task of a complete application interface, the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group; When the drawing task group includes a drawing task of an incomplete application interface, the drawing duration of the drawing task group is a first drawing duration, which is the sum of the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the segmentation duration, and the segmentation duration is the product of the number of the incomplete application interfaces and the segmentation overhead; When the drawing task group includes a drawing task of a complete application interface and a drawing task of an incomplete application interface, the drawing duration of the drawing task group is a second drawing duration, the second drawing duration is the sum of the first drawing duration and a third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

11. The method according to claim 10, characterized in that The drawing time of the complete application interface is any one of the following: a weighted average drawing time of the application interface, wherein the weighted average drawing time is a weighted calculation value of a fourth drawing time, a fifth drawing time and their respective weights, the fourth drawing time is an actually measured drawing time of the application interface in a second frame before the frame to be displayed, and the fifth drawing time is an actually measured drawing time of the application interface in a frame before the frame to be displayed; The total number of rendering instructions of the application interface, wherein the total number of rendering instructions is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

12. A rendering method, characterized in that: The method comprises: The rendering tree corresponding to the user interface is split to obtain at least two spatial trees, wherein the state operation of the drawing nodes in each of the spatial trees remains unchanged before and after the splitting, and the state operation is an operation of modifying the properties of the canvas; and the at least two spatial trees are used to render the user interface.

13. The method according to claim 12, characterized in that The rendering areas corresponding to the at least two spatial trees do not overlap.

14. The method according to claim 12 or 13, characterized in that The rendering tree corresponding to the user interface is split to obtain at least two space trees, including: Determine an external graphic of a rendering area corresponding to each rendering node in the rendering tree; Dividing the external graphics corresponding to the drawing nodes in the rendering tree into at least two sets, wherein the difference between the numbers of the external graphics in any two of the at least two sets is less than a first threshold, and / or the difference between the sums of the areas of the external graphics in any two of the at least two sets is less than a second threshold; The rendering tree is split according to the at least two sets to obtain the at least two spatial trees, wherein the spatial tree corresponding to the set includes a drawing node corresponding to the external graphics in the set and a state operation node passed when tracing back from the drawing node to the root node of the rendering tree.

15. The method according to claim 14, characterized in that The rendering areas corresponding to the at least two sets do not overlap.

16. The method according to claim 14 or 15, characterized in that The external graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

17. A rendering device, characterized in that: include: a determination module, configured to determine a grouping result of frames to be displayed, wherein the frames to be displayed are used to display at least one application interface, and the grouping result is used to group the drawing tasks of the at least one application interface into at least one drawing task group, so that the drawing time of the frames to be displayed is less than or equal to the sum of the drawing time of the at least one application interface; A division module, configured to divide the drawing task of the at least one application interface into the at least one drawing task group according to the grouping result; The processing module is used to use at least one drawing thread to process the drawing tasks of the at least one drawing task group to obtain at least one drawing result, and the at least one drawing result is used to synthesize and obtain the total drawing result of the frame to be displayed.

18. The device according to claim 17, characterized in that The device also includes: A sending module is used to send the at least one drawing result.

19. The device according to claim 17 or 18, characterized in that The device also includes: A splitting module is used to split the rendering tree corresponding to the user interface to obtain at least two spatial trees, and the state operation of the drawing nodes in each of the spatial trees remains unchanged before and after the splitting, and the state operation is an operation to modify the properties of the canvas; the at least two spatial trees are used to render the user interface, and the user interface is at least one of the at least one application interface.

20. The device according to claim 19, characterized in that The rendering areas corresponding to the at least two spatial trees do not overlap.

21. The device according to claim 19 or 20, characterized in that The splitting module is specifically used for: Determine an external graphic of a rendering area corresponding to each rendering node in the rendering tree; Dividing the circumscribed graphics corresponding to each rendering node in the rendering tree into at least two sets, wherein a difference between the numbers of the circumscribed graphics in any two of the at least two sets is less than a first threshold, and / or a difference between the sums of the areas of the circumscribed graphics in any two of the at least two sets is less than a second threshold; The rendering tree is split according to the at least two sets to obtain the at least two spatial trees.

22. The device according to claim 21, characterized in that The rendering areas corresponding to the at least two sets do not overlap.

23. The device according to claim 21 or 22, characterized in that The external graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

24. The device according to any one of claims 17 to 23, characterized in that At the same time, one core of the central processing unit (CPU) in the electronic device executes one of the drawing threads.

25. The device according to any one of claims 17 to 24, characterized in that The grouping result is a grouping scheme of the at least one application interface whose first value satisfies a preset condition, and the first value of the grouping scheme is the sum of the drawing duration and the synthesis duration of the longest drawing task group corresponding to the grouping scheme.

26. The device according to claim 25, characterized in that The drawing duration of the drawing task group is at least one of the following: When the drawing task group includes a drawing task of a complete application interface, the drawing duration of the drawing task group is the sum of the drawing durations of the complete application interfaces included in the drawing task group; When the drawing task group includes a drawing task of an incomplete application interface, the drawing duration of the drawing task group is a first drawing duration, which is the sum of the sum of the drawing durations of the incomplete application interfaces included in the drawing task group and the segmentation duration, and the segmentation duration is the product of the number of the incomplete application interfaces and the segmentation overhead; When the drawing task group includes a drawing task of a complete application interface and a drawing task of an incomplete application interface, the drawing duration of the drawing task group is a second drawing duration, the second drawing duration is the sum of the first drawing duration and a third drawing duration, and the third drawing duration is the sum of the drawing durations of the complete application interfaces included in the drawing task group.

27. The device according to claim 26, characterized in that The drawing time of the complete application interface is any one of the following: a weighted average drawing time of the application interface, wherein the weighted average drawing time is a weighted calculation value of a fourth drawing time, a fifth drawing time and their respective weights, the fourth drawing time is an actually measured drawing time of the application interface in a second frame before the frame to be displayed, and the fifth drawing time is an actually measured drawing time of the application interface in a frame before the frame to be displayed; The total number of rendering instructions of the application interface, wherein the total number of rendering instructions is a weighted calculation value of the number of different types of rendering instructions and their corresponding weights.

28. A rendering device, characterized in that: include: A splitting module is used to split the rendering tree corresponding to the user interface to obtain at least two spatial trees, wherein the state operation of the drawing nodes in each of the spatial trees remains unchanged before and after the splitting, and the state operation is an operation for modifying the properties of the canvas; the at least two spatial trees are used to render the user interface.

29. The device according to claim 28, characterized in that The rendering areas corresponding to the at least two spatial trees do not overlap.

30. The device according to claim 28 or 29, characterized in that The splitting module is specifically used for: Determine an external graphic of a rendering area corresponding to each rendering node in the rendering tree; Dividing the external graphics corresponding to the drawing nodes in the rendering tree into at least two sets, wherein the difference between the numbers of the external graphics in any two of the at least two sets is less than a first threshold, and / or the difference between the sums of the areas of the external graphics in any two of the at least two sets is less than a second threshold; The rendering tree is split according to the at least two sets to obtain the at least two spatial trees, and the spatial trees corresponding to the sets include The drawing nodes corresponding to the external graphics in the set, and the state operation nodes passed when tracing back from the drawing nodes to the root node of the rendering tree.

31. The device according to claim 30, characterized in that The rendering areas corresponding to the at least two sets do not overlap.

32. The device according to claim 30 or 31, characterized in that The external graphic of the rendering area corresponding to the rendering node is a bounding box of the rendering area corresponding to the rendering node.

33. An electronic device, characterized in that: It comprises a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute the rendering method according to any one of claims 1 to 16.

34. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the rendering method according to any one of claims 1 to 16.