Hotspot analysis method and device based on rendering performance data and electronic equipment

Abstract

The invention discloses a hotspot analysis method and device based on rendering performance data and electronic equipment, and the method comprises the steps: obtaining the power consumption information of an intelligent terminal, and determining a target image with the power consumption greater than a preset threshold value based on the power consumption information; extracting rendering information of the plurality of picture frames from the target picture; restoring state information of the rendering pipeline according to the rendering information, and obtaining a restored second shader set in the rendering pipeline; determining mutually matched shader combinations from the first shader set and the second shader set; according to the rendering data of the shaders of the first shader set, the rendering heating information of the multiple shader combinations in the corresponding picture frames is summarized and sorted, and a hotspot data aggregation result is obtained; and determining a performance hotspot object in the actual picture rendering process according to a hotspot data aggregation result. The method can accurately find the heating hot spot with the maximum performance consumption, is small in thermal granularity, can improve the user experience, and can be widely applied to the technical field of computers.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a hotspot analysis method, device and electronic equipment based on rendering performance data. Background Art

[0002] With the rapid development of mobile Internet, mobile terminals such as smart phones and tablets are becoming more and more popular, and APPs in various fields have emerged. At present, there are many types of APPs on the market, and the quality is uneven. Many APPs require mobile terminals to have strong graphics rendering performance to give full play to the complete functions of APPs. For example, game APPs, VR product display APPs, educational learning APPs, etc. all have high rendering performance requirements, but high rendering can easily cause the device to heat up too much, which in turn affects the overall performance of smart terminals. This hinders the further popularization of APPs with high rendering requirements and poor user experience.

[0003] When locating heat hotspots caused by high rendering, related technologies usually use a module comparison test method. By switching different modules on and off, and then comparing the changes in module power consumption before and after the switching, the impact of each module on power consumption is analyzed. This black box test analysis has a large thermal granularity and ultimately can only make trade-offs between modules. It is difficult to reduce power consumption within the module and has low accuracy. Summary of the invention

[0004] The main purpose of the embodiments of the present invention is to propose a hotspot analysis method, device and electronic device based on rendering performance data with small thermal granularity and high accuracy, which can accurately locate the hotspots of rendering performance data and thus improve the user experience.

[0005] To achieve the above object, an embodiment of the present invention provides a hot spot analysis method based on rendering performance data, comprising the following steps:

[0006] Acquire power consumption information of the smart terminal, and determine a target screen whose power consumption is greater than a preset threshold based on the power consumption information;

[0007] Extracting rendering information of at least one picture frame from the target picture; wherein the rendering information includes a first shader set used in an actual picture rendering process and rendering data of the shader;

[0008] Restoring state information of a rendering pipeline according to the rendering information, and obtaining a second shader set restored in the rendering pipeline;

[0009] Determine a shader combination that matches each other from the first shader set and the second shader set;

[0010] According to the rendering data of the shaders of the first shader set, the rendering heat generation information of the plurality of shader combinations in the corresponding picture frame is summarized and sorted to obtain a hotspot data aggregation result;

[0011] The performance hotspot object in the actual picture rendering process is determined according to the hotspot data aggregation result.

[0012] In some embodiments, the acquiring of power consumption information of the smart terminal and determining, based on the power consumption information, a target screen whose power consumption is greater than a preset threshold value comprises the following steps:

[0013] According to the power consumption query instruction, the power consumption of the smart terminal in the target application scenario is obtained to obtain the power consumption information of the smart terminal;

[0014] According to the power consumption information, the power consumption data of a plurality of candidate pictures are sorted, and the target picture with high power consumption is determined.

[0015] In some embodiments, extracting rendering information of at least one picture frame from the target picture comprises the following steps:

[0016] By using a rendering analysis tool, the rendering process of multiple picture frames is intercepted from the target picture to obtain the issuing time information of the drawing instruction corresponding to each picture frame;

[0017] According to the command time information, the rendering information of the corresponding time node is intercepted through a performance tool; wherein the rendering information includes performance indicator data of the smart terminal at the time node and rendering instructions; the rendering instructions include a first shader set used in the actual picture rendering process.

[0018] In some embodiments, restoring the state information of the rendering pipeline according to the rendering information and obtaining the second shader set restored in the rendering pipeline includes the following steps:

[0019] Obtaining the pipeline call state during the rendering process according to the performance indicator data in the rendering information;

[0020] According to the pipeline call state, a second shader set corresponding to each switching of the rendering pipeline is restored.

[0021] In some embodiments, obtaining the pipeline call state in the rendering process according to the performance indicator data in the rendering information includes the following steps:

[0022] Get rendering information captured by performance tools;

[0023] Decompressing the rendering information to obtain performance indicator data and a rendering database in the rendering information; wherein the rendering database is used to store the corresponding relationship between the rendering pipeline and the shader;

[0024] According to the performance indicator data, first identification information of each rendering pipeline when switching the rendering pipeline during the rendering process is determined.

[0025] In some embodiments, restoring the second shader set corresponding to each switching of the rendering pipeline according to the pipeline call state includes the following steps:

[0026] Determining, from the rendering database, second identification information of a shader used by the rendering pipeline according to the first identification information of the rendering pipeline;

[0027] According to the correspondence between the first identification information and the second identification information, a shader used by the current rendering pipeline is determined each time the rendering pipeline is switched, and the second shader set is constructed.

[0028] In some embodiments, the step of restoring the state information of the rendering pipeline according to the rendering information and obtaining the second shader set restored in the rendering pipeline further includes the following steps:

[0029] Determining a calling order of the rendering pipeline in a rendering process according to the performance indicator data and the first identification information of the rendering pipeline;

[0030] Recording the second identification information of the shader used by the rendering instruction involved after each configuration of the calling state of the rendering pipeline;

[0031] The second identification information of the shader used by the current rendering instruction is used as the identification information of the shader matched by the rendering pipeline, and the pairing relationship between the rendering pipeline and the shader is stored in the rendering database.

[0032] In some embodiments, determining a matching shader combination from the first shader set and the second shader set comprises the following steps:

[0033] Extracting drawing instructions from rendering information obtained by a rendering analysis tool;

[0034] Extracting a first shader file of each shader in a first shader set used by the drawing instruction, and extracting a second shader file of each shader in the second shader set;

[0035] Calculating file identification information of the first shader file and the second shader file respectively;

[0036] When the file identification information of the first shader file is the same as the file identification information of the second shader file, a corresponding shader matching relationship is established to obtain the shader combination.

[0037] In some embodiments, respectively calculating the file identification information of the first shader file and the second shader file comprises the following steps:

[0038] Using a hash algorithm, respectively calculating a first hash value of the first shader file and a second hash value of the second shader file;

[0039] Using the first Hash value and the second Hash value as the file identification information;

[0040] Wherein, the hash values ​​of two shaders that match each other in the shader combination are equal.

[0041] In some embodiments, according to the rendering data of the shader of the first shader set, the rendering heat generation information of the plurality of shader combinations in the corresponding picture frame is aggregated and sorted to obtain the hotspot data aggregation result, including the following steps:

[0042] According to the rendering data of the shaders of the first shader set, the parameters to be sorted corresponding to each of the shader combinations are obtained; wherein the parameters to be sorted include at least one of the total drawing time of each frame, bandwidth occupancy, number of floating-point operation instructions, and number of sampling instructions;

[0043] According to the parameters to be sorted, the parameters are summarized and sorted in ascending or descending order of numerical values ​​to obtain the order of each shader combination, and then the hot spot data aggregation result is determined.

[0044] In some embodiments, determining the performance hotspot object in the actual screen rendering process according to the hotspot data aggregation result includes the following steps:

[0045] Determine the performance consumption contribution of each pair of shader combinations to the actual picture rendering process according to the hotspot data aggregation result;

[0046] A shader combination whose performance consumption contribution is higher than a preset threshold is determined as a performance hotspot object in the actual picture rendering process.

[0047] In some embodiments, the method further comprises:

[0048] Optimization processing is performed on the performance hotspot object; wherein the optimization processing includes at least one of stopping the shader execution action, restarting the shader execution action, and reconfiguring the shader to be executed in the rendering pipeline.

[0049] Another aspect of the embodiment of the present invention further provides a hot spot analysis device based on rendering performance data, comprising:

[0050] The first module is used to obtain power consumption information of the smart terminal, and determine a target screen whose power consumption is greater than a preset threshold based on the power consumption information;

[0051] A second module is used to extract rendering information of at least one picture frame from the target picture; wherein the rendering information includes a first shader set used in an actual picture rendering process and rendering data of the shader;

[0052] A third module is used to restore the state information of the rendering pipeline according to the rendering information, and obtain a second shader set restored in the rendering pipeline;

[0053] A fourth module, configured to determine a matching shader combination from the first shader set and the second shader set;

[0054] A fifth module is used to summarize and sort the rendering heat generation information of multiple shader combinations in corresponding picture frames according to the rendering data of the shaders of the first shader set, so as to obtain a hotspot data aggregation result;

[0055] The sixth module is used to determine the performance hotspot object in the actual picture rendering process according to the hotspot data aggregation result.

[0056] In some embodiments, the first module includes:

[0057] The first unit is used to perform power consumption acquisition processing on the operation status of the smart terminal in the target application scenario according to the power consumption query instruction to obtain the power consumption information of the smart terminal;

[0058] The second unit is configured to sort the power consumption data of a plurality of candidate pictures according to the power consumption information, and determine the target picture with high power consumption.

[0059] In some embodiments, the second module includes:

[0060] The third unit is used to intercept the rendering process of multiple picture frames from the target picture through a rendering analysis tool to obtain the issuing time information of the drawing instruction corresponding to each picture frame;

[0061] The fourth unit is used to intercept the rendering information of the corresponding time node through the performance tool according to the command time information; wherein the rendering information includes the performance indicator data and rendering instructions of the smart terminal at the time node; the rendering instructions include the first shader set used in the actual picture rendering process.

[0062] In some embodiments, the third module includes:

[0063] A fifth unit, configured to obtain a pipeline call state in a rendering process according to performance indicator data in the rendering information;

[0064] The sixth unit is used to restore, according to the pipeline calling state, a second shader set corresponding to each switching of the rendering pipeline.

[0065] In some embodiments, the fifth unit comprises:

[0066] A first subunit is used to obtain rendering information intercepted by a performance tool;

[0067] A second subunit is used to decompress the rendering information to obtain performance indicator data and a rendering database in the rendering information; wherein the rendering database is used to store the corresponding relationship between the rendering pipeline and the shader;

[0068] The third subunit is used to determine the first identification information of each rendering pipeline when switching the rendering pipeline during the rendering process according to the performance indicator data.

[0069] In some embodiments, the sixth unit comprises:

[0070] A fourth subunit, configured to determine, from the rendering database, second identification information of a shader used by the rendering pipeline according to the first identification information of the rendering pipeline;

[0071] The fifth subunit is used to determine the shader used by the current rendering pipeline each time the rendering pipeline is switched according to the correspondence between the first identification information and the second identification information, and to construct the second shader set.

[0072] In some embodiments, the third module further includes:

[0073] A seventh unit, configured to determine a calling order of the rendering pipeline in a rendering process according to the performance indicator data and the first identification information of the rendering pipeline;

[0074] An eighth unit is used to record the second identification information of the shader used by the rendering instruction involved after the calling state of the rendering pipeline is configured each time;

[0075] The ninth unit is configured to use the second identification information of the shader used by the current rendering instruction as the identification information of the shader matched by the rendering pipeline, and store the pairing relationship between the rendering pipeline and the shader in a rendering database.

[0076] In some embodiments, the fourth module includes:

[0077] A tenth unit is used for extracting drawing instructions from rendering information obtained by a rendering analysis tool;

[0078] An eleventh unit is used to extract a first shader file of each shader in the first shader set used by the drawing instruction, and to extract a second shader file of each shader in the second shader set;

[0079] A twelfth unit, used to calculate the file identification information of the first shader file and the second shader file respectively;

[0080] The thirteenth unit is used to establish a corresponding shader matching relationship to obtain the shader combination when the file identification information of the first shader file is the same as the file identification information of the second shader file.

[0081] In some embodiments, the twelfth unit comprises:

[0082] a sixth subunit, configured to respectively calculate a first hash value of the first shader file and a second hash value of the second shader file by using a hash algorithm;

[0083] A seventh subunit, configured to use the first Hash value and the second Hash value as the file identification information;

[0084] Wherein, the hash values ​​of two shaders that match each other in the shader combination are equal.

[0085] In some embodiments, the fifth module includes:

[0086] A fourteenth unit is used to obtain a parameter to be sorted corresponding to each of the shader combinations according to the rendering data of the shader of the first shader set; wherein the parameter to be sorted includes at least one of the total drawing time of each frame, bandwidth occupancy, number of floating-point operation instructions, and number of sampling instructions;

[0087] The fifteenth unit is used to summarize and sort the parameters to be sorted in ascending or descending order of numerical values ​​to obtain the order of each shader combination, and then determine the hot spot data aggregation result.

[0088] In some embodiments, the sixth module includes:

[0089] A sixteenth unit is used to determine the performance consumption contribution of each pair of shader combinations to the actual picture rendering process according to the hotspot data aggregation result;

[0090] The seventeenth unit is used to determine the shader combination whose performance consumption contribution is higher than a preset threshold as a performance hotspot object in the actual picture rendering process.

[0091] In some embodiments, the method further comprises:

[0092] The seventh module is used to perform optimization processing on the performance hotspot object; wherein the optimization processing includes at least one of stopping the shader execution action, restarting the shader execution action, and reconfiguring the shader to be executed in the rendering pipeline.

[0093] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0094] The memory is used to store programs;

[0095] The processor executes the program to implement the method described above.

[0096] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0097] Another aspect of an embodiment of the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0098] The embodiments of the present invention include at least the following beneficial effects: the present invention provides a hotspot analysis method, device and electronic device based on rendering performance data, the scheme first obtains the power consumption information of the intelligent terminal, and determines the target screen whose power consumption is greater than a preset threshold based on the power consumption information; then extracts the rendering information of at least one screen frame from the target screen; wherein the rendering information includes a first shader set and rendering data of the shader used in the actual screen rendering process; and restores the state information of the rendering pipeline according to the rendering information, and obtains the second shader set restored in the rendering pipeline; then determines the mutually matching shader combination from the first shader set and the second shader set; and according to the rendering data of the shader of the first shader set, summarizes and sorts the rendering heat information of multiple shader combinations in the corresponding screen frame to obtain the hotspot data aggregation result; finally, determines the performance hotspot object in the actual screen rendering process according to the hotspot data aggregation result. The present invention can accurately find the heat hotspots with the largest performance consumption by analyzing the power consumption information and restoring the shader state of the pipeline rendering process when the smart terminal performs high rendering processing. Compared with the module comparison test method in the prior art, the thermal granularity of this solution is small and the accuracy is high. It can accurately locate the hotspots of rendering performance data and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

[0100] Figure 2 is a flow chart of the overall steps provided by an embodiment of the present invention;

[0101] Figure 3 It is a flowchart of a specific implementation process provided by an embodiment of the present invention;

[0102] Figure 4 is a flowchart of specific implementation steps of step S210 provided in an embodiment of the present invention;

[0103] Figure 5 is a flowchart of specific implementation steps of step S220 provided in an embodiment of the present invention;

[0104] Figure 6 is a flowchart of specific implementation steps of step S230 provided in an embodiment of the present invention;

[0105] Figure 7 is a flowchart of specific implementation steps of step S240 provided in an embodiment of the present invention;

[0106] Figure 8 is a flowchart of specific implementation steps of step S250 provided in an embodiment of the present invention;

[0107] Fig. 9 is a flowchart of specific implementation steps of step S260 provided in an embodiment of the present invention;

[0108] Fig.10 A schematic diagram of the operation interface of the RenderDoc tool according to an embodiment of the present invention;

[0109] Fig.11 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0110] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0111] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0112] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0113] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0114] The hotspot analysis method, device and electronic device based on rendering performance data provided by the embodiments of the present invention relate to the field of computer technology. The hotspot analysis method based on rendering performance data provided by the embodiments of the present invention can be applied to a terminal, can be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a hotspot analysis method based on rendering performance data, etc., but is not limited to the above forms.

[0115] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0116] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0117] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0118] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0119] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal 102 may also be a vehicle-mounted terminal of various device types described above, but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the present invention.

[0120] Based on the example Figure 1In the implementation environment shown, an embodiment of the present invention provides a hotspot analysis method based on rendering performance data. The following is explained using the example of applying the hotspot analysis method based on rendering performance data in the terminal 102. It can be understood that the method can also be applied to the server 101.

[0121] It should be noted that in the relevant technology, the method for locating rendering performance hotspots usually adopts module comparison test or data analysis method. Among them, for the module comparison test method, the influence of each module on power consumption is generally analyzed by comparing the power consumption changes before and after switching different modules. The thermal granularity of this black box test analysis is large, and it can only make trade-offs between modules, and it is difficult to reduce power consumption for modules. As for the data analysis method, the time consumption, memory bandwidth, and floating-point operation volume data of each rendering in the rendering process are generally sorted and compared, and the performance hotspots are determined by analyzing the drawing calls with high time consumption, high bandwidth, and operation volume. This method has a large workload, because the drawing of each frame in the application requires hundreds of drawing calls. On the one hand, the amount of data is large and difficult to analyze in detail. On the other hand, these collected performance data are not intuitive enough and are not convenient to correspond to the actual rendering screen, so as to optimize the number of faces, shaders, pipeline processes, etc. of these rendering objects in a targeted manner.

[0122] For common graphics processing, it is generally as follows: a graphics processing unit (GPU) implements a graphics processing pipeline that simultaneously processes copies of commands retrieved from a command buffer. The graphics pipeline includes one or more shaders and one or more fixed-function hardware blocks that are executed using the resources of the graphics pipeline. The graphics pipeline is usually divided into a geometric part that performs geometric operations on patches or other primitives (such as triangles formed by vertices and edges and representing parts of an image). The shaders in the geometric part may include vertex shaders, full shaders, domain shaders, and geometry shaders. When the primitives generated by the geometric part of the pipeline are rasterized (for example, by one or more scan converters) to form a set of pixels representing a part of the image, the geometric part of the graphics pipeline is completed. Subsequent processing of pixels is called pixel processing and includes operations performed by shaders (such as pixel shaders executed using the resources of the graphics pipeline). GPUs and other multithreaded processing units usually implement multiple processing elements (also referred to as processor cores or computing units) that simultaneously execute multiple instances of a single program on multiple data sets as a single wave. A hierarchical execution model is used to match the hierarchical structure implemented in hardware. The execution model defines the kernel of instructions executed by all waves (also called wavefronts, threads, streams, or work items). Therefore, a rendering hotspot location method with small thermal granularity and high accuracy is needed to find the hot spots in the image rendering process, so as to find targeted optimization targets and optimization directions in the many process steps involved in the above-mentioned graphics processing process and improve processing efficiency.

[0123] In view of this, in an embodiment of the present invention, a hotspot analysis method based on rendering performance data is proposed, which first obtains the power consumption information of the intelligent terminal, and determines the target screen whose power consumption is greater than a preset threshold based on the power consumption information; then extracts the rendering information of at least one screen frame from the target screen; wherein the rendering information includes a first shader set and the rendering data of the shader used in the actual screen rendering process; and restores the state information of the rendering pipeline according to the rendering information, and obtains the second shader set restored in the rendering pipeline; then determines the mutually matching shader combination from the first shader set and the second shader set; and according to the rendering data of the shader of the first shader set, summarizes and sorts the rendering heat information of the plurality of shader combinations in the corresponding screen frame to obtain the hotspot data aggregation result; finally, determines the performance hotspot object in the actual screen rendering process according to the hotspot data aggregation result. The present invention can accurately find the heat hotspot with the largest performance consumption by analyzing the power consumption information and restoring the shader state of the pipeline rendering process for the intelligent terminal when performing high rendering processing. Compared with the module comparison test method of the prior art, the thermal granularity of this scheme is small and the accuracy is high, which can accurately locate the hotspot of rendering performance data and improve the user experience.

[0124] Reference Figure 2 , Figure 2 A flowchart of a hotspot analysis method based on rendering performance data applied to a terminal provided in an embodiment of the present invention, wherein the execution subject of the method may be any of the aforementioned computer devices (including a server or a terminal).

[0125] Reference Figure 2 , the method may include the following steps:

[0126] S210, acquiring power consumption information of the smart terminal, and determining a target screen whose power consumption is greater than a preset threshold based on the power consumption information;

[0127] Specifically, the intelligent terminal of the embodiment of the present invention can be a convenient device such as a smart phone, a tablet computer, a laptop computer, etc., which is not limited here. In some embodiments, when the relevant smart terminal runs a high-rendering APP, there will be a situation of excessive heat and excessive power consumption, which will cause the performance of the smart device to decline. For example, when a game APP is opened through a smart phone, the real-time changing game screen will have a higher rendering performance requirement. At this time, it is easy to appear the technical problem to be solved mentioned in the present invention, how to locate the heating hotspot, so as to perform targeted optimization. The embodiment of the present invention is to obtain the corresponding high-power target screen in the high-rendering process. Among them, in some embodiments, a power consumption threshold can be preset, and then the power consumption information of each screen is compared with the power consumption threshold in step S210. When it is higher than the power consumption threshold, the corresponding target screen is obtained. The power consumption threshold here can be flexibly set according to the change of the application scenario. For example, for the screens of some scenes with faster power consumption changes, such as game scenes, the power consumption threshold can be set higher, and in some scenes with slower power consumption changes, such as browsing novel scenes, the power consumption threshold can be set lower, which is not limited here.

[0128] It should be noted that the hotspot analysis method provided in the embodiment of the present invention can also be applied to other scenarios with high rendering requirements for terminal devices, such as VR house viewing, product display, educational live broadcast and other fields, which are not limited here.

[0129] S220, extracting rendering information of at least one picture frame from the target picture; wherein the rendering information includes a first shader set used in an actual picture rendering process and rendering data of the shader;

[0130] Among them, a shader is a computer program, mainly used in the field of computer graphics, for processing image shading, calculating the lighting, brightness, color, etc. in the image, and is also used in processing CG special effects, film post-processing and other fields. Shaders have a high degree of freedom in calculating rendering effects on graphics hardware, allowing developers to program them using shader languages. The position, hue, saturation, brightness, contrast, etc. of the pixels, vertices, and textures that make up the final image can be dynamically adjusted using the algorithms defined in the shader. In addition, external programs that call shaders can also use the external variables and textures it provides to the shader to modify the parameters in these shaders.

[0131] Specifically, the embodiment of the present invention extracts the corresponding screen frame for each target screen captured, and obtains the corresponding rendering information. Exemplarily, the rendering information may include performance indicators such as memory bandwidth, number of sampling instructions, number of floating-point operation instructions, and all rendering instructions. In addition, the embodiment of the present invention may obtain rendering information of at least one screen frame, and perform corresponding processing on the rendering information of one or more screen frames obtained in subsequent steps, and the number of screen frames is not limited here.

[0132] S230, restoring state information of a rendering pipeline according to the rendering information, and obtaining a second shader set restored in the rendering pipeline;

[0133] The rendering pipeline is a key component of the graphics rendering process, which is responsible for converting 3D scenes into 2D images. The functions of the rendering pipeline mainly include converting the 3D coordinates of objects into 2D coordinates in screen space and shading each pixel on the screen. This process involves multiple stages, and each stage uses the output of the previous stage as the input of this stage. The following are the main stages of the rendering pipeline and their functions:

[0134] Application phase: occurs in the CPU and is fully controlled by the developer. The CPU pre-processes the data to be rendered and sends the results to the GPU for further processing. This phase involves sending information such as rendering primitives and rendering methods to the GPU.

[0135] Geometry processing stage: includes vertex shader, which is a programmable stage responsible for coordinate transformation, such as transforming the input local coordinates into world coordinates, view coordinates and clipping coordinates. In addition, this stage can also control some attributes of the vertex, such as calculating color information and lighting information.

[0136] Rasterization stage: The process of converting the 3D coordinates of an object into 2D coordinates in screen space. This stage is done automatically by the hardware and involves converting objects in the 3D scene into pixel representations on the screen.

[0137] Pixel stage: Shading for each pixel on the screen, which includes steps such as fragment shader and blending test. The fragment shader performs lighting calculations, while the blending test determines the final pixel's color and transparency.

[0138] Specifically, the pipeline status information in the embodiment of the present invention may include the pipeline calling sequence, calling time, pipeline ID, pipeline corresponding shader and other contents, which are not limited here. Through the pipeline status information, the corresponding shader set can be mirrored and restored to determine which shaders are used in the process of calling each pipeline.

[0139] S240, determining a shader combination that matches each other from the first shader set and the second shader set;

[0140] Specifically, step S220 of the embodiment of the present invention parses the extracted rendering information to obtain the first shader, and then reversely maps and infers the second shader from the pipeline data in S230, so that a shader combination that can match the previous two steps can be found in step S240.

[0141] S250, according to the rendering data of the shaders of the first shader set, summarizing and sorting the rendering heat generation information of the plurality of shader combinations in the corresponding picture frame to obtain a hotspot data aggregation result;

[0142] It is understandable that real-time rendering is a complex process that mixes a variety of different solutions. It can be essentially regarded as two stages: the pre-calculation stage and the specific real-time rendering stage. For the problem of affecting rendering performance and causing excessive heating of rendering, the following common reasons are: 1. GPU load is too high: When too many complex special effects, high-resolution textures and models are involved in the rendering process, the GPU load will be too high, which will cause the phone or computer to heat up. This is because the GPU needs to handle a lot of graphics rendering work, including special effects rendering, loading of high-resolution textures, etc. These operations will consume a lot of computing resources and cause the device to heat up. 2. Excessive CPU calculation: Complex game logic and physical calculations will cause the CPU load to be too high, causing the phone or computer to heat up. The CPU needs to handle a large number of computing tasks, such as complex algorithms, physical simulations, etc. These computing-intensive tasks will consume a lot of computing resources and generate heat. 3. Unreasonable memory usage: Excessive resource loading and untimely resource release will cause excessive memory usage, which will cause the device to heat up. Unreasonable memory management will cause excessive memory usage, increase the burden on the device, and cause heating problems. In order to solve these problems, a series of optimization measures can be taken, such as reducing texture size, reducing rendering quality, using object pool technology to manage the creation and destruction of game objects, timely releasing unused resources, using texture compression technology, and using memory management tools properly. These measures help to reduce the load on the GPU and CPU, reduce memory usage, and thus reduce device heating problems.

[0143] However, the current related technologies cannot specifically find the heating hotspots of the rendering process, and can only be achieved by checking one by one, so the efficiency is low and lacks accuracy. In an embodiment of the present invention, by collecting the hotspot data aggregation results, such as the total drawing time, bandwidth occupancy, number of floating-point operation instructions, number of sampling instructions, etc. occupied by the shader in the rendering process, the heating hotspots can be located in subsequent steps based on the aggregation results, thereby achieving efficient and accurate subsequent rendering optimization and other processing. It should be noted that the hotspot data aggregation results listed in the embodiments of the present invention are not limited to the above examples, and the content of the collected hotspot data is not limited here.

[0144] S260 , determining the performance hotspot object in the actual picture rendering process according to the hotspot data aggregation result.

[0145] Specifically, an embodiment of the present invention can determine the shader combination that is ranked high in the above step S250, that is, the shader combination with a higher power consumption ratio, as a performance hotspot object, so as to accurately locate the hotspot of rendering performance data of the smart terminal during high rendering, and can execute subsequent optimization measures to improve user experience.

[0146] like Figure 4As shown, in some embodiments, the above step S210, obtaining the power consumption information of the smart terminal, and determining the target screen whose power consumption is greater than a preset threshold based on the power consumption information, includes the following steps:

[0147] S2101, performing power consumption acquisition processing on the operation status of the smart terminal in the target application scenario according to the power consumption query instruction, to obtain power consumption information of the smart terminal;

[0148] S2102: Sort the power consumption data of multiple candidate pictures according to the power consumption information, and determine the target picture with high power consumption.

[0149] Specifically, when the smart terminal is a mobile phone terminal, the embodiment of the present invention can use the power consumption query instruction adb shell to obtain information such as power, ROM, memory, frame rate, etc. of the Android device.

[0150] It should be noted that the adb shell command in the embodiment of the present invention can obtain the operating status of the Android device, such as battery information (including power percentage, battery status, battery temperature, battery voltage, charge and discharge current), CPU share, memory share, total ROM size, ROM used size, ROM available size, APP interface frame rate, etc., and encapsulate it into a bat script.

[0151] Exemplarily, the query instruction for the relevant power consumption information may be, for example:

[0152] 1. Check the phone battery information: adb shell dumpsys battery

[0153] 2. Set the battery to charging status - adb shell dumpsys battery set status 2

[0154] 3. Set the battery to non-charging state - adb shell dumpsys battery set status 1

[0155] 4. Set the battery percentage - adb shell dumpsys battery set level 100

[0156] 5. Reset and restore the actual state - adb shell dumpsys battery reset

[0157] 6. Set to disconnect charging (Android 6.0 and above)——adb shell dumpsys battery unplug.

[0158] In addition, this embodiment illustrates the method used by a smartphone terminal running the Android system to obtain power consumption information. Relevant technical personnel can use smartphones running the IOS system or Hongmeng system, or terminal devices such as tablets running Windows, and can use corresponding power consumption query instructions to obtain power consumption information, which will not be repeated here.

[0159] In some embodiments, the above step S220, extracting rendering information of at least one picture frame from the target picture, includes the following steps:

[0160] S2201, intercepting the rendering process of multiple picture frames from the target picture through a rendering analysis tool, and obtaining the issuing time information of the drawing instruction corresponding to each picture frame;

[0161] Specifically, in the embodiment of the present invention, the RenderDoc cross-platform rendering framework analysis tool is used to capture the rendering process of multiple picture frames from the target picture.

[0162] Among them, RenderDoc has the following four advantages: 1) It can capture frame buffer information at runtime and then play it back at any time point to analyze the generation process of each frame. 2) It can display the status of resources such as textures, buffers, and shader constants in detail, making it convenient for users to check whether the data is correct. 3) Through the event tree structure, you can traverse the rendering commands one by one to view the effect of each operation. 4) RenderDoc provides a detailed view of the underlying API calls, which can deeply understand performance bottlenecks and optimization directions.

[0163] The embodiment of the present invention uses the above-mentioned RenderDoc tool to find the DrawCall time point of the rendering process corresponding to the picture frame, so as to find the working time of the corresponding shader. It should be noted that DrawCall is a command sent by the CPU to the GPU to render a mesh. This command only specifies whether a mesh is rendered / drawn and whether any material information is drawn. After obtaining the command, the GPU obtains the value of the rendering state (material, texture, shader, etc.), and all vertex data on the screen is converted into corresponding pixels through some code logic.

[0164] S2202. According to the command time information, the rendering information of the corresponding time node is intercepted through a performance tool; wherein the rendering information includes the performance indicator data of the smart terminal at the time node and the rendering instructions; the rendering instructions include a first shader set used in the actual picture rendering process.

[0165] Specifically, the embodiment of the present invention may use the performance tool Snapdragon Profiler to intercept performance indicators such as memory bandwidth, number of sampling instructions, number of floating-point operation instructions, and all rendering instructions of the scene.

[0166] Among them, Snapdragon Profiler can analyze software running on platforms such as Windows, Mac, and Linux. It connects Android devices powered by Snapdragon processors via USB. Snapdragon Profiler allows developers to analyze CPU, GPU, DSP, memory, power, thermal and network data so that they can find and fix performance bottlenecks. The main features of Snapdragon Profiler include: 1. Real-time view: You can easily associate system resource usage on the timeline, providing real-time monitoring of CPU, GPU, DSP, memory, power, thermal and network data indicators. 2. Trace capture mode: allows visualization of kernel and system events on the timeline, analysis of low-level system events on the CPU and GPU, viewing CPU scheduling and GPU stage data, and understanding where the application spends time. 3. Snapshot Capture mode: allows you to capture and debug rendered frames from any OpenGL ES application, step through and replay rendered frame draw calls, view and edit shaders, and preview the results on the device.

[0167] In some embodiments, when the Snapdragon Profiler performance tool is used to capture rendering information, the captured results can be stored in the form of a memory snapshot and used in subsequent steps. A memory snapshot is a tool for analyzing the memory usage of an application. By recording a memory snapshot of an application at runtime, you can quickly view the memory usage of the application at a certain moment and the memory usage details.

[0168] like Figure 6 As shown, in some embodiments, the above step S230, restoring the state information of the rendering pipeline according to the rendering information, and obtaining the second shader set restored in the rendering pipeline, includes the following steps:

[0169] S2301, acquiring the pipeline call state in the rendering process according to the performance indicator data in the rendering information;

[0170] S2302: According to the pipeline call state, restore and obtain a second shader set corresponding to each switching of the rendering pipeline.

[0171] Specifically, an embodiment of the present invention can obtain the pipeline ID set each time the rendering pipeline call is switched based on the performance indicator data in the rendering information, and can obtain the pipeline call status during the rendering process based on the pipeline ID, for example, determining the calling order based on the appearance order or configuration order of the pipeline ID.

[0172] Furthermore, the embodiment of the present invention can reversely infer the calling state of the shader corresponding to each pipeline ID based on the calling state of the pipeline. For example, when the rendering pipelines called by the current rendering screen are a, b, and c respectively, if the shaders corresponding to pipeline a in the configuration table are A1 and A2; the shaders corresponding to pipeline b in the configuration table are B1 and B2; the shaders corresponding to pipeline c in the configuration table are C1 and C2; then the corresponding shader calling order can be determined to be A1, A2, B1, B2, C1, C2 based on the pipeline calling order of a, b, and c. Based on this, the embodiment of the present invention can construct a second shader set.

[0173] In addition, the graphics rendering pipeline can be divided into two main parts: the first part converts 3D coordinates into 2D coordinates, and the second part converts 2D coordinates into actual colored pixels. The stage flow of the graphics rendering pipeline can include the following processes:

[0174] ①The first part of the graphics rendering pipeline is the vertex shader, which takes a single vertex as input. The main purpose of the vertex shader is to convert 3D coordinates into another type of 3D coordinates.

[0175] ②The Primitive Assembly stage takes all vertices output by the vertex shader as input and assembles all vertices into the shape of the specified primitive.

[0176] ③The output of the primitive assembly stage is passed to the geometry shader. The geometry shader takes a set of vertices in the form of primitives as input, and it constructs new primitives from the newly generated vertices to generate other shapes (but the deformation does not change the shape).

[0177] ④The output of the geometry shader will be passed to the rasterization stage, where it will map the primitives to the corresponding pixels on the final screen and generate fragments for use by the fragment shader.

[0178] ⑤The main purpose of the fragment shader (also called pixel shader) is to calculate the final color of a pixel.

[0179] In some embodiments, the above step S2301, obtaining the pipeline call state in the rendering process according to the performance indicator data in the rendering information, includes the following steps:

[0180] S23011. Obtain rendering information captured by the performance tool;

[0181] S23012, decompressing the rendering information to obtain performance indicator data and a rendering database in the rendering information; wherein the rendering database is used to store the corresponding relationship between the rendering pipeline and the shader;

[0182] S23013. Determine, according to the performance indicator data, first identification information of each rendering pipeline when switching the rendering pipeline during the rendering process.

[0183] Specifically, when the embodiment of the present invention uses the above-mentioned Snapdragon Profiler performance tool to intercept rendering information, the performance snapshot file can be first obtained, and then the performance snapshot file can be decompressed as a zip compressed file, so as to obtain the pipeline ID set each time the rendering pipeline call is switched from the performance data. In addition, the embodiment of the present invention queries the vertex shader ID and fragment shader ID used by different rendering pipeline IDs from the decompressed rendering database. Then, according to the corresponding relationship between the rendering pipeline ID and the shader ID, the shader used by the current pipeline after each setting of the pipeline state can be obtained.

[0184] In some embodiments, the above step S2302, restoring the second shader set corresponding to each switching of the rendering pipeline according to the pipeline call state, includes the following steps:

[0185] S23021. Determine, from the rendering database, second identification information of a shader used by the rendering pipeline according to the first identification information of the rendering pipeline;

[0186] S23022. Determine a shader used by the current rendering pipeline each time the rendering pipeline is switched according to a correspondence between the first identification information and the second identification information, and construct the second shader set.

[0187] Specifically, the rendering database of the embodiment of the present invention can record the shader ID called by the rendering pipeline of each ID. That is to say, according to the identification information (ID) of the rendering pipeline, the identification information (ID) of the corresponding shader can be found through the method of the embodiment of the present invention, and all the second shaders can be aggregated to construct a second shader set.

[0188] In some embodiments, the above step S230, restoring the state information of the rendering pipeline according to the rendering information, and obtaining the second shader set restored in the rendering pipeline, further includes the following steps:

[0189] S2303, determining a calling order of a rendering pipeline in a rendering process according to the performance indicator data and the first identification information of the rendering pipeline;

[0190] S2304, recording the second identification information of the shader used by the rendering instruction involved after configuring the calling state of the rendering pipeline each time;

[0191] S2305: Use the second identification information of the shader used by the current rendering instruction as the identification information of the shader matched by the rendering pipeline, and store the pairing relationship between the rendering pipeline and the shader in the rendering database.

[0192] Specifically, the embodiment of the present invention can record the shader ID used by all Draw Calls after each pipeline state setting call as the rendering pipeline shader ID in the order of rendering interface calls until the rendering pipeline state is set next time.

[0193] like Figure 7 As shown, in some embodiments, the above step S240, determining a matching shader combination from the first shader set and the second shader set, includes the following steps:

[0194] S2401, extracting drawing instructions from rendering information obtained by a rendering analysis tool;

[0195] The drawing instruction in the embodiment of the present invention may be a Draw Call instruction.

[0196] Specifically, a Draw Call is an instruction that instructs the GPU to draw an object on the screen. In computer graphics, a DrawCall is a command executed by the GPU to draw an object on the screen. This command tells the GPU how to render one or more objects, including information such as the object's geometry, material, texture, etc. Reducing the number of Draw Calls is an important means of optimizing graphics rendering performance because it can reduce the communication overhead between the CPU and GPU and improve rendering efficiency.

[0197] S2402, extracting a first shader file of each shader in a first shader set used by the drawing instruction, and extracting a second shader file of each shader in the second shader set;

[0198] The embodiment of the present invention can export the shader code file used for each Draw Call (drawing instruction) in the RenderDoc data.

[0199] S2403, respectively calculating file identification information of the first shader file and the second shader file;

[0200] The embodiment of the present invention can query the shader code file corresponding to each shader ID from the Snapdragon Profiler database.

[0201] Based on the above steps S2402 and S2403, the shader code file stored in the relevant configuration file (ie, the first shader file) and the shader code file that may be used in the rendering process obtained by reverse deduction (ie, the second shader file) can be obtained respectively.

[0202] S2404: When the file identification information of the first shader file is the same as the file identification information of the second shader file, a corresponding shader matching relationship is established to obtain the shader combination.

[0203] The embodiment of the present invention compares the identifiers of the first shader file and the second shader file. Exemplarily, the shader file may record the shader identifier and the shader calling sequence corresponding to each picture frame. By comparing the two shader files, it can be determined whether the same shader and the same calling sequence are used for the same picture frame. If so, a matching relationship between the first shader and the second shader can be established to obtain a shader combination.

[0204] In some embodiments, the S2403, respectively calculating the file identification information of the first shader file and the second shader file, comprises the following steps:

[0205] S24031. Use a hash algorithm to respectively calculate a first hash value of the first shader file and a second hash value of the second shader file;

[0206] S24032. Use the first Hash value and the second Hash value as the file identification information;

[0207] Wherein, the hash values ​​of two shaders that match each other in the shader combination are equal.

[0208] Specifically, the embodiment of the present invention uses the same hash algorithm to calculate the hash values ​​of the shader files obtained by RenderDoc and SnapdragonProfiler respectively. When the hash values ​​of the vertex shader and the fragment shader used by two Draw Calls in the RenderDoc data and the Snapdragon Profiler data are equal, a correspondence is established between the two. Since the same shader combination may be used in different Draw Calls, the data correspondence is many-to-many.

[0209] Among them, the hash algorithm is a process of converting input data of any length into a hash value of fixed length through a specific algorithm. For example, MD5, SHA-1 and SHA-256 can be used, which will not be described in detail here.

[0210] like Figure 8 As shown, in some embodiments, the above step S250, according to the rendering data of the shader of the first shader set, summarizes and sorts the rendering heat information of multiple shader combinations in the corresponding picture frame to obtain the hotspot data aggregation result, includes the following steps:

[0211] S2501, according to the rendering data of the shader of the first shader set, obtaining the to-be-sorted parameters corresponding to each of the shader combinations; wherein the to-be-sorted parameters include at least one of the total drawing time of each frame, bandwidth usage, number of floating-point operation instructions, and number of sampling instructions;

[0212] S2502. According to the parameters to be sorted, the parameters are summarized and sorted in ascending or descending order to obtain the order of each shader combination, and then determine the hotspot data aggregation result.

[0213] Specifically, after determining the shader combination, the embodiment of the present invention can aggregate and sort the data of each shader combination using the rendering information and performance information obtained by the rendering analysis tool in the above step S220.

[0214] For example, through the above step S220, the total drawing time, bandwidth usage, number of floating-point operation instructions, and number of sampling instructions occupied by each shader in the drawing process of the current frame can be obtained. Therefore, these performance data can be sorted, and the shader corresponding to the performance hot spot can be determined according to the sorting result.

[0215] The process of data aggregation can be, for example:

[0216] 1. Refer to Table 1. In the aggregated Snapdragon Profiler data, sort the performance indicators to be analyzed in descending order. You can find a pair of shaders with a high overhead (such as the Fragment ALU Instructions (Half) indicator, which is the total number of fragment shader instructions in Table 1) and write down the two hash values.

[0217] In this example, the first ranked shader combination sequence [A1, B1] is the two hash values ​​corresponding to this step.

[0218] Table 1 Data table obtained by performance tools

[0219]

[0220] 2. Refer to Table 2 to find the corresponding hash value pair in the aggregated RenderDoc data, and sort out the Draw Call numbers in RenderDoc that use this pair of shaders in the RenderDoc data table.

[0221] As shown in Table 2, for the two hash values ​​[A1, B1] above, the corresponding Draw Call numbers can be found as [1052, 1063, 1085].

[0222] Table 2 Data table obtained by rendering analysis tool

[0223]

[0224] 3. If Fig.10 As shown, Fig.10 FIG. 1 is a schematic diagram of the operation interface of the RenderDoc tool according to an embodiment of the present invention. In the operation interface, the function button box area, the model parameter selection box area, and the drawing content viewing area may be included. Fig.10 Enter the corresponding search content in the function button box and find the Draw Call number), then use the highlight tool provided by RenderDoc to confirm the drawing content (such as Fig.10 1001, 1002, 1003 in ), you can also view the specific model information used by this Draw Call in RenderDoc (for example, you can Fig.10 to view the relevant model parameters in the Model Parameters selection box in the Model Parameters selection box).

[0225] Through the above steps 1-3, you can combine Snapdragon Profiler data and RenderDoc data to find the corresponding performance consumption hotspots.

[0226] like Fig. 9 As shown, in some embodiments, the above step S260, determining the performance hotspot object in the actual picture rendering process according to the hotspot data aggregation result, includes the following steps:

[0227] S2601, determining the performance consumption contribution of each pair of shader combinations to the actual picture rendering process according to the hotspot data aggregation result;

[0228] S2602, determining a shader combination whose performance consumption contribution is higher than a preset threshold as a performance hotspot object in an actual picture rendering process;

[0229] Specifically, in an embodiment of the present invention, after obtaining the hotspot data aggregation results, the calculation method of the performance consumption contribution can be found according to the focus of the current application scenario. For example, in the scenario of a smartphone running a game APP, for the problem of game lag, it is necessary to pay attention to the time-consuming data; and for the problem of mobile phone heating, it is necessary to pay attention to the memory bandwidth and floating-point operations. Therefore, for different application scenarios and focus requirements, the embodiment of the present invention can find the corresponding performance data from the hotspot data aggregation results in a targeted manner, and then determine the performance consumption contribution.

[0230] After determining the performance consumption contribution, the shader combination with a performance consumption contribution higher than a preset threshold can be identified as a performance hotspot object in the actual picture rendering process, and the next optimization processing object can be identified.

[0231] It should be noted that, for different application scenarios, step S2602 of the embodiment of the present invention can set different performance consumption contribution thresholds as a basis for judgment. For example, in some scenarios where the performance consumption changes rapidly, such as game running scenarios, the threshold can be set higher to accurately find the shader combination with high performance consumption contribution; and in some scenarios where the performance consumption changes less, such as browsing novels, the threshold can be set lower to find the shader combination with high performance consumption contribution within a smaller performance consumption range. The embodiment of the present invention does not limit the preset threshold of the performance consumption contribution.

[0232] In some embodiments, the method further comprises:

[0233] S270, performing optimization processing on the performance hotspot object; wherein the optimization processing includes at least one of stopping the shader execution action, restarting the shader execution action, and reconfiguring the shader to be executed in the rendering pipeline.

[0234] It should be noted that in addition to the optimization processing scheme described in step S270 above, other targeted optimization processing can be performed according to the performance hotspot object determined in step S260. For example, when the performance hotspot object is a Draw Call instruction that is not efficient enough, the following optimization processing scheme can be adopted: 1. GPU Instancing: When the same mesh and material are copied in large quantities, the use of GPU instancing technology can reduce the number of Draw Calls. For example, this technology can be used for trees and bushes that frequently appear in the scene. 2. Draw Call Batching: Reduce the number of Draw Calls by merging meshes. Unity provides two methods, Static Batching and Dynamic Batching. The former is used to merge different meshes with fewer vertices, and the latter is used to merge different static meshes. 3. Manually merge meshes: By manually merging meshes, multiple objects share the same Draw Call. 4. SRP Batching: In the same scene, multiple methods can be used to optimize Draw Calls, but Unity will sort these optimization methods and give priority to the method with the highest priority.

[0235] The embodiments of the present invention illustrate possible implementation methods for performing optimization processing on the performance hotspot objects. In specific application scenarios, technicians can also perform other optimization processing on the determined performance hotspot objects based on the ideas and methods of the present invention. All of them are within the scope of protection of this application and will not be described in detail here.

[0236] In summary, the embodiment of the present invention first obtains the power consumption information of the intelligent terminal, determines the target screen whose power consumption is greater than the preset threshold based on the power consumption information; then extracts the rendering information of at least one screen frame from the target screen; wherein the rendering information includes the first shader set used in the actual screen rendering process and the rendering data of the shader; and restores the state information of the rendering pipeline according to the rendering information, and obtains the second shader set restored in the rendering pipeline; then determines the mutually matching shader combination from the first shader set and the second shader set; and according to the rendering data of the shader of the first shader set, summarizes and sorts the rendering heat information of the plurality of shader combinations in the corresponding screen frame to obtain the hotspot data aggregation result; finally, determines the performance hotspot object in the actual screen rendering process according to the hotspot data aggregation result. The present invention can accurately find the heat hotspot with the largest performance consumption by analyzing the power consumption information and restoring the shader state of the pipeline rendering process for the intelligent terminal when performing high rendering processing. Compared with the module comparison test method of the prior art, the thermal granularity of this scheme is small and the accuracy is high, and the hotspot of the rendering performance data can be accurately located to improve the user experience.

[0237] The following example uses a smartphone to run a game app. Figure 3 , the specific implementation process of the method of the present invention is described in detail:

[0238] 1. Obtain power consumption data through the adb power query command provided by an Android phone (or an IOS phone), obtain power consumption data in real time during the game, and find scenes with higher power consumption.

[0239] 2. Collect performance data. Use the RenderDoc tool (rendering analysis tool) to capture the rendering process of a frame and obtain the GPU time of each Draw Call. Then use the performance tool Snapdragon Profiler to capture the memory bandwidth, number of sampled instructions, number of floating-point operation instructions, and other performance indicators of the scene as well as all rendering instructions.

[0240] 3. Restore the rendering pipeline state. The steps are as follows:

[0241] a) Extract the Snapdragon Profiler snapshot file as a zip archive.

[0242] b) Obtain the pipeline ID set each time the rendering pipeline is switched from the performance data exported from the Snapdragon Profiler, and query the vertex shader ID and fragment shader ID used by different rendering pipeline IDs from the decompressed database. Based on the correspondence between the rendering pipeline ID and the shader ID, obtain the shader used by the current pipeline after each pipeline state is set.

[0243] c) In the order of rendering interface calls, record the shader ID used by all Draw Calls after each pipeline state setting call as the rendering pipeline shader ID until the next rendering pipeline state is set.

[0244] 4. Establish data correspondence. The steps are as follows:

[0245] a) Export the shader code file used for each Draw Call in the RenderDoc data, and query the shader code file corresponding to each shader ID from the Snapdragon Profiler database.

[0246] b) Use the same hash algorithm to calculate the hash value of the shader file obtained by RenderDoc and Snapdragon Profiler respectively.

[0247] c) When the hash values ​​of the vertex shader and fragment shader used by two Draw Calls in the RenderDoc data and the Snapdragon Profiler data are equal, a correspondence is established between the two. Since the same shader combination may be used in different Draw Calls, the data correspondence is many-to-many.

[0248] 5. Data aggregation. Aggregate data from the perspective of shaders. According to the different shader hash values, summarize and sort the total drawing time, bandwidth usage, number of floating-point operation instructions, and number of sampling instructions occupied by each shader combination in one frame of drawing.

[0249] 6. Analyze hot spots. For different shaders, the contribution of each shader to performance consumption can be analyzed through aggregated data. For different Draw Calls, the performance overhead and rendering content can be associated through the corresponding relationship to determine the drawing content of the performance hotspot, thereby determining the optimization direction.

[0250] The calculation method of the contribution to performance consumption in the embodiment of the present invention is not fixed, and is determined according to the different performance indicators of the problem to be analyzed. For example, the analysis of game freeze problems requires attention to time consumption data, and the analysis of heat problems requires attention to memory bandwidth and floating-point operations. After sorting these data indicators, look at the big part.

[0251] The embodiment of the present invention further provides a hot spot analysis device based on rendering performance data, comprising:

[0252] The first module is used to obtain power consumption information of the smart terminal, and determine a target screen whose power consumption is greater than a preset threshold based on the power consumption information;

[0253] A second module is used to extract rendering information of at least one picture frame from the target picture; wherein the rendering information includes a first shader set used in an actual picture rendering process and rendering data of the shader;

[0254] A third module is used to restore the state information of the rendering pipeline according to the rendering information, and obtain a second shader set restored in the rendering pipeline;

[0255] A fourth module, configured to determine a matching shader combination from the first shader set and the second shader set;

[0256] A fifth module is used to summarize and sort the rendering heat generation information of multiple shader combinations in corresponding picture frames according to the rendering data of the shaders of the first shader set, so as to obtain a hotspot data aggregation result;

[0257] The sixth module is used to determine the performance hotspot object in the actual picture rendering process according to the hotspot data aggregation result.

[0258] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0259] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0260] The memory is used to store programs;

[0261] The processor executes the program to implement the method described above.

[0262] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0263] See also Fig.11 , Fig.11 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0264] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0265] The memory 1102 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 may store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the hotspot analysis method based on rendering performance data in the embodiments of the present invention.

[0266] Input / output interface 1103, used to implement information input and output;

[0267] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0268] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0269] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0270] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0271] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0272] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0273] Another aspect of an embodiment of the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0274] It should be noted that in various specific embodiments of the present invention, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present invention needs to obtain the user's sensitive personal information, it will obtain the user's separate permission or consent through a pop-up window or jump to a confirmation page, and after clearly obtaining the user's separate permission or consent, it will obtain the necessary user-related data for the normal operation of the embodiment of the present invention.

[0275] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0276] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0277] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0278] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0279] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0280] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0281] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0282] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0283] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0284] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store programs.

[0285] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the embodiments of the present invention is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of the rights of the embodiments of the present invention.

Claims

1. A hot spot analysis method based on rendering performance data, characterized in that: The following steps are involved: Acquire power consumption information of the smart terminal, and determine a target screen whose power consumption is greater than a preset threshold based on the power consumption information; Extracting rendering information of at least one picture frame from the target picture; wherein the rendering information includes a first shader set used in an actual picture rendering process and rendering data of the shader; Restoring state information of a rendering pipeline according to the rendering information, and obtaining a second shader set restored in the rendering pipeline; Determine a shader combination that matches each other from the first shader set and the second shader set; According to the rendering data of the shaders of the first shader set, the rendering heat generation information of the plurality of shader combinations in the corresponding picture frame is summarized and sorted to obtain a hotspot data aggregation result; The performance hotspot object in the actual picture rendering process is determined according to the hotspot data aggregation result.

2. The hot spot analysis method based on rendering performance data according to claim 1, characterized in that: The acquiring of the power consumption information of the intelligent terminal and determining, based on the power consumption information, a target screen whose power consumption is greater than a preset threshold value comprises the following steps: According to the power consumption query instruction, the power consumption of the smart terminal in the target application scenario is obtained to obtain the power consumption information of the smart terminal; According to the power consumption information, the power consumption data of a plurality of candidate pictures are sorted, and the target picture with high power consumption is determined.

3. The hot spot analysis method based on rendering performance data according to claim 1, characterized in that: The step of extracting rendering information of at least one picture frame from the target picture comprises the following steps: By using a rendering analysis tool, the rendering process of multiple picture frames is intercepted from the target picture to obtain the issuing time information of the drawing instruction corresponding to each picture frame; According to the command time information, the rendering information of the corresponding time node is intercepted through a performance tool; wherein the rendering information includes performance indicator data of the smart terminal at the time node and rendering instructions; the rendering instructions include a first shader set used in the actual picture rendering process.

4. The hot spot analysis method based on rendering performance data according to claim 1, characterized in that: The step of restoring the state information of the rendering pipeline according to the rendering information and obtaining the second shader set restored in the rendering pipeline comprises the following steps: Obtaining the pipeline call state during the rendering process according to the performance indicator data in the rendering information; According to the pipeline call state, a second shader set corresponding to each switching of the rendering pipeline is restored.

5. The hot spot analysis method based on rendering performance data according to claim 4, characterized in that: The step of obtaining the pipeline call state in the rendering process according to the performance indicator data in the rendering information comprises the following steps: Get rendering information captured by performance tools; Decompressing the rendering information to obtain performance indicator data and a rendering database in the rendering information; wherein the rendering database is used to store the corresponding relationship between the rendering pipeline and the shader; According to the performance indicator data, first identification information of each rendering pipeline when switching the rendering pipeline during the rendering process is determined.

6. The hot spot analysis method based on rendering performance data according to claim 5, characterized in that: The method of restoring the second shader set corresponding to each switching of the rendering pipeline according to the pipeline call state comprises the following steps: Determining, from the rendering database, second identification information of a shader used by the rendering pipeline according to the first identification information of the rendering pipeline; According to the correspondence between the first identification information and the second identification information, a shader used by the current rendering pipeline is determined each time the rendering pipeline is switched, and the second shader set is constructed.

7. The hot spot analysis method based on rendering performance data according to claim 1, characterized in that: The step of determining a matching shader combination from the first shader set and the second shader set comprises the following steps: Extracting drawing instructions from rendering information obtained by a rendering analysis tool; Extracting a first shader file of each shader in a first shader set used by the drawing instruction, and extracting a second shader file of each shader in the second shader set; Calculating file identification information of the first shader file and the second shader file respectively; When the file identification information of the first shader file is the same as the file identification information of the second shader file, a corresponding shader matching relationship is established to obtain the shader combination.

8. The hot spot analysis method based on rendering performance data according to claim 1, characterized in that: The step of aggregating and sorting the rendering heat generation information of the plurality of shader combinations in the corresponding picture frame according to the rendering data of the shader of the first shader set to obtain the hotspot data aggregation result includes the following steps: According to the rendering data of the shaders of the first shader set, the parameters to be sorted corresponding to each of the shader combinations are obtained; wherein the parameters to be sorted include at least one of the total drawing time of each frame, bandwidth occupancy, number of floating-point operation instructions, and number of sampling instructions; According to the parameters to be sorted, the parameters are summarized and sorted in ascending or descending order of numerical values ​​to obtain the order of each shader combination, and then the hot spot data aggregation result is determined.

9. The hot spot analysis method based on rendering performance data according to claim 1, characterized in that: Determining the performance hotspot object in the actual picture rendering process according to the hotspot data aggregation result includes the following steps: Determine the performance consumption contribution of each pair of shader combinations to the actual picture rendering process according to the hotspot data aggregation result; A shader combination whose performance consumption contribution is higher than a preset threshold is determined as a performance hotspot object in the actual picture rendering process.

10. A hot spot analysis device based on rendering performance data, characterized in that: include: The first module is used to obtain power consumption information of the smart terminal, and determine a target screen whose power consumption is greater than a preset threshold based on the power consumption information; A second module is used to extract rendering information of at least one picture frame from the target picture; wherein the rendering information includes a first shader set used in an actual picture rendering process and rendering data of the shader; A third module is used to restore the state information of the rendering pipeline according to the rendering information, and obtain a second shader set restored in the rendering pipeline; A fourth module, configured to determine a matching shader combination from the first shader set and the second shader set; A fifth module is used to summarize and sort the rendering heat generation information of multiple shader combinations in corresponding picture frames according to the rendering data of the shaders of the first shader set to obtain a hotspot data aggregation result; The sixth module is used to determine the performance hotspot object in the actual picture rendering process according to the hotspot data aggregation result.

11. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images