Target platform system performance test method and device and storage medium

Abstract

The invention discloses a target platform system performance test method and device and a storage medium, and belongs to the field of intelligent computing. According to the test method, code line granularity monitoring of system performance key indexes is achieved. According to the testing method, data visualization is achieved based on key operation steps of the intelligent model, a clear view of the system resource use condition is provided, the system resource occupation condition and changes of the system resource occupation condition can be visually displayed through the testing method, specific code lines causing sudden increase or sudden decrease of the resources are marked, and a user is helped to rapidly position performance bottlenecks and optimization points. According to the method, important reference is provided for intelligent model optimization on a domestic platform, and the technical progress and application development of the domestic intelligent computing platform can be promoted.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent computing, and in particular to a target platform system performance testing method, device and storage medium. Background Art

[0002] With the rapid development of intelligent computing technology, especially in the intelligent model training and reasoning stages, performance monitoring has become increasingly important. Most of the existing performance testing methods are targeted at international mainstream platforms such as X86 architecture and Nvidia accelerator cards, which leads to a very scarce performance testing method available on domestic platforms.

[0003] Existing testing methods often have deficiencies in visualization, especially in the visualization analysis of resource utilization in intelligent model systems.

[0004] In order to fill this gap, the present application provides a target platform system performance testing method, which can realize code line granularity monitoring, user-defined monitoring and system performance testing method based on intelligent model key step visualization. Summary of the invention

[0005] The embodiment of the present application provides a target platform system performance test method, device and storage medium, which are used to solve the problem of scarcity of performance test methods on domestic platforms. The technical solution is as follows: On the one hand, a target platform system performance testing method is provided, the method comprising: Run the target platform through settrace function tracking program; The behavior when tracing to a code line is defined by defining the trace function, where the trace function tracks each line of code through event=line; The current system resource occupancy is obtained by calling the interface in the tracing function trace, wherein the tracing function trace is a sub-function in the settrace function tracing program; Realize monitoring of different granularities by locating the current code line position; After completing the granular monitoring of the current code line, the monitoring results are visualized based on the running steps of the intelligent model.

[0006] Optionally, the current system resource occupancy is obtained by calling an interface in a tracing function trace, wherein the tracing function trace is a sub-function in the settrace function tracing program, including: Get the code running time by using the time() function; Use the memory_reserved() function to get the size of the video memory currently allocated to Pytorch, or use the xpu-smi command to get the video memory usage of the current machine; Run the memory_full_info() function to get the current system resource usage.

[0007] Optionally, the monitoring of different granularities is achieved by locating the current code line position, including: Determine the current code line position through the frame obtained during the monitoring process, wherein the frame refers to the frame object obtained during the monitoring process, and the frame object is used to provide multiple attributes, and the multiple attributes include information of the current frame; Each line of the main program code is monitored, and the called library code is monitored at a preset line frequency.

[0008] Optionally, after completing the granular monitoring of the current code line, the monitoring result is visualized based on the intelligent model running steps, including: After completing the granularity monitoring of the current code line, the key operation steps of the intelligent model are obtained by performing data cleaning on the granularity monitoring results; Use the fill and text functions to visualize the key running steps and mark the key running codes of the intelligent model under the control of the flame graph.

[0009] Optionally, after completing the granularity monitoring of the current code line, obtaining the key operation steps of the intelligent model by performing data cleaning on the granularity monitoring result includes: Calculating the gradient of the particle size monitoring result; The granularity monitoring results are screened by setting a threshold and the corresponding program code is obtained by positioning, wherein the corresponding program code is used to determine the key operation steps of the intelligent model; Visual data corresponding to the granularity monitoring results are obtained by deduplication and sorting.

[0010] Optionally, the key running steps are visualized using fill and text functions under the flame graph and the key running codes of the intelligent model are marked, including: In contrast to the flame graph, use the fill function of matplotlib to fill the rectangle. The filled rectangle is used to show the usage of system resources. The text function is used to annotate the part where the system resources increase or decrease suddenly, and the code lines of the main program are marked. The code lines of the main program are used to display the code lines that cause the system resources to be occupied.

[0011] On the other hand, a target platform system performance testing device is provided, the device comprising: Platform tracing module, used to trace the target platform where the program is running through the settrace function; The behavior definition module is used to define the behavior when tracing the code line by defining the trace function, where the trace function tracks each line of code through event=line; A resource occupation module, used to obtain the current system resource occupation status through an interface call in a trace function trace, wherein the trace function trace is a sub-function in the settrace function trace program; Granularity monitoring module, used to achieve monitoring of different granularities by locating the current code line position; The step visualization module is used to visualize the operation steps of the intelligent model based on the monitoring results after completing the granular monitoring of the current code line.

[0012] On the other hand, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the target platform system performance testing method as described above.

[0013] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the target platform system performance testing method as described in the above aspects.

[0014] On the other hand, a computer program product is also provided, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the target platform system performance testing method described in the above aspect.

[0015] The target platform system performance testing method suitable for domestic platforms proposed in this application has significant technical effects and advantages.

[0016] Accurate performance monitoring: The testing method enables code line granularity monitoring of key system performance indicators such as memory, video memory, and running time. This high-precision monitoring method allows developers to accurately understand the specific impact of each line of code on system resources, thereby performing more sophisticated performance analysis and optimization during the training and reasoning of intelligent models.

[0017] High degree of customization: Users can customize monitoring parameters according to actual needs, including monitoring granularity, intelligent platform, intelligent model, and data set size. This flexibility enables the test method to adapt to different types of intelligent computing tasks and platforms, providing tailored performance analysis solutions for various application scenarios.

[0018] Visualization of key steps of the intelligent model: The test method visualizes data based on the key running steps of the intelligent model, providing a clear view of system resource usage. By simulating the flame graph mode, the test method can intuitively display the occupation of system resources and their changes, mark the specific lines of code that cause a sudden increase or decrease in resources, and help users quickly locate performance bottlenecks and optimization points.

[0019] Filling the market gap: There is a lack of system performance testing methods for domestic intelligent computing platforms in the current market. The testing method of the present invention fills this gap, provides an important reference for the optimization of intelligent models on domestic platforms, and helps promote the technological progress and application development of domestic intelligent computing platforms.

[0020] Improve development efficiency: By providing detailed performance monitoring and visual analysis, the test method helps developers identify and solve performance problems more efficiently. This efficient performance analysis method can significantly improve the training and reasoning efficiency of intelligent models, thereby shortening the development cycle and reducing costs.

[0021] In summary, the system performance testing method of the present invention not only provides significant improvements in technical functions, but also effectively enhances the performance monitoring and optimization capabilities of the intelligent computing platform in practical applications, and plays an important role in promoting the development of intelligent computing on domestic platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flow chart of a target platform system performance testing method provided by an embodiment of the present application is shown; Figure 2 A flow chart of a monitoring process provided by an embodiment of the present application is shown; Figure 3 A visual flow chart provided by an embodiment of the present application is shown; Figure 4 A schematic diagram illustrating the video memory occupancy and internal memory occupancy and their changes on an mlu platform is shown in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0024] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0025] Example 1

[0026] Figure 1 A flow chart of a target platform system performance testing method provided by an embodiment of the present application is shown, and the method includes: Step 101, trace the program running on the target platform through the settrace function.

[0027] Among them, the target platform system refers to a combination of hardware and software that supports high-performance computing and artificial intelligence tasks. It can efficiently run the training and reasoning tasks of intelligent models, has powerful computing power, memory management capabilities, and an architectural design that adapts to specific intelligent computing needs. In this application, the target platform mainly refers to domestic intelligent computing platforms, such as domestic chips and accelerator cards that support deep learning frameworks, that is, in each embodiment of this application, the target platform system is used as a "system with intelligent computing capabilities", and the characteristics of the current domestic platform are combined for example.

[0028] In this scenario, intelligent computing tasks usually include training, reasoning, data processing and optimization of deep learning models, which require the target platform to have efficient computing power, multi-threaded processing capabilities and support for complex resource scheduling. The target platform system targeted by this application can provide optimized design in terms of memory and video memory allocation, parallel computing, network bandwidth, etc. to adapt to the needs of artificial intelligence technology.

[0029] Compared with traditional general-purpose computing platforms, the target platform system (intelligent computing system) that this application can handle has higher computing power and resource optimization capabilities when processing deep learning and other AI tasks. It supports efficient execution of key steps of intelligent models, while flexibly managing resources such as video memory and memory, and providing strong adaptability to complex models and large-scale data sets.

[0030] The target platform system that this application can handle, as an intelligent computing system, supports high-precision monitoring and optimization of AI tasks. Intelligent computing systems include but are not limited to hardware and software architectures that support deep learning frameworks (such as Pytorch and TensorFlow) on domestic platforms. In addition, as an intelligent computing system, the target platform can support the operation of key steps based on deep learning models, and realize the monitoring of code line granularity and visual analysis of resource usage through the performance testing method in this application.

[0031] Step 102, defining the behavior when tracing a code line by defining a trace function.

[0032] Among them, the trace function uses event=line to track each line of code.

[0033] Step 103, obtaining the current system resource occupancy status through an interface call in a tracing function trace, wherein the tracing function trace is a sub-function in the settrace function tracing program.

[0034] Specifically, we can use the time() function to get the code running time; use the memory_reserved() function to get the size of the video memory currently allocated to Pytorch, or use the xpu-smi command to get the video memory usage of the current machine; and run the memory_full_info() function to get the current system resource usage. Among them, the related interfaces called are such as memory_reserved, etc.

[0035] Step 104, monitoring at different granularities is achieved by locating the current code line position.

[0036] In this step, the current code line position is determined by the frame obtained during the monitoring process, each line of the main program code is monitored, and the called library code is monitored at a preset line frequency to achieve monitoring of different granularities of the intelligent model code. Each line of the main program code is monitored, such as monitoring every 5 lines of the called library code to save program running time. The frame refers to the frame object obtained during the monitoring process, which represents a frame of the current execution stack. The frame object is used to provide multiple attributes, and the multiple attributes include information about the current frame, which is then used to implement analysis of the current code line situation.

[0037] Step 105, after completing the granular monitoring of the current code line, the monitoring results are visualized based on the intelligent model running steps.

[0038] Content 1: After completing the granular monitoring of the current code line, the key operating steps of the intelligent model are obtained by performing data cleaning on the granular monitoring results.

[0039] This is a key step in intelligent modeling.

[0040] like Figure 3 As shown, Figure 3 A visual flowchart provided by an embodiment of the present application is shown, wherein corresponding content one includes a data cleaning stage.

[0041] First, the gradient of the granular monitoring results is calculated, including the gradient of the system video memory, memory usage, etc.

[0042] Granular monitoring results are filtered by setting thresholds and the corresponding program code is obtained by positioning, that is, locating the main program. The corresponding program code is used to determine the key operating steps of the intelligent model; the execution time is calculated, and the visual data corresponding to the granular monitoring results is obtained by deduplication and sorting.

[0043] Content 2: Use the fill and text functions to visualize the key running steps and mark the key lines of code of the intelligent model under the comparison of the flame graph.

[0044] Continue as Figure 3 As shown, the corresponding content two includes a visualization stage.

[0045] First, set the relevant format according to the visualization format requirements; determine the visualization area; use matplotlib's fill function to fill the rectangle under the flame chart control. The filled rectangle is used to display the main code, including the usage of system resources; use the text function to annotate the part where the system resources increase or decrease sharply, and mark the code lines of the main program. The code lines of the main program are used to display the code lines that cause the system resources to be occupied.

[0046] The target platform system performance testing method suitable for domestic platforms proposed in this application has significant technical effects and advantages.

[0047] Accurate performance monitoring: The testing method enables code line granularity monitoring of key system performance indicators (such as memory, video memory, and running time). This high-precision monitoring method allows developers to accurately understand the specific impact of each line of code on system resources, thereby performing more detailed performance analysis and optimization during the training and reasoning of intelligent models. Figure 2 As shown, a monitoring process flow chart provided in an embodiment of the present application is shown, which sets hyperparameters, monitors program operation, records key performance indicators, obtains current program information, and finally saves it. This is the monitoring stage.

[0048] High degree of customization: Users can customize monitoring parameters according to actual needs, including monitoring granularity, intelligent platform, intelligent model, and data set size. This flexibility enables the test method to adapt to different types of intelligent computing tasks and platforms, providing tailored performance analysis solutions for various application scenarios.

[0049] Visualization of key steps of the intelligent model: The test method visualizes data based on the key running steps of the intelligent model, providing a clear view of system resource usage. By simulating the flame graph mode, the test method can intuitively display the occupation of system resources and their changes, mark the specific lines of code that cause a sudden increase or decrease in resources, and help users quickly locate performance bottlenecks and optimization points.

[0050] Filling the market gap: There is a lack of system performance testing methods for domestic intelligent computing platforms in the current market. The testing method of the present invention fills this gap, provides an important reference for the optimization of intelligent models on domestic platforms, and helps promote the technological progress and application development of domestic intelligent computing platforms.

[0051] Improve development efficiency: By providing detailed performance monitoring and visual analysis, the test method helps developers identify and solve performance problems more efficiently. This efficient performance analysis method can significantly improve the training and reasoning efficiency of intelligent models, thereby shortening the development cycle and reducing costs.

[0052] In summary, the system performance testing method of the present invention not only provides significant improvements in technical functions, but also effectively improves the performance monitoring and optimization capabilities of the intelligent computing platform in practical applications, and plays an important role in promoting the development of intelligent computing on domestic platforms.

[0053] Example 2

[0054] Based on the above embodiment, the customized monitoring object also includes customized monitoring of the intelligent platform, intelligent model and data set size.

[0055] Then, before running the target platform through the settrace function tracking program, a process of customizing the selection of monitoring objects is also included, and the method also includes: Use the argparse.ArgumentParser() function to set different hyperparameters for each custom monitoring object; When starting monitoring, enter the hyperparameters corresponding to the custom monitoring object.

[0056] In response to the custom monitoring object being the monitoring granularity, the execution traces the content of the program running on the target platform through the settrace function.

[0057] like Figure 4 As shown, a schematic diagram of the video memory usage and memory usage of the mlu platform and their changes is shown. The memory usage of the mlu platform can be monitored, and the monitoring situation is as follows.

[0058] Figure 4In the figure, the legend in the upper left corner means the video memory usage and memory usage of the mlu platform and their changes. The English in the figure is what the code corresponding to the video memory / memory change is. Specifically, the lines of mlu_memory / Mb and mlu_memory_change correspond to the codes corresponding to the video memory / memory change on the mlu platform, and the lines of os_memory / Mb and os_memory_change correspond to the codes corresponding to the video memory / memory change on the os platform. The vertical axis is the amount of memory usage, and the horizontal axis is the time (in seconds). The model.load_state annotation represents the memory usage during the model loading phase, the model.load_state annotation represents the video memory usage during the model loading phase, outputs=model(inputs) represents the video memory usage during the forward propagation process in the model inference, and print(i) represents the video memory release after the model inference completes 1 batch. The data shown in this figure shows that the main video memory and memory usage of the mlu platform when using the resnet model for inference occurs in the model loading phase, and the video memory release process is not obvious when the data set is small.

[0059] The present application also provides a target platform system performance test device, which is used for a live broadcast platform, and includes: Platform tracing module, used to trace the target platform where the program is running through the settrace function; The behavior definition module is used to define the behavior when tracing the code line by defining the trace function, where the trace function tracks each line of code through event=line; A resource occupation module, used to obtain the current system resource occupation status through an interface call in a trace function trace, wherein the trace function trace is a sub-function in the settrace function trace program; Granularity monitoring module, used to achieve monitoring of different granularities by locating the current code line position; The step visualization module is used to visualize the operation steps of the intelligent model based on the monitoring results after completing the granular monitoring of the current code line.

[0060] Optionally, the resource occupation module includes: The first occupancy unit is used to obtain the code running time by using the time() function; The second occupation unit is used to use the memory_reserved() function to get the size of the video memory currently allocated to Pytorch, or use the xpu-smi command to get the video memory usage of the current machine; The third occupation unit is used to run the memory_full_info() function to obtain the current system resource occupation status.

[0061] Optionally, the particle size monitoring module includes: A first monitoring unit, configured to determine a current code line position through a frame obtained during the monitoring process, wherein the frame refers to a frame object obtained during the monitoring process, and the frame object is configured to provide a plurality of attributes, wherein the plurality of attributes include information of the current frame; The second monitoring unit is used to monitor each line of the main program code and monitor the called library code at a preset line frequency.

[0062] Optionally, the step visualization module includes: A first visualization unit is used to obtain key operation steps of the intelligent model by performing data cleaning on the granularity monitoring results after completing the granularity monitoring of the current code line; The second visualization unit is used to visualize the key running steps and mark the key running codes of the intelligent model by using fill and text functions under the control of the flame graph.

[0063] Optionally, the first visualization unit is also used to calculate the gradient of the granularity monitoring results; filter the granularity monitoring results by setting a threshold and obtain the corresponding program code through positioning, and the corresponding program code is used to determine the key operating steps of the intelligent model; and obtain the visualization data corresponding to the granularity monitoring results by deduplication and sorting.

[0064] Optionally, the second visualization unit is further used to fill a rectangle using the fill function of matplotlib under the control of the flame chart, and the filled rectangle is used to display the occupation of system resources; use the text function to annotate the part where the system resources increase or decrease suddenly, and mark the code lines of the main program, and the code lines of the main program are used to display the code lines that cause the occupation of system resources.

[0065] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0066] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0067] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A target platform system performance testing method, characterized in that: The method comprises: Run the target platform through settrace function tracking program; The behavior when tracing to a code line is defined by defining the trace function, where the trace function tracks each line of code through event=line; The current system resource occupancy is obtained by calling the interface in the tracing function trace, wherein the tracing function trace is a sub-function in the settrace function tracing program; Realize monitoring of different granularities by locating the current code line position; After completing the granular monitoring of the current code line, the monitoring results are visualized based on the intelligent model running steps.

2. The method according to claim 1, characterized in that The current system resource occupancy is obtained by calling an interface in the trace function trace, wherein the trace function trace is a sub-function in the settrace function trace program, including: Get the code running time by using the time() function; Use the memory_reserved() function to get the size of the video memory currently allocated to Pytorch, or use the xpu-smi command to get the video memory usage of the current machine; Run the memory_full_info() function to get the current system resource usage.

3. The method according to claim 1, characterized in that The monitoring of different granularities is achieved by locating the current code line position, including: Determine the current code line position through the frame obtained during the monitoring process, wherein the frame refers to the frame object obtained during the monitoring process, and the frame object is used to provide multiple attributes, and the multiple attributes include information of the current frame; Each line of the main program code is monitored, and the called library code is monitored at a preset line frequency.

4. The method according to claim 1, characterized in that: After completing the granularity monitoring of the current code line, the monitoring result is visualized based on the intelligent model operation steps, including: After completing the granularity monitoring of the current code line, the key operation steps of the intelligent model are obtained by performing data cleaning on the granularity monitoring results; Use the fill and text functions to visualize the key running steps and mark the key running codes of the intelligent model under the control of the flame graph.

5. The method according to claim 4, characterized in that After completing the granularity monitoring of the current code line, the key operation steps of the intelligent model are obtained by performing data cleaning on the granularity monitoring results, including: Calculating the gradient of the particle size monitoring result; The granularity monitoring results are screened by setting a threshold and the corresponding program code is obtained by positioning, wherein the corresponding program code is used to determine the key operation steps of the intelligent model; Visual data corresponding to the granularity monitoring results are obtained by deduplication and sorting.

6. The method according to claim 4, characterized in that The key operation steps are visualized using fill and text functions under the flame graph and the key operation codes of the intelligent model are marked, including: In contrast to the flame graph, use the fill function of matplotlib to fill the rectangle. The filled rectangle is used to show the usage of system resources. The text function is used to annotate the part where the system resources increase or decrease suddenly, and the code lines of the main program are marked. The code lines of the main program are used to display the code lines that cause the system resources to be occupied.

7. A target platform system performance testing device, characterized in that: The device comprises: Platform tracing module, used to trace the target platform where the program is running through the settrace function; The behavior definition module is used to define the behavior when tracing the code line by defining the trace function, where the trace function tracks each line of code through event=line; A resource occupation module, used to obtain the current system resource occupation status through an interface call in a trace function trace, wherein the trace function trace is a sub-function in the settrace function trace program; Granularity monitoring module, used to achieve monitoring of different granularities by locating the current code line position; The step visualization module is used to visualize the operation steps of the intelligent model based on the monitoring results after completing the granular monitoring of the current code line.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the target platform system performance testing method as described in any one of claims 1 to 7 above.

Images