AMD compatibility expansion method based on performance analysis tool PyTorch Profilter

Abstract

The invention discloses an AMD compatibility expansion method based on a performance analysis tool PyTorch Profiler, and belongs to the technical field of artificial intelligence performance analys.The AMD compatibility expansion method comprises the steps that according to performance analysis requirements, function expansion is conducted on a performance analysis library Kinto in the performance analysis tool PyTorch Profiler; the extended function in the performance analysis library Kinto is verified; in response to the performance analysis tool PyTorch Profilter which passes verification, capturing the performance data of the large model program, and carrying out visual analysis verification on the performance data of the large model program; and in response to the passing of the visualization analysis verification, AMD compatibility expansion of the performance analysis tool PyTorch Profiler is completed, so that the performance analysis tool PyTorch Profiler can be compatible with performance data analysis of the AMD GPU, Tensorboard visualization is also supported, the optimization efficiency of an artificial intelligence application program on the AMD GPU is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence performance analysis, and in particular to an AMD compatibility extension method based on a performance analysis tool PyTorch Profiler. Background Art

[0002] Nowadays, large-scale deep learning models are constantly evolving and have been widely used in natural language processing, image recognition, and intelligent speech, which has promoted the rapid development of the artificial intelligence industry. However, in actual application scenarios, these algorithm models usually face problems such as limited computing resources, slow training and inference speed, and insufficient memory, which cannot meet the actual needs of users. In order to solve the above problems, the existing method is to use artificial intelligence performance evaluation tools to analyze their performance to identify the corresponding bottlenecks, so as to optimize them to improve efficiency, enhance user experience and competitive advantage.

[0003] At present, there are two main types of commonly used AI performance evaluation tools. The first is the performance analysis tool provided directly by hardware manufacturers, such as nvprof provided by NVIDA and rocprof provided by AMD. Such tools provide key indicators such as execution time, memory usage, kernel function call statistics, memory transactions, etc. to identify performance bottlenecks in CUDA or HIP applications. The second is the performance analyzer built into the AI ​​framework, such as PyTorch Profiler. Such tools can wrap the entire model training or inference process, automatically collect relevant execution statistics, and write the collected performance data to a format file that can be read by the AI ​​framework visualization tool TensorBoard for subsequent visualization and analysis of performance.

[0004] In the existing AI performance analysis technology, the performance analysis tools provided by hardware manufacturers have unclear functional positioning, and the analysis speed is slow for large files. The performance analyzer built into the AI ​​framework is highly integrated with the framework and has higher analysis efficiency. It can not only analyze the performance of the CPU and GPU, but also provide visualization functions in combination with Tensorboard, making up for the shortcomings of the performance analysis tools provided by manufacturers and becoming a popular tool in the field of AI performance analysis.

[0005] However, the built-in performance analyzer of the artificial intelligence framework mainly supports detailed performance analysis and optimization based on NVIDIA CUDA, and has limited support for AMD graphics cards. In actual application scenarios, when using the performance analysis tool PyTorch Profiler to analyze the performance of large model programs on domestic platforms, x86 and other platforms equipped with AMD graphics cards, the performance data obtained lacks key attributes and information, which limits the subsequent visualization analysis of Tensorboard, and thus prevents users from deeply analyzing performance and making it difficult to accurately locate program performance bottlenecks. Summary of the invention

[0006] The present invention provides an AMD compatibility extension method based on the performance analysis tool PyTorch Profiler, which expands the functions in the performance analysis library Kineto, and uses the PyTorch Profiler with the performance analysis function extension to re-analyze the performance of the large model, and complete the missing performance data, so that PyTorchProfiler can be compatible with AMD GPU performance data analysis and can use Tensorboard visualization, thereby solving the problems existing in the prior art.

[0007] To achieve the above object, the present invention is implemented by adopting the following technical solutions: The present invention provides an AMD compatibility extension method based on a performance analysis tool PyTorch Profiler, comprising: According to the performance analysis requirements, the performance analysis library Kineto in the performance analysis tool PyTorch Profiler is extended with new functions; Verify the extended functions in the performance analysis library Kineto; In response to the verified performance analysis tool PyTorch Profiler capturing the performance data of the large model program, visual analysis and verification of the performance data of the large model program is performed; In response to the visual analysis verification being passed, the AMD compatibility extension of the performance analysis tool PyTorch Profiler was completed.

[0008] Optionally, before expanding the functions of the performance analysis library Kineto in the performance analysis tool PyTorch Profiler according to the performance analysis requirements, the method further includes: providing underlying environment support of the open source computing platform ROCm for the basic hardware environment.

[0009] Optionally, providing open source computing platform ROCm underlying environment support for the basic hardware environment includes: installing a customized ROCm software package under the domestic platform architecture, and / or, under the x86 architecture, consulting the installation documentation officially provided by AMD, configuring the environment according to the document steps and installing the corresponding software package.

[0010] Optionally, the performance analysis requirement is to improve the key attributes and information that are missing in the performance analysis tool PyTorch Profiler for analyzing the performance of large model programs on AMD; According to the performance analysis requirements, the performance analysis library Kineto in the performance analysis tool PyTorch Profiler is extended to include: Plan the specific extension content of the performance analysis library Kineto based on the missing key attributes and information; Add an interface for separately obtaining AMD GPU information in the performance analysis library Kineto, and define the attribute string required to be called by the interface according to the specific extended content; The attribute character string corresponding to the specific extended content is obtained through the interface to complete the function extension.

[0011] Optionally, the verification of the extended functions in the performance analysis library Kineto includes: performing basic function testing, boundary condition testing, and exception handling testing on the extended functions respectively.

[0012] Optionally, the performing basic function test, boundary condition test and exception handling test on the extended function respectively includes: In the basic function verification phase, verify whether the log file generated after the performance analysis function of the performance analysis library Kineto is turned on contains the correct AMD GPU information; In the boundary condition testing phase, check whether the prompts after executing the instance in different environments are logical, wherein the environments include an environment without an AMD GPU, an environment with a driver configuration error, and an environment with a different operating system; During the exception testing phase, check whether the exception message and log output after simulating the exception situation are as expected.

[0013] Optionally, in response to the verified performance analysis tool PyTorch Profiler capturing the large model program performance data, visually analyzing and verifying the large model program performance data includes: Recompile PyTorch Profiler, a performance analysis tool that improves Kineto's extended functions; Use the recompiled performance analysis tool PyTorch Profiler to analyze the performance of large model programs on AMD, capture performance data and store it in a format that can be parsed by the visualization analysis tool Tensorboard.

[0014] Optionally, in response to the visual analysis verification being passed, completing the AMD compatibility extension of the performance analysis tool PyTorch Profiler, including: Use the visualization analysis tool Tensorboard to visualize the performance data collected by the performance analysis tool PyTorch Profiler; Verify in the browser whether the extension function supports Tensorboard visualization and check whether each view is displayed normally. If so, obtain the performance analysis tool PyTorch Profiler that is compatible with AMD GPU performance data analysis and supports Tensorboard visualization. If not, repeat the function expansion steps of the performance analysis library Kineto until the visualization analysis verification passes.

[0015] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: by targeted customization and modification of the Kineto library that the PyTorch Profiler depends on, it is ensured that the Profiler is compatible with and can correctly parse the performance data specific to the AMD GPU, and the missing attributes and information are supplemented, so that Tensorboard can successfully visualize the performance data, thereby solving the problem that the performance analysis of the PyTorch Profiler on the AMD GPU cannot capture key data and cannot use Tensorboard for visualization, which affects the user's ability to locate the performance bottleneck of the program, and improves the optimization efficiency of artificial intelligence applications on the AMD GPU, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure is a flow chart of an AMD compatibility extension method based on the performance analysis tool PyTorch Profiler in one embodiment of the present invention;

[0017] Figure 2 Shown is a schematic diagram of an artificial intelligence performance analysis framework in an embodiment of the present invention;

[0018] Figure 3 Shown is an overview view of using Tensorboard visual analysis on an AMD GPU in one embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0020] Example 1

[0021] like Figure 1 As shown, this embodiment provides an AMD compatibility extension method based on the performance analysis tool PyTorch Profiler, including:

[0022] According to the performance analysis requirements, the performance analysis library Kineto in the performance analysis tool PyTorch Profiler is extended with new functions;

[0023] Verify the extended functions in the performance analysis library Kineto;

[0024] In response to the verified performance analysis tool PyTorch Profiler capturing the performance data of the large model program, visual analysis and verification of the performance data of the large model program is performed;

[0025] In response to the visual analysis verification being passed, the AMD compatibility extension of the performance analysis tool PyTorch Profiler was completed.

[0026] By making targeted functional expansions to the performance analysis library Kineto, making it compatible with subsequent AMD GPU performance data capture and visualization analysis, and then using the performance analysis tool PyTorchProfiler with comprehensive performance analysis functions to capture performance data, you can obtain all the key information for in-depth analysis of large artificial intelligence models, so that the performance analysis tool PyTorch Profiler can perform in-depth performance analysis of large models on AMD.

[0027] Example 2

[0028] Based on Example 1, this embodiment also makes the following design: Figure 2 It is a schematic diagram of the architecture of the artificial intelligence performance analysis framework in the present invention, wherein the AMD compatibility extension method of the performance analysis tool PyTorch Profiler mainly includes the following steps.

[0029] Step 1: Provide ROCm underlying environment support

[0030] Under the domestic platform architecture, directly install the customized ROCm software package; under the x86 architecture, refer to the installation documentation provided by AMD, configure the environment according to the document steps and install the corresponding software package.

[0031] In this embodiment, the hardware environment is established based on a domestic platform or x86 architecture equipped with an AMD GPU device, so it is necessary to provide the underlying environment support of the open source stack developed by AMD, namely ROCm (Radeon Open Compute). For the domestic platform architecture, it is necessary to install a set of ROCm software packages customized for the domestic platform high-performance multi-way server platform to provide HIP and OpenCL general scientific computing parallel programming models and high-performance operator libraries such as rocblas, rccl, rocFFT, etc. for upper-level artificial intelligence applications. For the x86 architecture, it is necessary to configure and install the corresponding ROCm software package according to the documents provided by the AMD official website.

[0032] Step 2: Expand the functionality of the performance analysis library Kineto

[0033] Kineto is a third-party performance analysis library in the deep learning framework PyTorch. Its main goal is to provide a low-overhead and flexible performance analysis solution to collect detailed performance analysis data during the training and inference process of artificial intelligence applications.

[0034] In order to solve the problem that PyTorch Profiler lacks key attributes and information when analyzing the performance of large model programs on AMD, we planned to expand the functions of Kineto module, designed the performance analysis process of Profiler applications deployed on AMD GPUs, added an interface for obtaining AMD GPU information separately, defined the required device attribute strings, and improved the error message prompts; and expanded the functions of Kineto according to the plan to make it compatible with AMD GPU performance data capture and visualization analysis. The specific examples are as follows: First, conduct demand analysis and planning to clarify which key attributes and information are missing when the Profiler captures the performance of programs deployed on AMD GPUs, and plan the specific extension content of the Kineto performance analysis library. For example, in this embodiment, the device information and the device number device in the event data are missing. The lack of these two key information will cause Tensorboard to be unable to recognize kernel events, and thus unable to visualize performance data; secondly, according to the demand analysis and planning, modify the AMD-related code in Kineto. This embodiment adds an API interface for obtaining AMD GPU device information, and the attributes include device id, device name name, total global memory totalGlobalMem, etc. At the same time, the function of obtaining the device number device attribute is added at the GPU kernel event collection. Finally, the effect of matching the AMD GPU device id with the kernel event device number is achieved, and the function is successfully enhanced to make it compatible with AMD GPU performance data capture and visualization analysis.

[0035] Step 3: Perform functional testing on new features of Kineto

[0036] In order to verify the accuracy of the extended functions, corresponding unit tests are designed for verification, including basic function tests, boundary conditions, and exception handling tests. Basic function verification is to execute the test verification program to check whether the performance analysis log contains AMD GPU device information and the device number in the acquisition event. Boundary condition testing is to execute instances in environments that do not contain AMD GPUs, environments with incorrect driver configurations, and environments with different operating systems to check whether the prompts are logical. Exception testing is to simulate abnormal situations such as unauthorized permissions, and check whether the exception messages and log outputs are in line with expectations.

[0037] This embodiment adds a new function of reading GPU device information and collecting device numbers in kernel events in Kineto. In order to ensure the effectiveness of this function, corresponding unit tests need to be designed for verification, including basic function verification, boundary conditions and exception handling.

[0038] The basic function verification test needs to ensure that the newly added function can correctly read the AMD GPU device information and kernel event device number information; the boundary conditions need to verify whether the information can be correctly identified or error prompts can be given in the case of non-AMD GPU, driver problems, different operating systems, etc.; exception handling needs to verify whether reasonable error prompts can be given when encountering exceptions such as permission problems during the reading of graphics card information.

[0039] The specific steps are as follows: (1) In the basic function verification phase, enable Kineto's performance analysis function, execute a simple PyTorch code, trigger performance analysis, capture and analyze the generated log file, and verify whether it contains the correct AMD GPU information. (2) In the boundary condition testing phase, execute the instance in an environment that does not contain an AMD GPU, an environment with incorrect driver configuration, and an environment with different operating systems to see if the prompts are logical. (3) In the abnormal testing phase, it is necessary to simulate abnormal situations such as unauthorized permissions and check whether the abnormal messages and log outputs are in line with expectations.

[0040] Step 4: Capture the performance of your AI application using PyTorch Profiler

[0041] After perfecting the Kineto function, recompile PyTorch, add the performance analyzer function to the inference or training code of the program, and mark the start, end, and checkpoints. After adding, run the program to capture the performance analysis data.

[0042] In this embodiment, after improving the Kineto function, PyTorch is recompiled and the performance of the artificial intelligence application is captured using Profiler. The obtained performance data fills in the missing key attributes and information.

[0043] The specific steps are: add the performance analyzer function torch.profiler.profile in the artificial intelligence application, mark the start recording time (prof.start), end recording time (prof.stop) and checkpoint (prof.step).

[0044] 1. prof = torch.profiler.profile( 2.schedule=torch.profiler.schedule(wait=1, warmup=1, active=10,repeat=1), 3.on_trace_ready=torch.profiler.tensorboard_trace_handler(". / log"), 4.record_shapes=True, 5.with_stack=True 6. ) 7. prof.start() 8. with torch.no_grad(): 9.for i in range(10): 10.result = model(**inputs) 11.prof.step() 12. prof.stop()

[0045] After marking, run the application to collect relevant performance data for subsequent analysis and optimization.

[0046] Step 5: Verify that Tensorboard supports visual analysis of performance data captured by PyTorch Profiler

[0047] Profiler stores the collected performance data in a format that can be parsed by Tensorboard. Execute commands to visualize and analyze logs, such as Figure 3As shown, verify in the browser whether the new function supports Tensorboard visualization, and check whether the overview, kernel, Operator, and trace views can be displayed normally. If they can be displayed normally, it means that after the function is expanded, Profiler is compatible with AMD GPU performance data analysis and also supports Tensorboard visualization.

[0048] In this embodiment, PyTorch Profiler stores the collected performance data as a format file that can be parsed by Tensorboard. Use Tensorboard to read the files in the specified directory and visualize them to verify whether the new function supports Tensorboard visualization.

[0049] For example, Profiler stores performance data in the log directory. Tensorboard executes the "tensorboard --logdir=. / log --bind_all" command to visualize performance data. In addition, you need to ensure that the default port 6006 is open to the browser host. At this time, open the browser and enter "localhost:6006" for visualization analysis.

[0050] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0051] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0052] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0053] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0054] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. An AMD compatibility extension method based on the performance analysis tool PyTorch Profiler, characterized in that: include: According to the performance analysis requirements, the performance analysis library Kineto in the performance analysis tool PyTorch Profiler is extended with new functions; Verify the extended functions in the performance analysis library Kineto; In response to the verified performance analysis tool PyTorch Profiler capturing the performance data of the large model program, visual analysis and verification of the performance data of the large model program is performed; In response to the passing of the visual analysis verification, the AMD compatibility extension of the performance analysis tool PyTorch Profiler was completed.

2. The AMD compatibility extension method based on the performance analysis tool PyTorch Profiler according to claim 1, characterized in that: Before expanding the functions of the performance analysis library Kineto in the performance analysis tool PyTorch Profiler according to the performance analysis requirements, the method also includes: providing the underlying environment support of the open source computing platform ROCm for the basic hardware environment.

3. The AMD compatibility extension method based on the performance analysis tool PyTorch Profiler according to claim 2, characterized in that: The provision of ROCm underlying environment support for the open source computing platform for the basic hardware environment includes: installing a customized ROCm software package under the domestic platform architecture, and / or, under the x86 architecture, consulting the installation documents officially provided by AMD, configuring the environment according to the document steps and installing the corresponding software package.

4. The AMD compatibility extension method based on the performance analysis tool PyTorch Profiler according to claim 2 or 3, characterized in that: The performance analysis requirement is to improve the key attributes and information that the performance analysis tool PyTorch Profiler lacks in analyzing the performance of large model programs on AMD; According to the performance analysis requirements, the performance analysis library Kineto in the performance analysis tool PyTorch Profiler is extended to include: Plan the specific extension content of the performance analysis library Kineto based on the missing key attributes and information; Add an interface for separately obtaining AMD GPU information in the performance analysis library Kineto, and define the attribute string required to be called by the interface according to the specific extended content; The attribute character string corresponding to the specific extended content is obtained through the interface to complete the function extension.

5. The AMD compatibility extension method based on the performance analysis tool PyTorch Profiler according to claim 4, characterized in that: The verification of the extended functions in the performance analysis library Kineto includes: performing basic function tests, boundary condition tests, and exception handling tests on the extended functions respectively.

6. The AMD compatibility extension method based on the performance analysis tool PyTorch Profiler according to claim 5, characterized in that: The basic function test, boundary condition test and exception handling test are respectively performed on the extended functions, including: In the basic function verification phase, verify whether the log file generated after the performance analysis function of the performance analysis library Kineto is turned on contains the correct AMD GPU information; In the boundary condition testing phase, check whether the prompts after executing the instance in different environments are logical, wherein the environments include an environment without an AMD GPU, an environment with a driver configuration error, and an environment with a different operating system; During the exception testing phase, check whether the exception message and log output after simulating the exception situation are as expected.

7. The AMD compatibility extension method based on the performance analysis tool PyTorch Profiler according to claim 6, characterized in that: The process of capturing the large model program performance data in response to the verified performance analysis tool PyTorch Profiler and performing visual analysis and verification on the large model program performance data includes: Recompile PyTorch Profiler, a performance analysis tool that improves Kineto's extended functions; Use the recompiled performance analysis tool PyTorch Profiler to analyze the performance of large model programs on AMD, capture performance data and store it in a format that can be parsed by the visualization analysis tool Tensorboard.

8. The AMD compatibility extension method based on the performance analysis tool PyTorch Profiler according to claim 7, characterized in that: In response to the visual analysis verification being passed, the AMD compatibility extension of the performance analysis tool PyTorch Profiler is completed, including: Use the visualization analysis tool Tensorboard to visualize the performance data collected by the performance analysis tool PyTorch Profiler; Verify in the browser whether the extension function supports Tensorboard visualization and check whether each view is displayed normally. If so, obtain the performance analysis tool PyTorch Profiler that is compatible with AMD GPU performance data analysis and supports Tensorboard visualization. If not, repeat the function expansion steps of the performance analysis library Kineto until the visualization analysis verification passes.

Images