An optimization system and optimization method for Triton compiler pipeline

Through an optimization system for the Triton compiler pipeline, the lack of optimization process management in the existing technology is solved, the ability to generate high-performance binary code is realized, and the code performance and system adaptability are improved.

CN118605850BActive Publication Date: 2024-11-12ZHEJIANG LAB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411075543.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-12
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

The existing Triton compiler process lacks optimization process management and testing methods for building pipelines, and cannot generate high-performance binary code that is suitable for heterogeneous GPU targets.

Method used

An optimization system for the Triton compiler pipeline is designed, including dialect manager, conversion process manager, custom optimization process constructor, optimization process manager, pipeline constructor, optimization process tester and performance evaluator, through these Component management and optimization of the transformation process of the Triton language, generate object code that is adapted to heterogeneous GPU architecture, and perform performance evaluation.

Benefits of technology

Implement the writing and application of custom optimization strategies, improve code computing performance, ensure the correctness and performance evaluation of the optimization process, improve the system's GPU adaptability and scalability, simplify the compilation process, and improve code quality. and system maintainability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118605850B_ABST
    Figure CN118605850B_ABST
Patent Text Reader

Abstract

The present invention discloses an optimization system and optimization method for a Triton compiler pipeline, the system comprising a dialect manager, a conversion process manager, a custom optimization process constructor, an optimization process manager, a pipeline constructor, an optimization process tester and a performance evaluator. The present invention provides users with an interface and template for writing a performance optimization strategy for an operation combination of a dialect involved in a conversion process through a custom optimization process constructor, so that users can design optimization strategies in a targeted manner and improve code operation performance; the optimization process manager allows specific optimization pass processes to be added for different GPU architectures, so that the optimization system has better GPU adaptability; the optimization process tester ensures the correct execution of the optimization process, and the performance evaluator provides a quantitative evaluation so that the optimization effect can be verified. The optimization system of the present invention can perform fine control over dialects, conversion processes, and optimization processes, providing management flexibility and scalability of the optimization system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computing framework programming and compilers for large-scale artificial intelligence model training, and in particular to an optimization system and optimization method for a Triton compiler pipeline. Background Art

[0002] Triton is a programming language and compiler released by OpenAI in the United States for writing efficient custom deep learning primitives. Most of the existing Triton compiler processes focus on directly converting the Triton language into certain dialects for specific backend GPU architectures. They lack the optimization process management and testing methods for building pipelines and cannot generate high-performance binary code that adapts to heterogeneous GPU targets. Summary of the invention

[0003] In view of the deficiencies of the prior art, the present invention provides an optimization system and optimization method for the Triton compiler pipeline, and the specific technical solution is as follows:

[0004] An optimization system for Triton compiler pipeline, including a dialect manager, a conversion process manager, a custom optimization process constructor, an optimization process manager, a pipeline constructor, an optimization process tester and a performance evaluator;

[0005] The dialect manager is used to register the relevant dialects involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine into the optimization system, and manage the relevant information of the algorithm involved in the dialect, and the relevant information includes operations, attributes, characteristics, and constraints;

[0006] The conversion process manager is used to register the conversion process involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine into the optimization system, and manage the operation combination contained in the dialect involved in each conversion process;

[0007] The custom optimization process constructor provides an interface and template for users to write performance optimization strategies for the operation combination of the dialect involved in the conversion process; the optimization strategies include pattern matching, loop transformation, and data flow analysis;

[0008] The optimization process manager is used to insert the optimization process selected by the user into the compilation pipeline between different dialect conversion processes;

[0009] The pipeline constructor is used to integrate the conversion process registered by the conversion process manager and the optimization process selected by the user to generate a target code running on a heterogeneous backend GPU architecture;

[0010] The optimization process tester is used to test the target code constructed by the pipeline constructor to ensure the semantic correctness of the constructed pipeline;

[0011] The performance evaluator is used to evaluate the computing performance of the target code obtained by the pipeline constructor and provide it to the user to help the user adjust the optimization process selected by the user.

[0012] Further, the custom optimization process constructor includes a dialect selection module, a dialect optimization analysis module and an optimization process generation module;

[0013] The dialect selection module analyzes the descending path from the dialect involved in the high-level multi-layer intermediate representation to the dialect involved in the low-level GPU according to the characteristics of the target GPU architecture, including its memory hierarchy, execution unit and instruction set, and outputs a recommended descending path according to the computing performance of the operation combination included in the semantically equivalent dialect;

[0014] The dialect optimization analysis module is used to further analyze the intermediate representation composed of multiple dialects involved in the descending path selected by the user, parse out the operation type, data flow and control flow in the intermediate representation, and the way in which the operation type, data flow and control flow are mapped to the target GPU architecture, and identify possible optimization statements;

[0015] The optimization process generation module is used to generate a corresponding optimization process according to the optimization strategy and the corresponding algorithm.

[0016] Further, possible optimization statements include redundant operations, inefficient data access patterns, and unexploited parallelism.

[0017] Furthermore, for redundant operations, the performance optimization strategy written by the user is to merge multiple redundant operations into one operation;

[0018] For inefficient data access patterns, the performance optimization strategy written by the user is to transform the loop, including loop unrolling, loop merging or loop blocking;

[0019] For unused parallelism, user-written performance optimization strategies take advantage of the parallel architecture of the GPU and decompose the computing task into multiple parts that can be executed in parallel.

[0020] Furthermore, for the identified possible optimization statements, the performance optimization strategy written by the user is to optimize the memory access mode, reduce the use of memory bandwidth and improve cache utilization.

[0021] Furthermore, possible optimization statements are identified through analysis in three dimensions: dependency analysis, data flow analysis, and scheduling algorithm.

[0022] Furthermore, the adjustment of the optimization process selected by the user includes adding or deleting certain optimization processes, or adjusting the execution order of multiple optimization processes.

[0023] Furthermore, the performance evaluator evaluates the computing performance of the target code obtained by the pipeline constructor using evaluation indicators including execution time, memory usage, throughput and power consumption.

[0024] Further, the performance evaluator evaluates the computing performance of the target code obtained by the pipeline constructor using a method including a benchmark test, a comparison test, and an experimental test;

[0025] The benchmark test is to evaluate the computing performance of the target code using a standardized benchmark test program;

[0026] The comparative test is to compare the computing performance of the target code obtained by using the optimization process with the target code obtained without using the optimization process;

[0027] The experimental test is to change the input data scale or GPU architecture to evaluate the change in the computing performance of the target code caused by the optimization process.

[0028] An optimization method for a Triton compiler pipeline, the method is implemented based on an optimization system for a Triton compiler pipeline, and the method specifically includes:

[0029] Through the dialect manager, the relevant dialects involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine are registered in the optimization system, and the relevant information of the algorithm involved in the dialect is managed, and the relevant information includes at least operations, attributes, characteristics, and constraints;

[0030] Through the conversion process manager, the conversion process involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine is registered in the optimization system, and the operation combination contained in the dialect involved in each conversion process is managed;

[0031] Writing a performance optimization strategy for the combination of dialect operations involved in the conversion process through the interface and template provided by the custom optimization process builder; the optimization strategy includes pattern matching, loop transformation, and data flow analysis;

[0032] By means of the optimization process manager, the optimization process selected by the user is added to the compilation pipeline between different dialect conversion processes;

[0033] By integrating the conversion process registered by the conversion process manager and the optimization process selected by the user through the pipeline constructor, a target code running on a heterogeneous backend GPU architecture is generated;

[0034] Testing the pipeline constructed by the pipeline constructor through the optimization process tester to ensure the semantic correctness of the constructed pipeline;

[0035] The performance evaluator is used to evaluate the computing performance of the target code obtained by the pipeline constructor, and provides the performance to the user to adjust the optimization process selected by the user.

[0036] The beneficial effects of the present invention are as follows:

[0037] (1) Performance improvement: The custom optimization process builder provides users with interfaces and templates for writing performance optimization strategies for the dialect operation combinations involved in the conversion process, allowing users to design optimization strategies in a targeted manner to improve code computing performance; the optimization process manager allows the addition of specific optimization pass processes for different GPU architectures, making the optimization system more GPU-adaptable; the optimization process tester ensures the correct execution of the optimization process, and the performance evaluator provides quantitative evaluation, making the optimization effect verifiable.

[0038] (3) Flexibility and scalability: The optimization system of the present invention can perform fine control over dialects, conversion processes, and optimization processes, providing management flexibility and scalability for the optimization system.

[0039] (4) The optimization system of the present invention can realize automation of the optimization process, simplify the compilation process, and reduce manual configuration.

[0040] (5) The optimization system and optimization method of the present invention have modular compilation processes, which improves the code quality and the maintainability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of an optimization system for a Triton compiler pipeline provided in accordance with an embodiment of the present invention.

[0042] Figure 2 A schematic diagram of a dialect manager provided according to an embodiment of the present invention.

[0043] Figure 3 A schematic diagram of a conversion pass manager provided in accordance with an embodiment of the present invention.

[0044] Figure 4 A schematic diagram of a custom optimization pass constructor provided in accordance with an embodiment of the present invention.

[0045] Figure 5 A schematic diagram of an optimized pass manager provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more clear. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] Technical term explanation:

[0048] Triton, a language proposed by OpenAI for writing efficient custom deep learning primitives;

[0049] pipeline, assembly line;

[0050] pass, process;

[0051] dialect;

[0052] LLVM, Low Level Virtual Machine, low-level virtual machine;

[0053] GPU, Graphics Processing Unit, Graphics Processing Unit;

[0054] MLIR, Multiple Level Intermediate Representation, multi-layer intermediate representation;

[0055] Linalg, Linear Algebra, linear algebra;

[0056] SPIR-V, Standard Portable Intermediate Representation-V, Standard Portable Intermediate Representation;

[0057] IR, Intermediate Representation, intermediate representation;

[0058] Affine, affine;

[0059] SCF, Structured Control Flow

[0060] NVIDIA, Nvidia;

[0061] AMD, Advanced Micro Devices, Inc.

[0062] NVIDIA Nsight Systems

[0063] AMD CodeXL, a powerful GPU program performance analysis tool;

[0064] perf, a powerful performance analysis tool in Linux system, which provides system and application performance monitoring functions, allowing users to measure and analyze system performance indicators;

[0065] Vtune is a powerful performance analysis tool designed for software developers to help them analyze performance issues of serial and multi-threaded applications and optimize application performance.

[0066] like Figure 1 As shown, the optimization system for the Triton compiler pipeline of this embodiment includes a dialect manager 110, a conversion pass manager 120, a custom optimization pass constructor 140, an optimization pass manager 130, a pipeline constructor 150, an optimization pass tester 160 and a performance evaluator 170.

[0067] 1. Dialect Manager

[0068] The dialect manager is used to register the relevant dialects involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine into the optimization system, and manage the relevant information of the algorithm involved in the dialect, including operations, attributes, characteristics, and constraints. The namespace of each dialect is unique and is automatically loaded when the compiler is initialized through the registration mechanism. The operations contained in the dialect include defined types, attributes, and interfaces to support the implementation of specific algorithms; attribute definitions will cover basic and complex types; characteristics and constraints will guide the legal use of operations.

[0069] like Figure 2 As shown in the figure, the dialect manager imports all existing dialects in the MLIR community, including but not limited to Affine, GPU Dialect, Linalg, SPIR-V, LLVM, SCF, etc. The "management" here specifically includes the following aspects:

[0070] 1.Registration management of dialect

[0071] (1) Initialization: When the optimization system starts, the dialect manager automatically imports all existing dialects in the MLIR community and registers each dialect.

[0072] (2) Interface design: The dialect manager is designed with a unified interface for registering new dialects, ensuring that new dialects can be easily integrated into the optimization system.

[0073] (3) Data storage management: The dialect manager uses data structures (such as hash tables) to store relevant information of each dialect, including operations, attributes, characteristics, and constraints.

[0074] 2. Dialect information management

[0075] (1) Information storage management: The dialect manager creates an information storage unit for each registered dialect and records the detailed information of the dialect.

[0076] (2) Query management: The dialect manager provides a query interface that allows users to retrieve information about a specific dialect.

[0077] (3) Update management: The dialect manager allows users to modify and update existing dialect information to ensure the accuracy and timeliness of the information.

[0078] 3. Operation and management of dialect

[0079] (1) Operation registration: When registering a dialect, all operations it contains are also registered.

[0080] (2) Operation management: Provides an interface to manage the operations of each dialect and supports adding, deleting, and modifying operations.

[0081] (3) Operation execution: Design an execution mechanism that allows dialect-specific operations to be called and executed during the compilation process.

[0082] 4. Dependency management of dialect

[0083] (1) Dependency management: Record the dependencies between each dialect to ensure that the dialects are loaded and used in the correct order during the compilation process.

[0084] (2) Dependency check: When registering a new dialect, its dependencies are automatically checked to ensure that all dependent dialects have been correctly registered.

[0085] 5.Management of interaction between dialect and users

[0086] (1) Design a graphical user interface to facilitate users to browse and manage registered dialects.

[0087] (2) Provide a command line interface that allows users to manage dialect information through command line tools.

[0088] The dialect manager can efficiently manage and use all existing dialects in the MLIR community, and supports the registration and management of new dialects. In this way, the compiler can not only adapt to different hardware architectures, but also flexibly expand and optimize the compilation process, providing developers with an efficient and reliable compilation experience.

[0089] 2. Switch pass manager

[0090] The conversion pass manager is used to register the conversion process involved in the process of gradually lowering the Triton language to the intermediate representation of the low-level virtual machine into the optimization system, and manage the operation combination contained in the dialect involved in each conversion process. The conversion pass manager formulates detailed conversion rules for each dialect pair, including operation mapping and data flow conversion details.

[0091] like Figure 3 As shown, the management of the conversion pass manager specifically includes the following aspects:

[0092] 1. Registration management of conversion pass

[0093] (1) Initialization: When the system starts, the conversion pass manager automatically imports the existing conversion passes and registers each pass.

[0094] (2) Interface design: The conversion pass manager is designed with a unified interface for registering new conversion passes, ensuring that new conversion passes can be easily integrated into the optimization system.

[0095] (3) Data storage management: The conversion pass manager uses data structures (such as hash tables) to store relevant information of each conversion pass, including the name of the conversion pass, function description, input and output types, etc.

[0096] 2. Conversion pass information management

[0097] (1) Information storage management: Create an information storage unit for each registered conversion pass to record the detailed information of the conversion pass.

[0098] (2) Query management: The conversion pass manager provides a query interface that allows users to retrieve information about a specific conversion pass.

[0099] (3) Update management: The conversion pass manager allows modification and updating of existing conversion pass information to ensure the accuracy and timeliness of the information.

[0100] 3. Conversion pass execution management

[0101] (1) Execution interface: The transformation pass manager designs a unified execution interface for calling and executing specific transformation passes.

[0102] (2) Execution process management: The transformation pass manager manages the execution order and dependencies of the transformation passes to ensure that each transformation pass is executed in the correct order during the compilation process.

[0103] (3) Execution log: The conversion pass manager records the detailed log of each conversion pass execution, including execution time, input and output data, execution results, etc., to facilitate debugging and analysis.

[0104] 4. Manage the interaction between the conversion process and the user

[0105] (1) Design a graphical user interface to facilitate users to browse and manage registered conversion passes.

[0106] (2) Provide a command line interface that allows users to manage conversion passes through command line tools.

[0107] 3. Custom optimization pass constructor

[0108] like Figure 4 As shown, the custom optimization pass constructor in this embodiment is a key component in the optimization system for the Triton compiler pipeline. The custom optimization pass constructor provides users with interfaces and templates for writing performance optimization strategies for the operation combinations of the dialects involved in the conversion process. Optimization strategies include pattern matching, loop transformation, and data flow analysis. The custom optimization pass constructor can design efficient optimization strategies for specific dialects and implement corresponding optimization passes, thereby improving the performance and efficiency of Triton code. The custom optimization pass constructor includes a dialect selection module, a dialect optimization analysis module, and an optimization process generation module, as follows:

[0109] 1. Dialect selection module

[0110] The dialect selection module analyzes the characteristics of the target GPU architecture, including its memory hierarchy, execution units, and instruction sets, analyzes the descending path from the dialect involved in the high-level multi-layer intermediate representation to the dialect involved in the low-level GPU, and outputs the recommended descending path based on the computing performance of the operation combination included in the semantically equivalent dialect. For example, for NVIDIA GPU, Linalg dialect may be selected for optimization; for AMD GPU, SPIR-V dialect may be selected for optimization.

[0111] 2. Dialect optimization analysis module

[0112] The dialect optimization analysis module is used to further analyze the IR composed of multiple dialects involved in the descending path selected by the user, parse out the operation types, data flows and control flows in the intermediate representation, and the way the operation types, data flows and control flows are mapped to the target GPU architecture. Through the three dimensions of dependency analysis, data flow analysis and scheduling algorithm, it identifies possible optimization statements, including redundant operations, inefficient data access patterns, unused parallelism, etc.

[0113] For the identified possible optimization statements, the optimization strategy is designed in the following way:

[0114] For redundant operations, the performance optimization strategy written by the user is to merge multiple redundant operations into one operation to reduce the complexity of the intermediate representation;

[0115] For inefficient data access patterns, user-written performance optimization strategies are loop transformations, including loop unrolling, loop merging, or loop blocking, to improve parallelism and memory access efficiency;

[0116] For unused parallelism, user-written performance optimization strategies take advantage of the parallel architecture of the GPU and decompose the computing task into multiple parts that can be executed in parallel.

[0117] In addition, for the identified possible optimization statements, the memory access pattern can be optimized to reduce the use of memory bandwidth and improve cache utilization; or, for each optimization strategy, a suitable algorithm can be selected or designed to implement these optimizations.

[0118] 3. Optimize the pass generation module

[0119] The optimization pass generation module is used to generate the corresponding optimization process according to the optimization strategy and the corresponding algorithm, and integrate the implemented optimization pass into the Triton compiler. The optimization pass needs to follow the modularity and extensibility principles of MLIR and keep the code clear and efficient.

[0120] Here are a few examples of the optimization process:

[0121] (1) Linalg dialect optimization pass:

[0122] 1) Operation fusion: Fusion of operations such as matrix multiplication and vector addition to reduce the number of memory accesses.

[0123] 2) Loop transformation: expand, merge or block the loop to improve parallelism and memory access efficiency.

[0124] (2) SPIR-V dialect optimization pass:

[0125] 1) Memory optimization: Optimize memory access patterns, such as using local memory, shared memory, etc.

[0126] 2) Instruction optimization: select more efficient instructions or instruction sequences.

[0127] Different GPU architectures have different characteristics and optimization requirements, and different optimization strategies need to be designed for different architectures; different application scenarios have different performance bottlenecks, and appropriate optimization strategies need to be selected according to the application scenarios; Code complexity: The design of the optimization pass needs to consider the complexity and maintainability of the code. Preferably, a series of tools and libraries provided by the MLIR toolchain can help developers design and implement optimization passes; Preferably, the LLVM compiler provides a wealth of optimization technologies that can be borrowed and applied to custom optimization passes.

[0128] 4. Optimize the pass manager

[0129] The optimization pass manager is used to insert user-selected optimization passes into the compilation pipeline between different dialect conversion passes. Figure 5 As shown, the functions implemented by the optimized pass manager are as follows:

[0130] 1. Registration management of optimization pass

[0131] (1) Initialization: When the optimization system starts, the optimization pass manager automatically imports the existing optimization passes and registers each optimization pass.

[0132] (2) Interface design: The optimization pass manager is designed with a unified interface for registering new optimization passes, ensuring that new optimization passes can be easily integrated into the optimization system.

[0133] (3) Data storage management: The optimization pass manager uses data structures (such as hash tables) to store relevant information of each optimization pass, including the name of the optimization pass, function description, input and output types, etc.

[0134] 2. Information management of optimization pass

[0135] (1) Information storage management: The optimization pass manager creates an information storage unit for each registered optimization pass to record the detailed information of the dialect.

[0136] (2) Query management: The optimization pass manager provides a query interface that allows users to retrieve information about a specific optimization pass.

[0137] (3) Update management: The optimization pass manager allows modification and updating of existing optimization pass information to ensure the accuracy and timeliness of the information.

[0138] 3. Execution management of optimization pass

[0139] (1) Execution interface: The optimization pass manager is designed with a unified execution interface for calling and executing specific optimization passes.

[0140] (2) Execution process management: The optimization pass manager manages the execution order and dependencies of optimization passes to ensure that each optimization pass is executed in the correct order during the compilation process.

[0141] (3) Execution log: Records a detailed log of each optimization pass execution, including execution time, input and output data, execution results, etc., to facilitate debugging and analysis.

[0142] 4. Optimize the interaction management between pass and users

[0143] (1) Design a graphical user interface to facilitate users to browse and manage registered optimization passes.

[0144] (2) Provide a command line interface that allows users to manage optimization passes through command line tools.

[0145] 5. Pipeline Constructor

[0146] The pipeline constructor is used to integrate the conversion process registered by the conversion process manager and the optimization process selected by the user to generate target code running on the heterogeneous backend GPU architecture, thereby realizing a complete compilation and optimization process customized specifically for the heterogeneous backend GPU architecture. The pipeline constructor is a key component in the optimization system for the Triton compiler pipeline, and its main functions include:

[0147] 1. Pipeline construction:

[0148] (1) Select transformation pass: The pipeline constructor selects the corresponding transformation pass and optimization pass according to the order of the defined transformation passes.

[0149] (2) Parameter configuration: The pipeline constructor configures parameters for each transformation pass, such as optimization strategies, execution options, etc.

[0150] (3) Integrated transformation pass: The pipeline constructor integrates the selected transformation pass and optimization pass into a unified pipeline.

[0151] 2. Target code generation:

[0152] (1) Code generation: Use the MLIR toolchain or LLVM compiler to generate target code specific to the heterogeneous backend GPU architecture.

[0153] (2) Optimize target code: Preferably use optimization techniques of the target architecture, such as instruction scheduling, register allocation, etc., to further improve code performance.

[0154] 3. User Interface:

[0155] (1) Graphical User Interface: The pipeline constructor is designed with a graphical user interface to facilitate users to configure and build pipelines.

[0156] (2) Command line tool: The pipeline builder provides a command line tool that allows users to manage the pipeline through the command line.

[0157] 6. Optimize the pass tester

[0158] The optimization pass tester is used to test the target code built by the pipeline builder to ensure the semantic correctness of the built pipeline. When performing optimization pass testing, it is necessary to manually write or use tools or scripts to automatically generate test cases, and use simulators or emulators to build a simulation environment of the target GPU architecture so that simulation environment testing can be performed without a physical GPU. In addition, it is also necessary to test on a real GPU device to ensure the accuracy of the test results.

[0159] When testing the target code, the optimization pass tester needs to execute the pipeline constructor to generate the target code for the target GPU architecture and record the execution results, including execution time, memory usage, performance indicators, etc. Secondly, compare the test results with the expected results. Finally, perform error detection, including:

[0160] (1) Use static analysis tools to detect potential errors in the target code, such as type errors, memory leaks, etc.

[0161] (2) Use dynamic analysis tools to detect errors that occur during the execution of the target code, such as segmentation faults, deadlocks, etc.

[0162] (3) Use debugging tools to debug the optimization pass, such as setting breakpoints and viewing variable values.

[0163] 7. Performance Evaluator

[0164] The performance evaluator is used to evaluate the computing performance of the target code obtained by the pipeline constructor and provide it to the user to help the user adjust the optimization process selected by the user to improve the performance and efficiency of the code. When performing performance evaluation, use performance analysis tools such as NVIDIA Nsight Systems or AMD CodeXL to evaluate the performance of the pipeline; use performance statistics tools such as perf or VTune to collect performance data. Alternatively, you can also customize the performance evaluation tool: develop a custom performance evaluation tool to more accurately evaluate the performance in a specific application scenario.

[0165] When conducting performance evaluation, the performance evaluation methods used include benchmark testing, comparative testing, and experimental testing. Among them, benchmark testing is to use standardized benchmark testing programs to evaluate the computing performance of the target code; comparative testing is to compare the computing performance of the target code obtained using the optimization process with the target code obtained without the optimization process; experimental testing is to change the input data scale or GPU architecture to evaluate the changes in the computing performance of the target code caused by the optimization process. In the performance evaluation process, there are the following evaluation indicators:

[0166] (1) Execution time: Measure the execution time of the target code corresponding to the pipeline and compare it with the unoptimized target code or the target code using different optimization strategies.

[0167] (2) Memory usage: Measure the memory usage of the target code corresponding to the pipeline and compare it with the unoptimized target code or the target code using different optimization strategies.

[0168] (3) Throughput: Measure the throughput of the target code corresponding to the pipeline, such as the number of samples processed per second, and compare it with the unoptimized target code or the target code using different optimization strategies.

[0169] (4) Power consumption: Measure the power consumption of the target code corresponding to the pipeline, such as the energy consumed per second, and compare it with the unoptimized target code or the target code using different optimization strategies.

[0170] Finally, the performance evaluator analyzes and visualizes the performance evaluation results. For example, statistical analysis is performed on the performance evaluation results, such as calculating the mean, variance, standard deviation, etc.; the performance evaluation results are analyzed to identify performance bottlenecks and determine the optimization direction. The performance evaluation results are visualized using charts or graphs to facilitate user analysis and understanding.

[0171] Based on the evaluation results, the user can adjust the optimization process selected by the user, including adding or deleting certain optimization processes, or adjusting the execution order of multiple optimization processes, or optimizing the target code, such as optimizing algorithms or data structures.

[0172] On the other hand, an embodiment of the present invention further provides an optimization method for a Triton compiler pipeline, which is implemented based on the above optimization system and includes:

[0173] Through the dialect manager, the relevant dialects involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine are registered in the optimization system, and the relevant information of the algorithm involved in the dialect is managed, and the relevant information includes at least operations, attributes, characteristics, and constraints;

[0174] Through the conversion process manager, the conversion process involved in the process of gradually lowering the Triton language to the intermediate representation of the low-level virtual machine is registered in the optimization system, and the operation combination contained in the dialect involved in each conversion process is managed;

[0175] Through the interface and templates provided by the custom optimization process builder, write the performance optimization strategy of the dialect operation combination involved in the conversion process; the optimization strategy includes pattern matching, loop transformation, and data flow analysis;

[0176] Add user-selected optimization passes to the compilation pipeline between different dialect conversion passes through the optimization pass manager;

[0177] Through the pipeline builder, the conversion process registered by the conversion process manager and the optimization process selected by the user are integrated to generate the target code running on the heterogeneous backend GPU architecture;

[0178] By optimizing the process tester, the pipeline constructed by the pipeline constructor is tested to ensure the semantic correctness of the constructed pipeline;

[0179] The performance evaluator is used to evaluate the computing performance of the target code obtained by the pipeline constructor, and provides the user with the opportunity to adjust the optimization process selected by the user.

[0180] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention shall be included in the protection scope of the invention.

Claims

1. An optimization system for Triton compiler pipeline, characterized in that: Includes dialect manager, conversion process manager, custom optimization process builder, optimization process manager, pipeline builder, optimization process tester and performance evaluator; The dialect manager is used to register the relevant dialects involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine into the optimization system, and manage the relevant information of the algorithm involved in the dialect, and the relevant information includes operations, attributes, characteristics, and constraints; The conversion process manager is used to register the conversion process involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine into the optimization system, and manage the operation combination contained in the dialect involved in each conversion process; The custom optimization process constructor provides an interface and template for users to write performance optimization strategies for the operation combination of the dialect involved in the conversion process; the optimization strategies include pattern matching, loop transformation, and data flow analysis; The optimization process manager is used to insert the optimization process selected by the user into the compilation pipeline between different dialect conversion processes; The pipeline constructor is used to integrate the conversion process registered by the conversion process manager and the optimization process selected by the user to generate a target code running on a heterogeneous backend GPU architecture; The optimization process tester is used to test the target code constructed by the pipeline constructor to ensure the semantic correctness of the constructed pipeline; The performance evaluator is used to evaluate the computing performance of the target code obtained by the pipeline constructor and provide it to the user to help the user adjust the optimization process selected by the user; The custom optimization process constructor includes a dialect selection module, a dialect optimization analysis module and an optimization process generation module; The dialect selection module analyzes the descending path from the dialect involved in the high-level multi-layer intermediate representation to the dialect involved in the low-level GPU according to the characteristics of the target GPU architecture, including its memory hierarchy, execution unit and instruction set, and outputs a recommended descending path according to the computing performance of the operation combination included in the semantically equivalent dialect; The dialect optimization analysis module is used to further analyze the intermediate representation composed of multiple dialects involved in the descending path selected by the user, parse out the operation type, data flow and control flow in the intermediate representation, and the way in which the operation type, data flow and control flow are mapped to the target GPU architecture, and identify possible optimization statements; The optimization process generation module is used to generate a corresponding optimization process according to the optimization strategy and the corresponding algorithm.

2. The optimization system for Triton compiler pipeline according to claim 1, characterized in that: Possible optimization statements include redundant operations, inefficient data access patterns, and unexploited parallelism.

3. The optimization system for Triton compiler pipeline according to claim 2, characterized in that: For redundant operations, the performance optimization strategy written by the user is to merge multiple redundant operations into one operation; For inefficient data access patterns, the performance optimization strategy written by the user is to transform the loop, including loop unrolling, loop merging or loop blocking; For unused parallelism, user-written performance optimization strategies take advantage of the parallel architecture of the GPU and decompose the computing task into multiple parts that can be executed in parallel.

4. The optimization system for Triton compiler pipeline according to claim 1, characterized in that: For the identified possible optimization statements, the performance optimization strategy written by the user is to optimize the memory access mode, reduce the use of memory bandwidth and improve cache utilization.

5. The optimization system for Triton compiler pipeline according to claim 1, characterized in that: Through analysis in three dimensions: dependency analysis, data flow analysis, and scheduling algorithm, possible optimization statements can be identified.

6. The optimization system for Triton compiler pipeline according to claim 1, characterized in that: Adjustments to the optimization process selected by the user include adding or deleting certain optimization processes, or adjusting the execution order of multiple optimization processes.

7. The optimization system for Triton compiler pipeline according to claim 1, characterized in that: The performance evaluator evaluates the computing performance of the target code obtained by the pipeline constructor using evaluation indicators including execution time, memory usage, throughput and power consumption.

8. The optimization system for Triton compiler pipeline according to claim 1, characterized in that: The performance evaluator evaluates the computing performance of the target code obtained by the pipeline constructor using a method including a benchmark test, a comparison test, and an experimental test; The benchmark test is to evaluate the computing performance of the target code using a standardized benchmark test program; The comparative test is to compare the computing performance of the target code obtained by using the optimization process with the target code obtained without using the optimization process; The experimental test is to change the input data scale or GPU architecture to evaluate the change in the computing performance of the target code caused by the optimization process.

9. An optimization method for Triton compiler pipeline, characterized in that: The method is implemented based on the optimization system for the Triton compiler pipeline described in any one of claims 1 to 8, and the method specifically includes: Through the dialect manager, the relevant dialects involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine are registered in the optimization system, and the relevant information of the algorithm involved in the dialect is managed, and the relevant information includes at least operations, attributes, characteristics, and constraints; Through the conversion process manager, the conversion process involved in the process of gradually descending the Triton language to the intermediate representation of the low-level virtual machine is registered in the optimization system, and the operation combination contained in the dialect involved in each conversion process is managed; Writing a performance optimization strategy for the combination of dialect operations involved in the conversion process through the interface and template provided by the custom optimization process builder; the optimization strategy includes pattern matching, loop transformation, and data flow analysis; By means of the optimization process manager, the optimization process selected by the user is added to the compilation pipeline between different dialect conversion processes; By integrating the conversion process registered by the conversion process manager and the optimization process selected by the user through the pipeline constructor, a target code running on a heterogeneous backend GPU architecture is generated; Testing the pipeline constructed by the pipeline constructor through the optimization process tester to ensure the semantic correctness of the constructed pipeline; The performance evaluator is used to evaluate the computing performance of the target code obtained by the pipeline constructor, and provides the performance to the user to adjust the optimization process selected by the user.