Triton operator automatic fusion method and system based on compiling guide information

Abstract

The invention relates to the technical field of deep learning GPU operator compilation optimization, and provides a Triton operator automatic fusion method and system based on compilation guide information, and the method is realized based on a Triton intermediate presentation layer, and comprises the steps: converting the compilation guide information into a mapping table, dynamically constructing a fusion operator signature based on the mapping table, and removing redundant parameters. Recursive inheritance is carried out through the metadata label of the temporary buffer area; function body topology reconstruction is achieved through an intermediate representation mapper, automatic integration optimization is carried out on a fused loop structure, and iteration redundancy is eliminated; the deep optimization of the access chain is realized through address standardization, storage forwarding and dead storage elimination; a complete fusion operator is formed through terminator processing, and iterative fusion is supported to complete full calculation graph operator integration. According to the method, operator codes do not need to be manually modified, zero-intrusive development is achieved, video memory occupation and memory access overhead are remarkably reduced, kernel scheduling delay is reduced, and the hardware utilization rate and research and development efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning GPU operator compilation optimization technology, and in particular to an automatic fusion method and system for Triton operators based on compilation guidance information. Background Technology

[0002] Triton enables developers to bypass complex CUDA programming and directly write operators (kernels) that approach the performance of the underlying hardware by providing a high-level intermediate representation (IR). In Triton's compilation process, TTIR (Triton Intermediate Representation) is the key bridge connecting high-level domain-specific languages ​​with low-level hardware instructions (such as PTX / SASS). The current Triton compiler primarily uses a single-operator independent compilation mode, meaning that for each Python layer computation operator (such as Matrix Multiplication, Softmax, Element-wise Addition), the compiler treats it as an independent compilation unit and generates a corresponding GPU operator with an independent global function signature. While Triton excels in single-operator optimization, it has the following significant drawbacks when handling the execution flow of multiple consecutive operators: The lack of static analysis across operator boundaries: The original Triton compiler architecture treats each kernel as an independent, closed space, lacking the ability to perform static analysis across kernel boundaries and thus lacking the feature of automatic operator fusion. Because Triton relies heavily on pointer offsets for memory access, traditional compiler analysis techniques struggle to automatically deduce the data dependencies between two independent kernels without running the program, resulting in the inability to achieve automatic fusion between operators.

[0003] Expensive memory access overhead: Due to the inability to achieve static fusion across operators, the computation result of the previous operator in the operator chain must be physically written back through video memory (HBM / VRAM), and then read back through video memory for the next operator. For memory-intensive operators (such as bias addition after convolution, activation functions, etc.), frequent global video memory reads and writes become a performance bottleneck, greatly wasting video memory bandwidth.

[0004] Kernel scheduling and boot latency redundancy: The execution of multiple independent operators implies multiple kernel launch processes. Each boot involves instruction scheduling between the host and device sides, parameter passing in constant buffers, and synchronization overhead on the device side. In fine-grained, operator-intensive models, this accumulated scheduling overhead severely squeezes the effective utilization time of computing units.

[0005] The high barriers to entry and maintenance costs of operator fusion: Under current technology, to achieve operator fusion, developers typically need to manually rewrite the Python layer source code, hardcoding multiple computational logics into a large kernel. This approach not only increases code complexity but also exponentially increases the difficulty of development, debugging, and maintenance due to issues such as variable name conflicts, address recalculation, and register spilling.

[0006] Optimizing the loss of metadata during the transformation process: Even when attempting simple instruction stacking, existing compilation frameworks cannot automatically identify and mark which variables belong to "intermediate temporary buffers" that only exist within operators when processing intermediate variables. This prevents the compiler backend from performing aggressive register reuse or memory optimization, and the merged kernel often faces significant resource pressure. Summary of the Invention

[0007] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides an automatic fusion method and system for Triton operators based on compilation boot information, which eliminates the need for manual modification of operator code, achieves zero-intrusion development, significantly reduces video memory usage and memory access overhead, reduces kernel scheduling latency, and improves hardware utilization and R&D efficiency.

[0008] This invention provides an automatic fusion method for Triton operators based on compilation boot information. The method is implemented based on the Triton intermediate representation layer and includes: S1: The compiler receives hardware constraints, the original Triton intermediate representation layer function operation objects, and compilation boot information, and converts the compilation boot information into compiler-operable numerical pairs and stores them in the mapping table; S2: Based on hardware constraints, the original Triton intermediate representation layer function operation objects, and the mapping table, merge the input parameters of the producer and consumer operators in the Triton intermediate representation layer, remove redundant parameter types based on the mapping table, and use the consumer operator output return value type as the fusion operator output return value type to create a Triton intermediate representation layer fusion operator framework with globally unique function symbols. S3: Add temporary buffer metadata tags to the intermediate parameters of the Triton intermediate representation layer fusion operator framework, and support the recursive inheritance of the metadata tags in multi-level operator fusion through the attribute scanning mechanism; S4: Create an empty fusion function body and initialize the intermediate representation mapper. In the Triton intermediate representation layer, recursively clone the producer operator instructions and record the value mapping relationship through the intermediate representation mapper. Redirect the consumer operator mapping parameter placeholders and establish direct association through register-level data flow. In the Triton intermediate representation layer, eliminate the direct dependent instructions and reorganize the dependency chain of the indirect dependent instructions to obtain the reconstructed fusion function body. S5: Identify the structured control flow loop operation instruction blocks arranged continuously in the reconstructed fusion function body, verify the equivalence of the iteration space in the Triton intermediate representation layer, unify the loop iteration variables, inline the computational load and integrate the control flow instructions to obtain the fusion function body after control flow integration. S6: Normalize the fragmented pointer offset logic in the fusion function body after control flow integration into a base pointer plus unified offset mode. Identify the life cycle of intermediate variables through data flow analysis. Use register lifting and memory access folding techniques to eliminate redundant global storage and repeated loading, and construct a new fusion operator. S7: Bind the return value of the new fusion operator and close the static single assignment chain. Use the new fusion operator as the new producer operator and iteratively execute S1 to S6 to complete the fusion of all target operators.

[0009] Furthermore, the compilation guidance information includes producer operator parameter identifiers, consumer operator parameter identifiers, and mapping relationship strings; Step S1 includes: S11: Perform asymmetric splitting on the mapping string to obtain the producer-side identifier and the consumer-side identifier; S12: Traverse the parameter list of the producer operator. If the name attribute of the parameter of the producer operator matches the producer-side identifier exactly, record the absolute position index of the parameter of the producer operator in the parameter list to obtain the producer operator parameter index. S13: Traverse the parameter list of the consumer operator. If the name attribute of the parameter of the consumer operator matches the consumer-side identifier exactly, record the absolute position index of the parameter of the consumer operator in the parameter list to obtain the consumer operator parameter index. S14: The producer operator parameter index and the consumer operator parameter index form a numeric pair that the compiler can operate on, and the numeric pair is stored in the mapping table.

[0010] Furthermore, step S2 includes: S21: Based on hardware constraints, the original Triton intermediate representation layer function operation object stores all input parameter types of the producer operator into the input list of the fusion operator in the Triton intermediate representation layer; S22: Traverse the consumer operator parameter index. If there is a numerical pair belonging to the mapping table, the consumer operator parameter is a redundant input and is not included in the input list of the fusion operator. If there is no numerical pair belonging to the mapping table, the consumer operator parameter is a necessary external input and the parameter type of the consumer operator parameter is added to the input list of the fusion operator. S23: Use the output of the consumer operator as the output type of the fusion operator; S24: Based on the input list and output type of the fusion operator, allocate a new function operation node in the compiler's memory space, assign a globally unique function symbol to the new function operation node, and generate a Triton intermediate representation layer fusion operator framework with globally unique function symbols.

[0011] Furthermore, step S3 includes: S31: Add a persistent dictionary attribute to the parameters of the new function operation node of the Triton intermediate representation layer fusion operator framework, and use the persistent dictionary attribute as a temporary buffer label; S32: When the fusion operator participates in the next round of fusion as a producer, if a temporary buffer label is retrieved, the temporary buffer label is passed through to the newly generated function signature to realize the recursive inheritance of metadata labels.

[0012] Furthermore, step S4 includes: S41: Create an empty fusion function body and initialize the intermediate representation mapper; S42: In the Triton intermediate representation layer, the compiler traverses the original instruction stream of the producer operator in static single assignment order. For each cloned instruction, the intermediate representation mapper immediately establishes and records a pair of physical static single assignment mappings to complete the base clone and value registration of the producer instruction. S43: Before starting the consumer operator cloning, pre-injection logic is executed through the intermediate representation mapper, and the dependency chain of the producer operator is traversed in reverse using a depth-first search method, and the first... Placeholders for the mapping parameters are redirected to the producer operator. The cloning results of each parameter are stored in an empty fusion function body to form new calculated values ​​in the fusion function; S44: Skip the direct dependency instruction clones in the consumer operator that depend on the mapping parameters. For indirect dependency instructions that depend on the results of the direct dependency instructions, establish direct associations through register-level data streams. In the Triton intermediate representation layer, through the intermediate representation mapper, traverse the dependency chain of the producer operator in a depth-first search manner, rebind its operands to the corresponding new producer values ​​in the fusion operator, and obtain the reconstructed fusion function body.

[0013] Furthermore, step S5 includes: S51: Perform a depth-first search within the fusion function body to find consecutively arranged loop structure instruction blocks and extract the loop start value, loop end value, and loop step size of each loop structure instruction block. S52: In the Triton intermediate representation layer, verify the iteration space equivalence of two adjacent loop structure instruction blocks. If the loop start value, loop end value and loop step size of the two loop structure instruction blocks are the same, then merge the two loop structure instruction blocks. S53: Abolish the original loop counter of the consumer operator, create a globally unique loop iteration variable, and redirect the calculation logic involving the iteration index in the two original loops to the loop iteration variable; S54: The effective calculation instructions of the two loop structure instruction blocks are stripped through the intermediate representation mapper. The effective calculation instructions of the producer loop structure instruction block are moved to the beginning of the new loop. The effective calculation instructions of the consumer loop structure instruction block directly reference the calculation result of the producer loop structure instruction block through the register. S55: Integrates loop control flow instructions, reduces boundary checks, jumps, and loop iteration variable increment operations, lowers hardware thread synchronization frequency and pipeline pause count, and obtains a fused function body after control flow integration.

[0014] Furthermore, step S6 includes: S61: By address calculation normalization, the pointer offset logic scattered in different operators of the fused function body after the control flow is integrated is reorganized into a standard calculation mode with a base pointer plus a unified offset; S62: Within the same thread block execution context, retrieve the store instruction operation and load instruction operation that point to the same normalized address. If the addresses are equivalent and the masks are compatible, then use the Triton intermediate representation layer semantics through the intermediate representation mapper to directly map the original value of the store instruction to the downstream user of the load instruction, and eliminate redundant load instructions. S63: Perform lifetime analysis on the storage instruction carrying the temporary buffer label. If the storage address of the storage instruction is not referenced by any load instruction in the current kernel function and subsequent iterations, the storage instruction is a dead storage instruction. The storage instruction is discarded to obtain a new fusion operator.

[0015] Furthermore, step S61 includes: Distribution calculation before optimization: The Splat instruction is executed to convert the base address scalar into a tensor, the AddPtr instruction is used to add row offsets, and the Broadcast instruction is executed to expand the dimensions and add column offsets in the tensor space. Optimized normalized reorganization: Through algebraic reorganization, all offsets are first aggregated in the scalar domain, and then the Splat and AddPtr instructions are executed once.

[0016] Furthermore, step S7 includes: S71: Retrieve the original return operands of the consumer operators in the new fusion operator, trace the latest static single assignment generated in the fusion function body through the intermediate representation mapper, create a return instruction operation at the end of the fusion kernel function and bind the static single assignment; S72: Remove the integrated producer and consumer operator objects from the current compiled module, releasing compile-time memory; S73: Re-register the generated fusion kernel function into the symbol table as the producer operator for the next round of fusion; S74: Based on the upper_bound attribute of the producer and consumer loop instructions, extract and verify the consistency of the boundary values ​​of the two on the common iteration axis, uniformly set the termination condition of the fused loop to the verified boundary value, and through SSA value tracking technology, form a unique exit predicate in the fused IR, and iteratively execute S1 to S6 to complete the fusion of all target operators.

[0017] This invention also provides an automatic Triton operator fusion system based on compiler boot information, for executing the above-mentioned automatic Triton operator fusion method based on compiler boot information, comprising: The parsing and mapping module is used by the compiler to receive hardware constraints, raw Triton intermediate representation layer function operation objects, and compilation boot information, and convert the compilation boot information into compiler-operable numerical pairs and store them in the mapping table. The fusion operator signature construction module, based on hardware constraints, the original Triton intermediate representation layer function operation objects, and the mapping table, merges the input parameters of the producer and consumer operators in the Triton intermediate representation layer, removes redundant parameter types based on the mapping table, and uses the output return value type of the consumer operator as the output return value type of the fusion operator to create a Triton intermediate representation layer fusion operator framework with globally unique function symbols. The metadata tag recursive inheritance module adds temporary buffer metadata tags to the intermediate parameters of the Triton intermediate representation layer fusion operator framework, and supports the recursive inheritance of the metadata tags in multi-level operator fusion through an attribute scanning mechanism. The structure cloning module creates an empty fusion function body and initializes an intermediate representation mapper. In the Triton intermediate representation layer, the producer operator instructions are recursively cloned and the value mapping relationship is recorded through the intermediate representation mapper. The consumer operator mapping parameter placeholders are redirected. By establishing a direct association with the register-level data flow, the direct dependent instructions are eliminated and the dependency chain of the indirect dependent instructions is reorganized in the Triton intermediate representation layer to obtain the reconstructed fusion function body. The loop integration optimization module identifies the structured control flow loop operation instruction blocks arranged continuously in the reconstructed fusion function body, verifies the equivalence of the iteration space, unifies the loop iteration variables, inline the computational load and integrate the control flow instructions to obtain the fusion function body after control flow integration. The memory access optimization module normalizes the fragmented pointer offset logic in the fusion function body after control flow integration into a base pointer plus unified offset mode. It identifies the life cycle of intermediate variables through data flow analysis, and uses register lifting and memory access folding technology to eliminate redundant global storage and repeated loading, and constructs a new fusion operator. The iterative fusion module binds the return value of the new fusion operator and closes the static single assignment link. It uses the new fusion operator as a new producer operator and iteratively executes S1 to S6 to complete the fusion of all target operators.

[0018] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: This invention utilizes dynamic parameter remapping technology to directly direct the input of the consumer operator to the computational output of the producer operator at the instruction level. This transforms data exchange, which previously required the Global Boot Memory (HBM) as an intermediary, into direct access at the GPU's on-chip cache or register level. This completely eliminates redundant "write-back-read" operations for intermediate results, significantly reducing memory bandwidth usage for typical memory-intensive operator chains (such as Add-ReLU and Scale-Bias).

[0019] By fusing multiple independent operator logics into a single fused operator (Fused Kernel), this invention combines multiple kernel launches into a single launch. This eliminates CPU-GPU synchronization latency and driver call overhead between multiple operator launches, and reduces the amount of constant memory occupied by multiple operators repeatedly passing similar parameters (such as Stride and Shape information).

[0020] This invention employs a semantically guided declarative architecture, eliminating the need for developers to manually modify, concatenate, or rewrite complex underlying operator code. With only externally input compilation guidance information, the compiler can automatically handle complex logic concatenation and address management. This significantly shortens the development cycle of high-performance fusion operators and avoids logical errors that might be introduced by manual code refactoring.

[0021] This invention solves the problem of metadata loss during the fusion process. The recursive inheritance of attributes allows operator fusion to move beyond pairwise merging and support continuous deep fusion of three-level, four-level, and even longer operator sequences. When processing long-path computation flows in deep neural networks such as Attention, it enables globally optimal resource allocation.

[0022] The basic block-level instruction migration technology based on the TTIR layer can accurately maintain the computational topology of the original operators, automatically handle kernel termination semantics (ReturnOp), and ensure that the fused numerical results are completely consistent with those executed independently. Variable conflicts are avoided through symbolic namespace management, ensuring compilation stability in complex operator fusion scenarios.

[0023] This invention utilizes loop integration optimization technology to identify and merge adjacent loop operators with the same iteration space, eliminating redundant sensing variable calculations and loop control jump logic within the fusion operator, and reducing the hardware synchronization frequency during kernel execution. By integrating multiple computational tasks into a single loop pipeline, it effectively improves the utilization of the compute unit (ALU) and instruction issuance efficiency.

[0024] This invention utilizes memory access forwarding and dead memory elimination techniques to standardize and reorganize the address calculation link. By implementing direct forwarding from store to load at the instruction level and eliminating redundant load and store operations marked as temporary buffers, intermediate calculation results do not need to be exchanged through physical GPU memory. This truly achieves register-level data closure across operator boundaries, maximizing the mitigation of the limitations imposed by the GPU memory wall on computing power.

[0025] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Fig. 1 This is a flowchart illustrating an automatic fusion method for the Triton operator based on compilation guidance information provided by the present invention.

[0028] Fig. 2 This is a fusion diagram of an automatic fusion method for the Triton operator based on compilation guidance information provided by the present invention.

[0029] Fig. 3 These are experimental comparison diagrams before and after fusion in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0031] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0032] The following is combined Figs. 1 to 3 This invention describes an automatic fusion method and system for the Triton operator based on compilation boot information.

[0033] The system's input is a complete computation graph, which is a topological graph consisting of several operators arranged in a certain topological order, representing the operator execution sequence. The system iteratively merges adjacent operators until a complete fused operator is generated. The system compiles the Triton operators written by engineers into TTIR intermediate representations, i.e., function operands (FuncOps), which can be understood as a finer-grained, more easily compiled and optimized program. The first two FuncOps in the computation graph are designated as the producer and consumer operators, respectively. The consumer operator's input depends on the producer operator's output. After several subsequent compilation stages, combined with the compiler guidance information and hardware information provided by the developer, the two operators are merged into a new fused operator. This fused operator then serves as the producer operator, iteratively merging with subsequent operators to generate the final fused operator.

[0034] like Fig. 1 and Fig. 2 As shown, an automatic fusion method for Triton operators based on compilation boot information is described. This method is implemented using the Triton intermediate representation layer and includes: S1: The compiler receives hardware constraints, the original Triton intermediate representation layer function operation objects, and compilation boot information, and converts the compilation boot information into compiler-operable numerical pairs and stores them in the mapping table; The goal of this step is to establish a mapping and conversion mechanism from compilation boot information to operator physical addresses.

[0035] The compiler receives hardware constraints, the original FuncOp, and externally input compilation guidance information. FuncOp refers to the function definition containing the sequence of computational instructions in the TTIR code compiled from a manually written Triton operator using the Triton compiler; it fully preserves the operator's computational topology and parameter semantics. The compilation guidance information is passed as user input as a parameter to the pass, specifically including producer operator parameter identifiers, consumer operator parameter identifiers, and a mapping string. The producer and consumer operator parameter identifiers identify which is the producer and which is the consumer, and the mapping string identifies the parameter mapping relationship between the producer and consumer operator parameter identifiers. A key-value dictionary is used, where the key is the keyword name and the value is the corresponding value. For example, the format of the compilation guidance information is defined as mapping="n1->m1, n2->m2", where n1, n2 and m1, m2 correspond to the input parameter index numbers of the function to be fused at the Triton IR level. Internally, the compiler uses string parsing logic to extract numerical values ​​using delimiters and converts them into integer pairs, storing them in a memory mapping table. This process visualizes abstract textual guidance information into parameter bindings that the compiler can directly address and manipulate.

[0036] In the field of compilers, "pass" is a technical term specifically referring to an algorithmic flow module that traverses the intermediate representation of a program and executes specific optimization logic. In the operator fusion scheme disclosed in this invention, each pass corresponds to an execution stage. By performing specific topology reconstruction, instruction redirection, or redundancy elimination on FuncOp, the physical transformation from multiple independent operators to a single, efficient fused operator is achieved.

[0037] A parsing algorithm is used to parse the mapping string into a set of logical mapping relationships. Specifically: Let the set of parameters for the producer operator be... , , For the first The set of producer operator parameters and the set of consumer operator parameters are as follows: , , For the first Each consumer operator parameter. The parsing process extracts numerical pairs. Establish mapping relationship , The data source points directly to The storage space, of which, For the first Consumer operator parameters, For the first The parameters of the producer operator, that is, the parameters in the producer operator and the consumer operator that have a mapping relationship, are mathematically equivalent; therefore, all It can be replaced with This opens up possibilities for subsequent compilation optimizations.

[0038] To achieve accurate conversion from external descriptive strings to compiler internal parameter indices, this invention designs a matching algorithm based on symbol lookup and position mapping, specifically including: S11: Perform asymmetric splitting on the mapping string to obtain the producer-side identifier and the consumer-side identifier; The string scanner first performs asymmetric splitting on the input mapping string (e.g., "matmul_output->add_input_A"), using the delimiter "->" as an anchor point to divide the string into producer-side identifiers (left side) and consumer-side identifiers (right side), i.e., producer-side identifiers. and consumer-side identifiers The example shows only one mapping relationship in the mapping string, but in actual system execution, there may be several mapping relationships.

[0039] S12: Traverse the parameter list of the producer operator. If the name attribute of the parameter of the producer operator matches the producer-side identifier exactly, record the absolute position index of the parameter of the producer operator in the parameter list to obtain the producer operator parameter index. Since FuncOp is generated by the Triton compiler during the initial compilation stage, it retains all parameter name attributes from the function signature when the code was written manually. Therefore, the matching process is as follows: The compiler iterates through the parameter list of the producer operator, calling the MLIR framework's attribute retrieval interface to compare the `mlir::StringAttr` name attribute of each parameter. When a parameter name exactly matches the producer-side identifier (such as "matmul_output"), it records the parameter's absolute position index in the list. .

[0040] S13: Traverse the parameter list of the consumer operator. If the name attribute of the parameter of the consumer operator matches the consumer-side identifier exactly, record the absolute position index of the parameter of the consumer operator in the parameter list to obtain the consumer operator parameter index. Similarly, in the parameter list of the "consumer operator", the parameter that matches the identifier on the right (such as add_input_A) is retrieved, and its absolute position index is recorded. .

[0041] S14: The producer operator parameter index and the consumer operator parameter index form a numeric pair that the compiler can operate on, and the numeric pair is stored in the mapping table.

[0042] After matching is complete, the system converts the compilation boot information into numerical pairs that the compiler can manipulate. This is then stored in an intermediate data structure—a mapping table. At this point, no actual instruction replacement occurs; instead, a connection binding is established. This means that in future fusion function bodies, any instruction that interacts with the original consumer will be bound together. Data that is related to or has dependencies on any of the parameters must be forcibly redirected to the original producer. The parameter will also be referenced from the consumer. The parameter is converted into the producer's first parameter. One parameter.

[0043] S2: Based on hardware constraints, the original Triton intermediate representation layer function operation objects, and the mapping table, merge the input parameters of the producer and consumer operators in the Triton intermediate representation layer, remove redundant parameter types based on the mapping table, and use the consumer operator output return value type as the fusion operator output return value type to create a Triton intermediate representation layer fusion operator framework with globally unique function symbols. The goal of this step is to design a valid physical interface for the newly generated fusion operator. At the compiler level, a function's signature is defined by the sequence of types of its input parameters and the sequence of types of its output return value, which determines how the hardware allocates registers and video memory addresses for the operator.

[0044] From the compiler's perspective, each input parameter of an operator (such as a pointer or integer) is considered a "data slot" with specific position width and attributes. When creating a new function, the compiler must declare the data type of these slots in advance (e.g., f32). (Represents a 32-bit floating-point pointer) so that the data alignment and read / write protocol can be determined at the hardware layer.

[0045] S21: Based on hardware constraints, the original Triton intermediate representation layer function operation object stores all input parameter types of the producer operator into the input list of the fusion operator in the Triton intermediate representation layer; All input parameter types of the producer operator Store the input list of the fusion operator .

[0046] S22: Traverse the consumer operator parameter index. If there is a numerical pair belonging to the mapping table, the consumer operator parameter is a redundant input and is not included in the input list of the fusion operator. If there is no numerical pair belonging to the mapping table, the consumer operator parameter is a necessary external input and the parameter type of the consumer operator parameter is added to the input list of the fusion operator. The "redundant slot" deduplication logic based on mapping relationship: This is the core algorithm to reduce the complexity of the merged operator and is used to determine which slots can be removed.

[0047] Traversing the consumer operator parameter index If a logical mapping relationship exists Then determine This is redundant input and is not counted. , It is a set of logical mapping relationships; If it does not exist, then determine For necessary external input, append its type to ; Using this type of deduplication, the total number of parameters in the fusion operator is: in, The total number of parameters for the fusion operator. For the number of parameters of the producer operator, The number of parameters for the consumer operator. The number of logical mapping relationships; S23: Use the output of the consumer operator as the output type of the fusion operator; In the operator chain, the producer's output has been transformed into the consumer's internal input and is no longer the end point of the entire chain. Only the output type of the end of the chain (the consumer) can represent the data specification of the final output of the fusion operator. Therefore, the output return type of the "consumer operator" (i.e., the data format of the final calculation result) is directly used as the exit definition of the fusion operator.

[0048] S24: Based on the input list and output type of the fusion operator, allocate a new function operation node in the compiler's memory space, assign a globally unique function symbol to the new function operation node, and generate a Triton intermediate representation layer fusion operator framework with globally unique function symbols.

[0049] By assigning a globally unique function symbol, it is ensured that the fusion operator can be precisely addressed and executed during subsequent compilation, linking, and low-level hardware driver calls.

[0050] S3: Add temporary buffer metadata tags to the intermediate parameters of the Triton intermediate representation layer fusion operator framework, and support the recursive inheritance of the metadata tags in multi-level operator fusion through the attribute scanning mechanism; The temporary buffer metadata tag is intended to mark this intermediate parameter as an intermediate buffer parameter of the fusion operator, which can help the memory access optimization pass to perform optimizations. Otherwise, this pass will not perform redundant load and store instruction optimizations for safety.

[0051] In standard compiler theory, function parameters (pointers) are considered "externally visible," and the backend optimizer must conservatively assume that all writes to parameters (store) are necessary because external programs may need to read this data. The `temp_buffer` label breaks this restriction, explicitly telling the compiler that the memory pointed to by the parameter is only valid within the current fusion operator, and its final write result does not need to be written back to global memory.

[0052] The temporary buffer metadata tag is a priori condition for triggering subsequent optimization pass logic in the Triton IR layer design of this invention. It solves the problem of compiler optimization being hindered by "intermediate product parameterization" in the fusion operator. Without this tag, the backend pass would be forced to retain redundant video memory write operations because it cannot determine parameter safety, resulting in a significant reduction in fusion performance.

[0053] S31: Add a persistent dictionary attribute to the parameters of the new function operation node of the Triton intermediate representation layer fusion operator framework, and use the persistent dictionary attribute as a temporary buffer label; The compiler processes numeric pairs At that time, it was identified It's no longer just ordinary output, but a "bridge" between two operators. Through the interface provided by the MLIR framework, the parameters of the new function operation node... Add a persistent dictionary attribute {"triton.temp_buffer": unit} to it. This tag is to pave the way for future fusion with more operators and to provide guidance for the memory access optimization phase.

[0054] S32: When the fusion operator participates in the next round of fusion as a producer, if a temporary buffer label is retrieved, the temporary buffer label is passed through to the newly generated function signature to realize the recursive inheritance of metadata labels.

[0055] By using the attribute walker mechanism, when the fused operator enters the bottom feedback loop again as a producer, the system will search for existing temporary buffer (temp_buffer) tags. If a temporary buffer tag is detected, it will be directly passed through to the newly generated function signature, realizing the inheritance of function attributes in the process of multiple operator fusions and avoiding the problem of unrecoverable video memory due to metadata loss in multi-level fusion.

[0056] S4: Create an empty fusion function body and initialize the intermediate representation mapper. In the Triton intermediate representation layer, recursively clone the producer operator instructions and record the value mapping relationship through the intermediate representation mapper. Redirect the consumer operator mapping parameter placeholders and establish direct association through register-level data flow. In the Triton intermediate representation layer, eliminate the direct dependent instructions and reorganize the dependency chain of the indirect dependent instructions to obtain the reconstructed fusion function body. This step is crucial for the present invention to eliminate redundant calculations and achieve a performance leap.

[0057] Eliminating direct dependencies means that data transfer that originally required "Memory Write (Store) -> Memory Read (Load)" is converted into direct register assignment after fusion. This "slot-to-slot" mapping directly eliminates the intermediate memory instructions.

[0058] Reorganizing indirect dependencies refers to the possibility of duplicate address calculations (such as offset calculations) after two operators are merged. By normalizing address calculations, the scattered pointer offset logic is reorganized into a unified base address + total offset form, which is the "reorganization" of the dependency chain.

[0059] The function body of the fusion operator is not a simple code splicing, but an instruction redirection and cloning based on a mapping table. The core task of this step is to "stitch" the logic of two independent function bodies into a unified control flow, which serves as the function body of the fusion operator. It also utilizes the mapping table parsed in the first stage to eliminate redundant memory access instructions. Its core lies in breaking down old video memory dependency links and establishing a completely new register-level data flow.

[0060] S41: Create an empty fusion function body and initialize the intermediate representation mapper; Create an empty fusion function body and initialize the intermediate representation mapper (IRMapping). The intermediate representation mapper is essentially a hash table that spans function scopes. It records which SSA value in the new fusion function corresponds to the static single assignment (SSA) in the old operator. The SSA value is a concept used by the compiler to mark an instruction. For example, %1 may correspond to an addition instruction, and %2 may correspond to a division instruction, whose divisor may come from the result of %1.

[0061] S42: In the Triton intermediate representation layer, the compiler traverses the original instruction stream of the producer operator in static single assignment order. For each cloned instruction, the intermediate representation mapper immediately establishes and records a pair of physical static single assignment mappings to complete the base clone and value registration of the producer instruction. The compiler traverses the original instruction stream of the producer operator in SSA order. For each new instruction cloned and generated, IRMapping immediately establishes and records a pair of physical value mappings: {old value in the producer operator: new calculated value in the fusion function}.

[0062] This step establishes the "data source" within the fusion operator that can be referenced later.

[0063] S43: Before starting the consumer operator cloning, pre-injection logic is executed through the intermediate representation mapper, and the dependency chain of the producer operator is traversed in reverse using a depth-first search method, and the first... Placeholders for the mapping parameters are redirected to the producer operator. The cloning results of each parameter are stored in an empty fusion function body to form new calculated values ​​in the fusion function; Before initiating consumer operator cloning, pre-injection logic is executed through the intermediate representation mapper to achieve three-level mapping transformation: Mapping transformation path: Consumer operator parameters Producer operator output parameters New calculated values ​​in the fusion function.

[0064] The system forces the first consumer operator to... Placeholders for the parameters are redirected to the producer operator's first parameter. The parameters are in the cloned result generated in step S42. At this point, the entry point of the consumer operator has been logically and physically truncated, and its data source has changed from external input to internal register value.

[0065] S44: Skip the direct dependency instruction clones in the consumer operator that depend on the mapping parameters. For indirect dependency instructions that depend on the results of the direct dependency instructions, establish direct associations through register-level data streams. In the Triton intermediate representation layer, through the intermediate representation mapper, traverse the dependency chain of the producer operator in a depth-first search manner, rebind its operands to the corresponding new producer values ​​in the fusion operator, and obtain the reconstructed fusion function body.

[0066] This process enables a shift from memory-driven to register-driven operations. By utilizing the mapping relationship established by S43, the compiler performs cascaded redundancy elimination on consumer operators, significantly improving execution efficiency.

[0067] Interception and elimination of DI (Direct Dependency Instructions): For all dependent mapping parameters in the consumer operator Instructions such as load instructions (tt.load) or address calculation instructions are considered computationally redundant. For any DI instruction, since the mapper already provides the directly computed value from the producer, the system determines that the DI is computationally redundant. In this case, the compiler skips the cloning process for the instruction; that is, the DI is not copied into the fusion operator. Therefore, buffer access instructions are removed from the fusion operator, thus physically eliminating the overhead of accessing global memory (HBM).

[0068] Cascading redirection of DDI (Dependency Instruction) instructions: Since DI instructions are eliminated, all subsequent instructions that logically depend on the DI result (named DDI, such as addition, multiplication and other calculation instructions) need to be legally processed, that is, dependency chain reorganization is performed.

[0069] When cloning a DDI, the mapper intercepts the operand request that originally pointed to the DI output and uses the mapping relationship in step S43 to rebind it to the corresponding new producer value in the fusion operator.

[0070] Physical effects: In normal execution, the producer's calculation results This will trigger a tt.store instruction to write data to the physical address of HBM (Global Memory). In the implementation of this invention, through IRMapping redirection, subsequent consumer instructions directly reference the register file. Thus, the allocation The memory bandwidth request was intercepted and eliminated by the compiler.

[0071] S5: Identify the structured control flow loop operation instruction blocks arranged continuously in the reconstructed fusion function body, verify the equivalence of the iteration space in the Triton intermediate representation layer, unify the loop iteration variables, inline the computational load and integrate the control flow instructions to obtain the fusion function body after control flow integration. This step further optimizes the fusion operator by deeply deconstructing and reorganizing the computational logic of multiple loop structures within it, thereby doubling the hardware execution efficiency. Its core lies in breaking down the original serial execution boundaries and integrating instruction flows belonging to different loops into a single iterative control flow.

[0072] S51: Perform a depth-first search within the fusion function body to find consecutively arranged loop structure (scf.for) instruction blocks and extract the loop start value of each loop structure instruction block. Loop termination value and cycle step size ; S52: In the Triton intermediate representation layer, verify the iteration space equivalence of two adjacent loop structure instruction blocks. If the loop start value, loop end value and loop step size of the two loop structure instruction blocks are the same, then merge the two loop structure instruction blocks. check , and Here, 1 and 2 refer to two consecutively arranged scf.for instruction blocks. Logical merging can only be performed when the iteration space is completely aligned.

[0073] In the context of a tieling-based parallel compiler, the mapping logic of the operators to be fused is aligned in the grid dimension and the intra-block iteration dimension. That is, the iteration space is consistent for each intermediate buffer parameter (a tensor that satisfies the producer-consumer relationship).

[0074] Taking the fusion of Gemm and Add operations with M'×N' dimensions as an example: if both are partitioned along the M' axis at the grid level (i.e., have the same Grid Config) and employ the same block iteration strategy along the N' axis at the block level (CTA), then their control flow logic (including start, end, step) is semantically equivalent. This means the grid level partitioning is consistent, and therefore the same data is processed at the block level. As for the reduction axis (K-axis) unique to Gemm, if its computational scale is within the on-chip storage capacity, it will be processed as a single load within a block or an inner nested loop, without changing the task step size of the operator on the M' and N' spatial axes. Therefore, by aligning the spatial iteration axes, the safe fusion of operators with different computational dimensions can be achieved.

[0075] The "the loop start value, loop end value and loop step size of the two loop structure instruction blocks are the same" mentioned in this invention does not require that all mathematical dimensions inside the operator be completely consistent. Rather, it requires that the "spatial mapping axes" participating in the fusion be aligned in the grid scheduling. Furthermore, when performing loop fusion in this invention, only the outermost loop is processed for fusion each time.

[0076] S53: Abolish the original loop counter of the consumer operator, create a globally unique loop iteration variable, and redirect the calculation logic involving the iteration index in the two original loops to the loop iteration variable; After confirming that two adjacent loops have completely consistent iteration spaces (i.e.) , and After all variables are identical, the system will discard the original loop counter of the consumer operator. The system will then create a new, globally unique loop iteration variable. This forces all computational logic involving iterative indices in the two original loops to be redirected to this variable. This single-variable driven mode eliminates the instruction redundancy of maintaining, updating, and overflow checking multiple counters at the hardware level.

[0077] S54: The effective calculation instructions of the two loop structure instruction blocks are stripped through the intermediate representation mapper. The effective calculation instructions of the producer loop structure instruction block are moved to the beginning of the new loop. The effective calculation instructions of the consumer loop structure instruction block directly reference the calculation result of the producer loop structure instruction block through the register. The compiler uses IRMapping as an intermediary to "strip" the valid computation instructions (payload) inside the two loop bodies. The producer operator's loop body instructions are first moved to the beginning of the new loop. Using DDI redirection technology, the instructions in the consumer loop body no longer make memory access requests to the outside, but directly reference the result just calculated by the producer through the register within the same loop iteration cycle.

[0078] S55: Integrates loop control flow instructions, reduces boundary checks, jumps, and loop iteration variable increment operations, lowers hardware thread synchronization frequency and pipeline pause count, and obtains a fused function body after control flow integration.

[0079] Previously, each loop required independent boundary checks, branch instructions, and loop iteration variable increment operations. After inlining and consolidation, these control flow instructions are significantly reduced. On parallel hardware (such as GPUs), the end of each loop is often accompanied by implicit thread synchronization or pipeline emptying. Combining two loops into one means that the number of hardware-level thread barriers and pipeline pauses is halved, greatly improving the operator's instruction execution throughput.

[0080] The achieved physical effect is a high-density closed loop for computation and access: the final fused loop, during physical execution, manifests as data flowing rapidly between the processing unit (ALU) and registers. This inline approach not only shortens the critical path of data but also ensures that the hardware execution unit is always in a saturated working state, thus overcoming the performance bottleneck caused by frequent loop starts and stops in traditional operator splicing schemes.

[0081] S6: Normalize the fragmented pointer offset logic in the fusion function body after control flow integration into a base pointer plus unified offset mode. Identify the life cycle of intermediate variables through data flow analysis. Use register lifting and memory access folding techniques to eliminate redundant global storage and repeated loading, and construct a new fusion operator. S61: By address calculation normalization, the pointer offset logic scattered in different operators of the fused function body after the control flow is integrated is reorganized into a standard calculation mode with a base pointer plus a unified offset; In the TTIR instruction stream after operator fusion, address calculation logic is often fragmented. This invention uses address calculation canonicalization technology to reorganize the pointer offset logic scattered in different operators into a standard calculation mode of "base pointer + unified offset". Taking a typical "row broadcast + column offset" memory access scenario as an example, the original instruction stream usually involves detour calculation: Step-by-step calculation (before optimization): The system first executes the Splat (scalar broadcast to tensor operation) instruction to convert the base address scalar into a tensor, then adds the row offset through the AddPtr (pointer offset calculation operation) instruction, then executes the Broadcast (dimension expansion broadcast operation) instruction to expand the dimension, and finally adds the column offset within the tensor space.

[0082] Normalized Reorganization (Optimized): This invention uses algebraic reorganization to prioritize the aggregation of all offsets in the scalar domain. The calculation expression is as follows: in, This is the total offset. This is the row offset. This is the column offset; Then only one Splat and one AddPtr command are executed; This significantly improves the hit rate of Common Subexpression Elimination (CSE) and provides a unique address identifier for subsequent memory access matching.

[0083] S62: Within the same thread block execution context, retrieve the store instruction operation and load instruction operation that point to the same normalized address. If the addresses are equivalent and the masks are compatible, then use the Triton intermediate representation layer semantics through the intermediate representation mapper to directly map the original value of the store instruction to the downstream user of the load instruction, and eliminate redundant load instructions. Within the execution context of the same thread block, the system retrieves store operations (StoreOp) and load operations (LoadOp) pointing to the same normalized address. If analysis confirms that their addresses are equivalent and their masks are compatible, IRMapping is used to directly map the original value of the store instruction to all downstream users of the load instruction. Subsequently, the system removes the redundant load instruction. This step achieves store-and-forward and load elimination of instructions. By eliminating redundant load instructions, the system eliminates the latency of reading data from global memory (HBM).

[0084] S63: Perform lifetime analysis on the storage instruction carrying the temporary buffer label. If the storage address of the storage instruction is not referenced by any load instruction in the current kernel function and subsequent iterations, the storage instruction is a dead storage instruction. The storage instruction is discarded to obtain a new fusion operator.

[0085] For store instructions labeled `temp_buffer`, the compiler performs lifetime analysis. If it is confirmed that the memory address is no longer referenced by any load instructions in the current kernel function and subsequent iterations, the store operation is considered dead code with no side effects. By eliminating such StoreOps, intermediate results reside only in the high-speed register file, physically achieving zero memory usage and zero bandwidth loss for intermediate variables.

[0086] When iterating through tt.store operations, the optimizer traces their base address. If the base address is found to be a function parameter with that label, the optimizer checks if there are any subsequent tt.load instructions within that operator. If not, the tt.store instruction is safely removed.

[0087] Register reuse and memory bandwidth optimization: Once tt.store is eliminated, the pressure of writing data back to memory via the memory bus is eliminated, and the relevant intermediate results (Tensors) can reside directly in registers or shared memory for use by subsequent instructions, thereby greatly reducing memory access latency.

[0088] S7: Bind the return value of the new fusion operator and close the static single assignment chain. Use the new fusion operator as the new producer operator and iteratively execute S1 to S6 to complete the fusion of all target operators.

[0089] S71: Retrieve the original return operands of the consumer operators in the new fusion operator, trace the latest static single assignment (SSA) generated in the fusion function body through the intermediate representation mapper, create a return instruction operation at the end of the fusion kernel function and bind the static single assignment; Return value binding and SSA link closure: The system retrieves the original return operand of the consumer operator and traces its latest generated SSA value in the fusion function body through IRMapping. The system creates a terminal symbol (ReturnOp) at the end of the newly generated fusion kernel function and physically binds the traced value to ensure topological consistency and high-fidelity output of the computation results. S72: Remove the integrated producer and consumer operator objects from the current compiled module, releasing compile-time memory; The system removes the integrated producer and consumer operator objects from the current compilation module, releasing compile-time memory; S73: Re-register the generated fusion kernel function into the symbol table as the producer operator for the next round of fusion.

[0090] The generated fusion kernel function will be re-registered in the Symbol Table as a new functional unit. This function can immediately serve as the producer operator in the next round of fusion, recursively fusion with subsequent operators in the computation graph, and repeat steps S1 to S6 to complete the fusion of all adjacent operators.

[0091] S74: Based on the upper_bound attribute of the producer and consumer loop instructions, extract and verify the consistency of the boundary values ​​of the two on the common iteration axis, uniformly set the termination condition of the fused loop to the verified boundary value, and through SSA value tracking technology, form a unique exit predicate in the fused IR, and iteratively execute steps S1 to S6 to complete the fusion of all adjacent operators.

[0092] The termination condition is a natural inheritance based on the premise that "the loop start value, loop termination value and loop step size of the two loop structure instruction blocks are the same".

[0093] The compiler uses SSA value tracking technology to make the end operands of the two operators point to the same symbol, thus forming a unique exit predicate in the generated fused IR.

[0094] This unified termination condition eliminates two separate sets of jump judgments, directly reducing the branch prediction overhead on the GPU, which is the underlying basis for improving throughput.

[0095] An automatic Triton operator fusion system based on compiler boot information, used to execute the aforementioned automatic Triton operator fusion method based on compiler boot information, includes: The parsing and mapping module is used by the compiler to receive hardware constraints, raw Triton intermediate representation layer function operation objects, and compilation boot information, and convert the compilation boot information into compiler-operable numerical pairs and store them in the mapping table; The fusion operator signature construction module, based on hardware constraints, the original Triton intermediate representation layer function operation objects, and the mapping table, merges the input parameters of the producer and consumer operators in the Triton intermediate representation layer, removes redundant parameter types based on the mapping table, and uses the output return value type of the consumer operator as the output return value type of the fusion operator, thus creating a Triton intermediate representation layer fusion operator framework with globally unique function symbols. The metadata tag recursive inheritance module adds temporary buffer metadata tags to the intermediate parameters of the Triton intermediate representation layer fusion operator framework, and supports the recursive inheritance of the metadata tags in multi-level operator fusion through the attribute scanning mechanism; The structure cloning module creates an empty fusion function body and initializes the intermediate representation mapper. In the Triton intermediate representation layer, the producer operator instructions are recursively cloned and the value mapping relationship is recorded through the intermediate representation mapper. The consumer operator mapping parameter placeholders are redirected and directly associated by establishing register-level data flow. In the Triton intermediate representation layer, the dependency chain of the directly dependent instructions is eliminated and the dependency chain of the indirectly dependent instructions is reorganized to obtain the reconstructed fusion function body. The loop integration optimization module identifies the structured control flow loop operation instruction blocks arranged continuously in the reconstructed fusion function body, verifies the equivalence of the iteration space, unifies the loop iteration variables, inline the computational load and integrate the control flow instructions to obtain the fusion function body after control flow integration. The memory access optimization module normalizes the fragmented pointer offset logic in the fusion function body after control flow integration into a base pointer plus a unified offset pattern. It identifies the life cycle of intermediate variables through data flow analysis and uses register lifting and memory access folding techniques to eliminate redundant global storage and repeated loading, and constructs a new fusion operator. The iterative fusion module binds the return value of the new fusion operator and closes the static single assignment chain, uses the new fusion operator as the new producer operator, and iteratively executes S1 to S6 to complete the fusion of all target operators.

[0096] Through the collaborative work of the above modules, this invention achieves zero-intrusion development without the need for manual modification of operator code, significantly reduces video memory usage and memory access overhead, reduces kernel scheduling latency, and improves hardware utilization and R&D efficiency.

[0097] like Fig. 3The diagram illustrates the actual performance of this invention in three typical operator chain scenarios. The three cases selected for this experiment are all core "matrix multiplication-addition operator chains" in the fields of high-performance computing and deep learning. The experiment quantitatively verifies the technical advancement of this invention by comparing the end-to-end execution time of "independent execution (before fusion)" and "execution of the present invention's solution (after fusion)" after undergoing the same conventional optimizations (such as compiler optimization level and memory access optimization measures).

[0098] Gemm (Generalized Matrix Multiplication): As the producer of operator chains, Gemm is the core of computationally intensive tasks, responsible for processing massive amounts of data and producing intermediate result matrices.

[0099] Div / ReLU / Gelu (elemental-level processing operators): These act as intermediate consumers in the operator chain, performing numerical scaling, nonlinear activation (ReLU), or the more complex mathematical distributed activation (Gelu), respectively. These operators are typically memory-intensive, meaning they are computationally simple but involve frequent reads and writes to video memory.

[0100] Add (element-wise addition): As the final consumer in the operator chain, it is responsible for summing the activated result with the bias or other matrices and outputting the final tensor.

[0101] In all test cases, the execution efficiency of the optimized operator was significantly improved. Specifically, as shown in... Fig. 3 As shown in Figure (a), in the Gemm+Div+Add scene, the time before fusion was 7.259ms, and after fusion it was 4.310ms, resulting in a speedup of 1.68x. Fig. 3 As shown in Figure (b), in the Gemm+Relu+Add scene, the time before fusion was 7.317ms, and after fusion it was 4.296ms, achieving a speedup of 1.70x. Fig. 3 As shown in Figure (c), in the more computationally complex Gemm+Gelu+Add scenario, the time before fusion is 7.622ms and the time after fusion is 4.415ms, achieving a speedup of 1.73x.

[0102] The experiments covered various operator combinations, including division (Div), activation functions (ReLU / GeLU), and element-wise addition (Add). The results consistently demonstrate that this invention, through "cascaded redundancy elimination" and "loop integration optimization" techniques, can stably and significantly reduce kernel execution time, proving its broad applicability across different operator topologies.

[0103] In conventional execution mode, the massive amounts of data generated by Gemm must be written to HBM (Global Memory), and then read back by operators such as Div / ReLU. This "write-back-read" process incurs huge bandwidth overhead. The significant reduction in execution time directly reflects that this invention eliminates a large amount of HBM memory access overhead.

[0104] This invention integrates these operator logics, enabling each line of data generated by Gemm to be passed directly to the subsequent Div and Add operators at the register level without leaving the GPU chip.

[0105] This invention eliminates the load and store instructions that intermediate operators must originally execute. The more complex the operator (e.g., from ReLU to the computationally intensive Gelu), the more memory access coordination time this invention saves, demonstrating the highest speedup (1.73x).

[0106] This invention not only achieves a logical "merger" but also a "simplification" at the physical execution level. It optimizes the originally loose execution of multiple single operators, which are limited by the speed of video memory, into a compact single high-efficiency core (fusion operator) that is limited by the processor's computing power. Thus, without changing the hardware, it maximizes the utilization of hardware throughput through compilation technology.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic fusion method for the Triton operator based on compiler boot information, characterized in that, The method is implemented based on the Triton intermediate representation layer and includes: S1: The compiler receives hardware constraints, the original Triton intermediate representation layer function operation objects, and compilation boot information, and converts the compilation boot information into compiler-operable numerical pairs and stores them in the mapping table; S2: Based on hardware constraints, the original Triton intermediate representation layer function operation objects, and the mapping table, merge the input parameters of the producer and consumer operators in the Triton intermediate representation layer, remove redundant parameter types based on the mapping table, and use the consumer operator output return value type as the fusion operator output return value type to create a Triton intermediate representation layer fusion operator framework with globally unique function symbols. S3: Add temporary buffer metadata tags to the intermediate parameters of the Triton intermediate representation layer fusion operator framework, and support the recursive inheritance of the metadata tags in multi-level operator fusion through the attribute scanning mechanism; S4: Create an empty fusion function body and initialize the intermediate representation mapper. In the Triton intermediate representation layer, recursively clone the producer operator instructions and record the value mapping relationship through the intermediate representation mapper. Redirect the consumer operator mapping parameter placeholders and establish direct association through register-level data flow. In the Triton intermediate representation layer, eliminate the direct dependent instructions and reorganize the dependency chain of the indirect dependent instructions to obtain the reconstructed fusion function body. S5: Identify the structured control flow loop operation instruction blocks arranged continuously in the reconstructed fusion function body, verify the equivalence of the iteration space in the Triton intermediate representation layer, unify the loop iteration variables, inline the computational load and integrate the control flow instructions to obtain the fusion function body after control flow integration. S6: Normalize the fragmented pointer offset logic in the fusion function body after control flow integration into a base pointer plus unified offset mode. Identify the life cycle of intermediate variables through data flow analysis. Use register lifting and memory access folding techniques to eliminate redundant global storage and repeated loading, and construct a new fusion operator. S7: Bind the return value of the new fusion operator and close the static single assignment chain. Use the new fusion operator as the new producer operator and iteratively execute S1 to S6 to complete the fusion of all target operators.

2. The method for automatic fusion of Triton operators based on compiler boot information according to claim 1, characterized in that, The compilation guidance information includes producer operator parameter identifiers, consumer operator parameter identifiers, and mapping relationship strings; Step S1 includes: S11: Perform asymmetric splitting on the mapping string to obtain the producer-side identifier and the consumer-side identifier; S12: Traverse the parameter list of the producer operator. If the name attribute of the parameter of the producer operator matches the producer-side identifier exactly, record the absolute position index of the parameter of the producer operator in the parameter list to obtain the producer operator parameter index. S13: Traverse the parameter list of the consumer operator. If the name attribute of the parameter of the consumer operator matches the consumer-side identifier exactly, record the absolute position index of the parameter of the consumer operator in the parameter list to obtain the consumer operator parameter index. S14: The producer operator parameter index and the consumer operator parameter index form a numeric pair that the compiler can operate on, and the numeric pair is stored in the mapping table.

3. The method for automatic fusion of Triton operators based on compiler boot information according to claim 1, characterized in that, Step S2 includes: S21: Based on hardware constraints, the original Triton intermediate representation layer function operation object stores all input parameter types of the producer operator into the input list of the fusion operator in the Triton intermediate representation layer; S22: Traverse the consumer operator parameter index. If there is a numerical pair belonging to the mapping table, the consumer operator parameter is a redundant input and is not included in the input list of the fusion operator. If there is no numerical pair belonging to the mapping table, the consumer operator parameter is a necessary external input and the parameter type of the consumer operator parameter is added to the input list of the fusion operator. S23: Use the output of the consumer operator as the output type of the fusion operator; S24: Based on the input list and output type of the fusion operator, allocate a new function operation node in the compiler's memory space, assign a globally unique function symbol to the new function operation node, and generate a Triton intermediate representation layer fusion operator framework with globally unique function symbols.

4. The automatic fusion method of the Triton operator based on compilation boot information according to claim 1, characterized in that, Step S3 includes: S31: Add a persistent dictionary attribute to the parameters of the new function operation node of the Triton intermediate representation layer fusion operator framework, and use the persistent dictionary attribute as a temporary buffer label; S32: When the fusion operator participates in the next round of fusion as a producer, if a temporary buffer label is retrieved, the temporary buffer label is passed through to the newly generated function signature to realize the recursive inheritance of metadata labels.

5. The method for automatic fusion of Triton operators based on compiler boot information according to claim 1, characterized in that, Step S4 includes: S41: Create an empty fusion function body and initialize the intermediate representation mapper; S42: In the Triton intermediate representation layer, the compiler traverses the original instruction stream of the producer operator in static single assignment order. For each cloned instruction, the intermediate representation mapper immediately establishes and records a pair of physical static single assignment mappings to complete the base clone and value registration of the producer instruction. S43: Before starting the consumer operator cloning, pre-injection logic is executed through the intermediate representation mapper, and the dependency chain of the producer operator is traversed in reverse using a depth-first search method, and the first... Placeholders for the mapping parameters are redirected to the producer operator. The cloning results of each parameter are stored in an empty fusion function body to form new calculated values ​​in the fusion function; S44: Skip the direct dependency instruction clones in the consumer operator that depend on the mapping parameters. For indirect dependency instructions that depend on the results of the direct dependency instructions, establish direct associations through register-level data streams. In the Triton intermediate representation layer, through the intermediate representation mapper, traverse the dependency chain of the producer operator in a depth-first search manner, rebind its operands to the corresponding new producer values ​​in the fusion operator, and obtain the reconstructed fusion function body.

6. The automatic fusion method of Triton operators based on compilation boot information according to claim 1, characterized in that, The S5 steps include: S51: Perform a depth-first search within the fusion function body to find consecutively arranged loop structure instruction blocks and extract the loop start value, loop end value, and loop step size of each loop structure instruction block. S52: In the Triton intermediate representation layer, verify the iteration space equivalence of two adjacent loop structure instruction blocks. If the loop start value, loop end value and loop step size of the two loop structure instruction blocks are the same, then merge the two loop structure instruction blocks. S53: Abolish the original loop counter of the consumer operator, create a globally unique loop iteration variable, and redirect the calculation logic involving the iteration index in the two original loops to the loop iteration variable; S54: The effective calculation instructions of the two loop structure instruction blocks are stripped through the intermediate representation mapper. The effective calculation instructions of the producer loop structure instruction block are moved to the beginning of the new loop. The effective calculation instructions of the consumer loop structure instruction block directly reference the calculation result of the producer loop structure instruction block through the register. S55: Integrates loop control flow instructions, reduces boundary checks, jumps, and loop iteration variable increment operations, lowers hardware thread synchronization frequency and pipeline pause count, and obtains a fused function body after control flow integration.

7. The automatic fusion method of the Triton operator based on compilation boot information according to claim 1, characterized in that, Step S6 includes: S61: By address calculation normalization, the pointer offset logic scattered in different operators of the fused function body after the control flow is integrated is reorganized into a standard calculation mode with a base pointer plus a unified offset; S62: Within the same thread block execution context, retrieve the store instruction operation and load instruction operation that point to the same normalized address. If the addresses are equivalent and the masks are compatible, then use the Triton intermediate representation layer semantics through the intermediate representation mapper to directly map the original value of the store instruction to the downstream user of the load instruction, and eliminate redundant load instructions. S63: Perform lifetime analysis on the storage instruction carrying the temporary buffer label. If the storage address of the storage instruction is not referenced by any load instruction in the current kernel function and subsequent iterations, the storage instruction is a dead storage instruction. The storage instruction is discarded to obtain a new fusion operator.

8. The automatic fusion method of the Triton operator based on compilation boot information according to claim 7, characterized in that, Step S61 includes: Distribution calculation before optimization: The Splat instruction is executed to convert the base address scalar into a tensor, the AddPtr instruction is used to add row offsets, and the Broadcast instruction is executed to expand the dimensions and add column offsets in the tensor space. Optimized normalized reorganization: Through algebraic reorganization, all offsets are first aggregated in the scalar domain, and then the Splat and AddPtr instructions are executed once.

9. The automatic fusion method of the Triton operator based on compilation boot information according to claim 1, characterized in that, Step S7 includes: S71: Retrieve the original return operands of the consumer operators in the new fusion operator, trace the latest static single assignment generated in the fusion function body through the intermediate representation mapper, create a return instruction operation at the end of the fusion kernel function and bind the static single assignment; S72: Remove the integrated producer and consumer operator objects from the current compiled module, releasing compile-time memory; S73: Re-register the generated fusion kernel function into the symbol table as the producer operator for the next round of fusion; S74: Based on the upper_bound attribute of the producer and consumer loop instructions, extract and verify the consistency of the boundary values ​​of the two on the common iteration axis, uniformly set the termination condition of the fused loop to the verified boundary value, and through SSA value tracking technology, make a unique exit predicate in the fused IR, and iteratively execute S1 to S6 to complete the fusion of all target operators.

10. An automatic fusion system for the Triton operator based on compiler boot information, characterized in that, To perform an automatic fusion method for Triton operators based on compilation boot information as described in any one of claims 1 to 9, comprising: The parsing and mapping module is used by the compiler to receive hardware constraints, raw Triton intermediate representation layer function operation objects, and compilation boot information, and convert the compilation boot information into compiler-operable numerical pairs and store them in the mapping table. The fusion operator signature construction module, based on hardware constraints, the original Triton intermediate representation layer function operation objects, and the mapping table, merges the input parameters of the producer and consumer operators in the Triton intermediate representation layer, removes redundant parameter types based on the mapping table, and uses the output return value type of the consumer operator as the output return value type of the fusion operator to create a Triton intermediate representation layer fusion operator framework with globally unique function symbols. The metadata tag recursive inheritance module adds temporary buffer metadata tags to the intermediate parameters of the Triton intermediate representation layer fusion operator framework, and supports the recursive inheritance of the metadata tags in multi-level operator fusion through an attribute scanning mechanism. The structure cloning module creates an empty fusion function body and initializes an intermediate representation mapper. In the Triton intermediate representation layer, the producer operator instructions are recursively cloned and the value mapping relationship is recorded through the intermediate representation mapper. The consumer operator mapping parameter placeholders are redirected. By establishing a direct association with the register-level data flow, the direct dependent instructions are eliminated and the dependency chain of the indirect dependent instructions is reorganized in the Triton intermediate representation layer to obtain the reconstructed fusion function body. The loop integration optimization module identifies the structured control flow loop operation instruction blocks arranged continuously in the reconstructed fusion function body, verifies the equivalence of the iteration space, unifies the loop iteration variables, inline the computational load and integrate the control flow instructions to obtain the fusion function body after control flow integration. The memory access optimization module normalizes the fragmented pointer offset logic in the fusion function body after control flow integration into a base pointer plus unified offset mode. It identifies the life cycle of intermediate variables through data flow analysis, and uses register lifting and memory access folding technology to eliminate redundant global storage and repeated loading, and constructs a new fusion operator. The iterative fusion module binds the return value of the new fusion operator and closes the static single assignment link. It uses the new fusion operator as a new producer operator and iteratively executes S1 to S6 to complete the fusion of all target operators.

Images