A multi-chip operator automatic generation method, device and equipment

Abstract

The present application relates to the field of artificial intelligence, and provides a multi-chip operator automatic generation method, device and equipment, the method comprising: obtaining operator demand description; in response to the operator demand description, generating a standard operator code conforming to a preset intermediate representation specification and independent of specific hardware; based on a hardware rule library of a target chip, converting the standard operator code into target operator code adapted to the target chip; wherein the hardware rule library at least includes application programming interface mapping rules, hardware characteristic constraint rules and performance and engineering experience rules; compiling and verifying the target operator code, and outputting the target operator code that passes the verification. The present application solves the problem of poor reusability and high migration cost of operator automatic generation logic in the prior art in a multi-chip scenario, realizes decoupling of algorithm logic generation and chip adaptation, and efficient and controllable multi-platform operator automatic generation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method, apparatus and device for automatic generation of multi-chip operators. Background Technology

[0002] With the continuous growth in the scale and complexity of artificial intelligence, especially deep learning models, the demand for underlying computing power is exploding. To meet this demand, computing hardware is developing towards diversification and heterogeneity. This trend makes the development of high-performance operators that can efficiently utilize the characteristics of different hardware key to improving the performance of large-scale models and scientific computing systems.

[0003] Against this backdrop, the technology of automatically generating high-performance operators based on domain-specific languages ​​combined with large language models or intelligent agents has become an important research direction for improving development efficiency and lowering technical barriers. Existing mainstream technical solutions typically adopt an "end-to-end" direct generation mode, that is, for a specific hardware platform, directly training or prompting a large model to generate code adapted to the platform's dedicated instruction set and constraints.

[0004] However, facing the current and future multi-chip, heterogeneous computing environments, the aforementioned "direct generation for a single chip" model reveals significant limitations. First, its generation logic is tightly coupled with the hardware details of a specific chip, making it difficult to reuse the core computational logic when generating operators for different chips. This necessitates repeatedly building hints or training dedicated models for each chip, resulting in enormous development and maintenance costs. Second, this model struggles to systematically accumulate and transfer engineering optimization experience for different hardware, making each migration almost a start from scratch, leading to low efficiency and uncontrollable quality. Therefore, designing an automatic operator generation architecture that decouples algorithm logic from hardware adaptation and efficiently supports multi-chip platforms has become a critical technical problem that the industry urgently needs to solve in the era of heterogeneous computing. Summary of the Invention

[0005] This invention provides a method, apparatus, and device for automatic generation of multi-chip operators, which solves the problem in the prior art where the automatic generation logic of operators is strongly coupled with the hardware details of specific chips, resulting in poor reusability and high migration costs in multi-chip scenarios. It achieves decoupling of algorithm logic generation and chip adaptation, as well as efficient and controllable automatic generation of multi-platform operators based on a structured rule base.

[0006] This invention provides a method for automatically generating multi-chip operators, comprising the following steps: Obtain the operator requirement description, which includes at least the mathematical definition, input and output tensor specifications, data type, parallel scale assumptions, and performance targets; In response to the operator requirement description, standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware is generated; Based on the hardware rule base of the target chip, the standard operator code is converted into target operator code adapted to the target chip; wherein, the hardware rule base includes at least application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; The target operator code is compiled and verified, and the verified target operator code is output.

[0007] According to the present invention, an automatic multi-chip operator generation method is provided, wherein generating standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware includes: inputting the operator requirement description into a large language model; and generating standard Triton operator code that follows Triton syntax and does not use target chip proprietary instructions based on the large language model.

[0008] According to a multi-chip operator automatic generation method provided by the present invention, the standard operator code is converted into target operator code adapted to the target chip based on the hardware rule base of the target chip, comprising: obtaining a hardware rule base corresponding to the target chip; inputting the standard operator code and the hardware rule base into a code conversion model; and automatically rewriting the standard operator code according to the hardware rule base based on the code conversion model to generate the target operator code.

[0009] According to the present invention, an automatic multi-chip operator generation method is provided, wherein the automatic rewriting of the standard operator code based on the hardware rule base to generate the target operator code includes: replacing general memory operation instructions and parallel built-in functions not supported by the target chip in the standard operator code with equivalent hardware memory access instructions and internal functions supported by the target chip, respectively, based on the application programming interface mapping rules; identifying syntax features in the standard operator code that are not supported on the target chip based on the hardware characteristic constraint rules, and performing avoidance processing or replacing them with equivalent implementations; inserting necessary boundary checks or data rearrangement logic according to the memory access characteristics of the target chip; and adjusting the parallel block parameters and thread block size in the standard operator code based on the performance and engineering experience rules to adapt to the computing unit and memory hierarchy structure of the target chip.

[0010] According to the multi-chip operator automatic generation method provided by the present invention, the hardware rule base further includes performance tuning rules, which include parallel dimension constraints, memory access alignment requirements and computing unit configuration experience data for the target chip.

[0011] According to a multi-chip operator automatic generation method provided by the present invention, the compilation and verification of the target operator code includes: performing just-in-time compilation of the target operator code to generate a computational kernel; executing the kernel to obtain initial output results and initial performance profile data; performing a numerical correctness alignment test between the initial output results and reference results; if the test fails, analyzing the failure mode and generating a first correction instruction, the first correction instruction being used to guide the revision of relevant application programming interface mapping rules or hardware characteristic constraint rules in the hardware rule base; if the test passes, comparing the initial performance profile data with the performance target; if the performance target is not achieved, generating a second correction instruction based on bottleneck analysis in the initial performance profile data, the second correction instruction being used to guide the optimization of the parallel structure in the standard operator code or the performance tuning rules in the hardware rule base; updating the hardware rule base or triggering the regeneration or conversion of the standard operator code according to the first correction instruction or the second correction instruction, and starting a new round of iteration based on the updated information until a target operator code that has passed verification and meets the performance target is generated.

[0012] The present invention also provides an automatic multi-chip operator generation device, comprising the following modules: The requirement acquisition module is used to acquire operator requirement descriptions, which include at least mathematical definitions, input and output tensor specifications, data types, parallel scale assumptions, and performance targets. The standard operator generation module is used to generate standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware in response to the operator requirement description. The target operator generation module is used to convert the standard operator code into target operator code adapted to the target chip based on the hardware rule base of the target chip; wherein, the hardware rule base includes at least application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; The compilation and verification module is used to compile and verify the target operator code and output the target operator code that has passed the verification.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-chip operator automatic generation method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-chip operator automatic generation method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multi-chip operator automatic generation method as described above.

[0016] This invention provides a method, apparatus, and device for automatically generating multi-chip operators, which offers the following advantages: By first generating standard operator code independent of specific hardware, the algorithm logic itself can be consolidated and reused, avoiding the repeated development of semantically identical but differently implemented operators for different chips, significantly improving development efficiency and reducing maintenance costs. By introducing a structured hardware rule base to drive code conversion, the adaptation knowledge for different chips is made explicit and modularized, so that when adding a new chip for adaptation, there is no need to retrain or build a complex generative model; only the rule base needs to be expanded, greatly improving the system's scalability and migration efficiency. The decoupling design of standard code and rule base allows for independent iteration updates of chip characteristics and optimization of algorithm logic, improving the flexibility of technological evolution and providing a systematic solution for quickly and reliably generating high-performance operators in heterogeneous computing environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is an overall architecture diagram of the multi-chip Triton operator automatic generation method provided by the present invention.

[0019] Figure 2 This is a flowchart illustrating the automatic generation method for multi-chip operators provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the multi-chip operator automatic generation device provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] The English terms involved in this invention will be explained below.

[0024] An agent is a software entity that can perceive the environment, make decisions, and perform actions to achieve specific goals. In this scheme, it can be used as one of the execution entities for generating standard operator code.

[0025] The Chip Adaptation Layer is responsible for converting standard Triton code into an implementation adapted to a specific chip platform while maintaining the semantics of the algorithm.

[0026] The Compile & Validate Layer is responsible for compiling, just-in-time compiling, correctness verification, and performance profiling the Triton operator after chip adaptation, and can form a feedback optimization closed loop.

[0027] Chip Rule Base is an interpretable, maintainable, and evolvable structured knowledge carrier that systematically describes the differences between Triton syntax and the target chip execution environment, including rules such as API mapping, feature constraints, and performance experience.

[0028] Just-In-Time (JIT) compilation is a program execution method that dynamically compiles code into native instructions for the target machine during runtime to improve execution efficiency.

[0029] The Logic Generation Layer, located at the top of the system, is responsible for mapping from "operator requirement descriptions" to "chip-independent standard Triton operator code".

[0030] Large Language Model (LLM) is an artificial intelligence model trained on massive amounts of data that can understand and generate natural language and code. In this scheme, it is used to understand and generate algorithm logic.

[0031] Profiling is an analytical method that collects performance data by running programs, and is used to evaluate and optimize program performance.

[0032] Triton, a domain-specific language and compiler for GPUs and other accelerators, is designed to simplify the writing of high-performance computing kernels.

[0033] The overall architecture and data flow of the "automatic generation method for multi-chip Triton operators based on hierarchical processing" proposed in this application are as follows: Figure 1 As shown, the entire process begins with the user's description of requirements, including operator functionality, inputs / outputs, and performance targets. These requirements are first input to the logic generation layer, which focuses on parsing algorithmic semantics and performing parallel computation modeling, with an intelligent agent or large language model at its core. Its output is standard Triton code that strictly follows the general syntax of Triton and is not bound to any specific hardware, thus achieving the first decoupling between algorithmic logic and hardware details.

[0034] Standard Triton code, along with target chip information, enters the core chip adaptation layer. This layer does not regenerate code; instead, it performs rule-driven code rewriting based on a structured chip rule base. The rule base systematically encapsulates multi-chip difference knowledge, including API mappings, hardware incompatibility characteristics, and performance tuning experience. Under the explicit constraints of this rule base, a lightweight code transformation model automatically transforms, rewrites, and completes the input standard code, thereby generating a Triton operator adapted to the specific target chip. This approach decouples complex chip engineering knowledge from the generation model and manages it through the rule base, significantly improving the controllability, stability, and maintainability of the adaptation process.

[0035] The adapted operators then enter the compilation and verification layer. Through a series of automated verification methods, such as just-in-time compilation, correctness checks, and performance profiling, it is ensured that the generated operators not only function correctly but also meet performance targets. The verification results can also provide feedback to optimize the rule base and model, thus forming a closed-loop system for continuous improvement.

[0036] The following is combined Figures 2-4 The embodiments of the present invention are described in detail.

[0037] The multi-chip operator automatic generation method provided in this embodiment of the invention is executed by a multi-chip operator automatic generation device, which can be configured in a computer. The computer can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0038] Figure 2 This is a flowchart illustrating the multi-chip operator automatic generation method provided by the present invention, as shown below. Figure 2 As shown, the method includes the following steps: S210. Obtain the operator requirement description. The operator requirement description should at least include the mathematical definition, input / output tensor specifications, data type, parallelism assumptions, and performance targets.

[0039] Specifically, the user or system provides the logic generation layer with a structured operator requirement description document. This document explicitly defines the specific function of the operator to be generated, such as "implementing matrix multiplication of two three-dimensional floating-point tensors". Its content includes: the mathematical definition of the operator, given in precise mathematical expressions or pseudocode, such as C[i, j] = sum_k(A[i, k] * B[k, j]), where i, j, and k are loop indices; input and output tensor specifications, explicitly stating that the shape of input tensor A is [M, K], the shape of input tensor B is [K, N], and the shape of output tensor C is [M, N]; data types, specifying that the element type of all tensors is float32; parallel scale assumptions, proposing the expectation of utilizing large-scale parallel computing, such as assuming that M, N, and K are all large values, and suggesting that the output space [M, N] be divided into multiple thread blocks for parallel computing; and performance targets, such as requiring near-theoretical peak memory bandwidth utilization or computational throughput on the target hardware. After receiving this complete operator requirement description, the large language model in the logic generation layer will perform semantic parsing and understanding, and then focus on the derivation of algorithm logic and the design of parallel structure to generate standard Triton operator code that conforms to the general Triton syntax specification, without having to care about the proprietary instructions of any specific GPU or AI chip.

[0040] This embodiment provides precise and unambiguous input specifications for the subsequent automatic generation process by clearly defining and structuring the constituent elements of the operator requirement description. This ensures that the large language model in the logic generation layer can accurately understand the user's algorithmic intent and performance expectations, thereby generating standard code that is semantically correct, architecturally sound, and chip-independent. This improves the quality and relevance of the generated results from the source, laying a reliable foundation for the subsequent chip adaptation stage.

[0041] S220: In response to the operator requirement description, generate standard operator code that conforms to the preset intermediate representation specification and is independent of specific hardware.

[0042] According to the present invention, an automatic multi-chip operator generation method is provided to generate standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware. The method includes: inputting an operator requirement description into a large language model; and generating standard Triton operator code that follows Triton syntax and does not use target chip proprietary instructions based on the large language model.

[0043] Specifically, the agent or large language model in the logic generation layer receives a structured operator requirement description from the previous step. This model first parses the mathematical definitions, tensor specifications, and parallel scale assumptions in the requirement description to understand the core computational task of the operator. For example, for a matrix multiplication requirement, the model understands its essence as calculating C = A @ B and identifies the shapes of the input tensors A and B, as well as the shape of the output tensor C. Based on this understanding, the model focuses on designing chip-independent parallel computing logic. It plans a data partitioning strategy, determining how to divide the output space into multiple thread blocks, for example, deciding that each thread block is responsible for computing a two-dimensional sub-block of the output matrix.

[0044] The model also designs how shared memory is used; for example, it decides to load specific blocks of input data into shared memory to reduce global memory access. Throughout the process, the model strictly adheres to the syntax and abstraction capabilities of the standard Triton programming language, generating a complete set of Triton kernel function code. The code generated at this stage completely avoids using any target chip-specific application programming interfaces or hardware constraints, ensuring that the generated "standard Triton operator code" is a pure intermediate representation that only expresses algorithmic semantics and general parallel logic.

[0045] This embodiment achieves hardware independence in operator logic generation by leveraging a large language model to focus on generating algorithmic semantics and general parallel logic, and strictly limiting its output to standard Triton code that does not contain any chip-specific instructions. This ensures that the generated code core has high portability and can serve as a unified and clean starting point for subsequent adaptation to various chips. This fundamentally avoids redundant work of repeatedly generating algorithmic logic for different hardware, improving the efficiency of the overall generation process and the reusability of the code.

[0046] S230, a hardware rule base based on the target chip, converts standard operator code into target operator code adapted to the target chip. The hardware rule base includes at least application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules.

[0047] According to the present invention, an automatic multi-chip operator generation method converts standard operator code into target operator code adapted to the target chip based on the hardware rule base of the target chip. The method includes: obtaining the hardware rule base corresponding to the target chip; inputting the standard operator code and the hardware rule base into a code conversion model; and automatically rewriting the standard operator code according to the hardware rule base based on the code conversion model to generate the target operator code.

[0048] Specifically, consider adapting a chip-independent standard Triton operator code generated for matrix multiplication to a graphics processor with a specific architecture. The system retrieves a hardware rule base predefined specifically for that graphics processor architecture. This rule base is a structured file or database that systematically describes the differences between the general Triton syntax and the execution environment of that graphics processor.

[0049] The chip adaptation layer takes this standard Triton code and the graphics processor's hardware rule base as joint input and provides it to a trained lightweight code translation model. The task of this lightweight code translation model (hereinafter referred to as the model) is not to understand matrix multiplication algorithms or the complex architecture of the processor from scratch, but rather to intelligently rewrite the input standard code under the explicit constraints and guidance of the rule base. For example, if the application programming interface mapping rule in the rule base indicates "mapping a specific atomic addition operation to the corresponding atomic operation supported by the graphics processor's instruction set," the model will perform the corresponding direct substitution; if the performance and engineering experience rules in the rule base suggest "optimizing the loop unrolling factor for the specific computational unit of the processor," the model will adjust the relevant loop structures in the code accordingly; if the rule base identifies a certain Triton built-in function as inefficient on the processor (hardware characteristic constraint), the model will find or generate an equivalent, more efficient sequence of basic operations to replace it. Through this rule-driven automatic rewriting, the code translation model ultimately outputs a target Triton operator code that fully adapts to the graphics processor's hardware characteristics and incorporates performance tuning experience.

[0050] This embodiment illustrates how the hardware rule base and lightweight code conversion model collaborate in the chip adaptation layer, demonstrating how to efficiently and reliably migrate general algorithmic logic to specific hardware. This approach encapsulates complex chip engineering knowledge within a maintainable rule base, allowing the conversion model to perform accurate code adaptation without requiring in-depth chip expertise; it simply acts as the rule executor. This significantly reduces the instability, high cost, and uncontrollable risks associated with directly generating chip-related code "from scratch" using large language models, thereby significantly improving the engineering feasibility, output quality consistency, and overall efficiency of the multi-chip operator generation process.

[0051] In one embodiment of the present invention, the automatic rewriting operation of the code conversion model on the standard operator code includes at least one of the following: direct replacement of the application programming interface, code structure adjustment of loop unrolling or merging, equivalent logic rewriting for unsupported syntax features, and insertion of padding code that meets memory access alignment requirements.

[0052] Specifically, the code transformation model performs a series of atomic-level automatic rewriting operations on the input standard Triton operator code based on a hardware rule base: The rule base instructs the mapping of the tl.dot (dot product) operation in the Triton standard library to internal functions supported by the target chip's dedicated computing core, such as hw_simd_dot, so that the model can perform an accurate one-to-one replacement.

[0053] To improve instruction-level parallelism, the model expands the inner loop body based on optimization suggestions for the pipeline characteristics of the target chip in the rule base; or to reduce loop overhead, it merges two independent loops with similar memory access patterns into one.

[0054] When the rule base identifies that the "tensor dynamic shape broadcasting" feature used in the standard code is not supported on the chip, the model does not directly delete the code. Instead, it rewrites it into an equivalent logical sequence that uses static shapes and explicit conditional judgments to keep the algorithm semantics unchanged.

[0055] To ensure memory access efficiency, when the model detects that the starting address of a vectorized load operation may not meet the alignment boundary required by the hardware, it will proactively insert logic before the operation, calculate and possibly execute a temporary, aligned address offset load, or insert padding elements at the data boundary to ensure aligned access to the main data block.

[0056] This embodiment reveals how rule-driven code adaptation is achieved at a fine-grained level by listing and detailing the specific automatic rewriting operations performed by the code conversion model. These operations (replacement, structural adjustment, equivalent rewriting, and insertion padding) work together to ensure that standard code accurately meets the specific requirements of the target chip in terms of instruction set, hardware constraints, and performance characteristics without altering its core algorithmic semantics. This approach decomposes the complex engineering task of hardware adaptation into a series of deterministic operations that can be reliably executed by the model, significantly improving the transparency, controllability, and quality of the final generated code in the code conversion process, and avoiding the difficult-to-debug errors and performance defects that may be introduced by "black box" generation.

[0057] According to the present invention, an automatic multi-chip operator generation method automatically rewrites standard operator code based on a hardware rule base to generate target operator code. This includes: replacing general memory operation instructions and parallel built-in functions not supported by the target chip in the standard operator code with equivalent hardware memory access instructions and internal functions supported by the target chip, based on application programming interface mapping rules; identifying syntax features in the standard operator code that are not supported on the target chip based on hardware characteristic constraint rules, and performing avoidance processing or replacing them with equivalent implementations; inserting necessary boundary checks or data rearrangement logic according to the memory access characteristics of the target chip; and adjusting the parallel block parameters and thread block sizes in the standard operator code based on performance and engineering experience rules to adapt to the computing unit and memory hierarchy structure of the target chip.

[0058] Specifically, given a chip-independent Triton code generated for standard matrix multiplication operators, the goal is to generate an adapted version for a coprocessor with a specific memory hierarchy and computational unit configuration. The code translation model in the chip adaptation layer performs the following specific rewrites based on a hardware rule base: Based on application programming interface (API) mapping rules, the model scans standard code and finds the use of the general memory load instruction `tl.load` and the specific parallel reduction function `tl.sum`. The rule base indicates that for this coprocessor, `tl.load` should be mapped to its supported dedicated vectorized load internal function `hw_vec_load`, and `tl.sum` should be mapped to a reduction loop consisting of a series of low-level computation instructions. The model then performs an exact replacement accordingly.

[0059] Based on hardware characteristic constraints, the model identified that the standard code used Triton's "dynamic shared memory indexing" feature, while the rule base explicitly indicated that this feature was not supported on this coprocessor. The model then implemented an circumvention process, reconstructing the original dynamic indexing logic into an equivalent memory access pattern based on pre-computed static offsets. Next, based on the coprocessor's memory access characteristics, the rule base instructed the model to insert address alignment checks and necessary data rearrangement logic before critical memory load operations.

[0060] Based on performance and engineering experience rules, the model analyzes the preset parallel block parameters in the standard code, and combines the optimal thread block size and shared memory capacity limit of the processor's computing core recorded in the rule base. The BLOCK_SIZE value is adjusted to 128, and the relevant loop boundaries and memory loading logic are adjusted simultaneously to ensure that the generated thread block organization structure and data block method can efficiently utilize the coprocessor's computing and storage resources.

[0061] This embodiment details how the code conversion model performs refined automatic rewriting of standard operator code based on three specific hardware rules, demonstrating the actual operation of the rule-driven adaptation mechanism. This approach explicitly encodes hardware-related engineering knowledge (instruction sets, constraints, optimization parameters) into the rule base, enabling the lightweight model to accurately and controllably complete code conversion without a deep understanding of hardware architecture details. This significantly reduces the risk of code errors and instability caused by direct generation, ensuring the consistency and performance potential of operators generated on different chip platforms, thus effectively supporting the high efficiency and reliability of automatic operator generation in multi-chip scenarios.

[0062] According to the multi-chip operator automatic generation method provided by the present invention, the hardware rule base also includes performance tuning rules, which include parallel dimension constraints, memory access alignment requirements and computing unit configuration experience data for the target chip.

[0063] Specifically, for a general-purpose coprocessor with a specific computing core and memory hierarchy architecture, its hardware rule base contains a detailed performance tuning rule. This rule clearly specifies empirical constraints and recommended data in three dimensions: First, there are constraints on the parallel dimension. For example, it is explicitly required that the number of active threads in a single thread block must be an integer multiple of 32, and the total number must not exceed 1024. At the same time, for the data block size of a specific dimension, it is recommended to set it to an integer fraction of the core local storage capacity (128 or 256) to maximize data reuse.

[0064] Secondly, there are memory alignment requirements. The performance tuning rules clearly state that in order to maximize the bandwidth of the coprocessor's vectorized memory units, the starting address of all global memory load / store operations must be aligned to the 128-byte boundary, and the size of consecutively accessed data blocks is recommended to be a multiple of 64 bytes.

[0065] Third, there is empirical data on computing unit configuration. The performance tuning rules record the number of thread bundles that can be executed simultaneously within each computing core of the coprocessor, the capacity and access latency characteristics of shared memory, and the optimal thread block shape recommended for operators with different compute-to-memory access ratios, based on extensive testing. When the code transformation model processes standard operator code, it directly references these performance tuning rules. For example, it adjusts the size of unverified thread blocks in the code to the nearest value that satisfies a multiple of 32, inserts address alignment assertions or padding code before critical memory operations, and adjusts the block factor of parallel loops based on the recommended empirical data.

[0066] This embodiment elaborates on the three core dimensions (parallel constraints, alignment requirements, and configuration experience) of performance tuning rules and their specific application in code conversion. It illustrates how to systematically encode and apply knowledge that is difficult to derive through algorithms and highly dependent on specific hardware engineering experience to the automatic generation process. This avoids relying on large language models to "guess" or generate performance parameters that may not meet hardware best practices. It ensures that the generated target operator code directly matches the physical characteristics of the target chip in key performance features, thus providing a reliable prior knowledge foundation for generating high-performance, high-resource-utilization operators, effectively improving the performance lower bound and optimization efficiency of the generated results.

[0067] In one embodiment of the present invention, hardware characteristic constraint rules in the hardware rule base are used to identify the inapplicable parts of the syntax or characteristics of the standard operator code on the target chip. The code conversion model adjusts or replaces the standard operator code based on the inapplicable parts. The standard operator code includes parallel block parameters, shared memory usage declarations, and the grid and thread block organization structure of the computing core. The hardware rule base performs collaborative optimization of the parallel block parameters and the grid and thread block organization structure based on the number of computing units and memory hierarchy of the target chip.

[0068] Specifically, during chip adaptation, hardware characteristic constraint rules are first used to identify the parts of the standard Triton operator code that are inapplicable on the target chip due to specific syntax or features. For example, the standard code might declare the use of dynamically sized shared memory, but the hardware rule base for chip A explicitly indicates that its shared memory only supports static allocation. The standard operator code itself is an intermediate representation containing complete parallel logic, defining specific parallel block parameters (e.g., BLOCK_M=128, BLOCK_N=128), shared memory usage declarations, and the grid and thread block organization structure of the computation core. Upon receiving this identification information, the code conversion model performs structural adjustments: it reconstructs the declarations of dynamic shared memory and all related access logic in the standard code operator into a static allocation scheme based on a predefined maximum capacity, and may introduce boundary checks to avoid out-of-bounds errors.

[0069] The hardware rule base optimizes the parallel block parameters and the grid and thread block organization structure in a coordinated manner based on the number of computing units and memory hierarchy of the target chip. Assume target chip B has 64 computing units and two levels of on-chip cache (L1 and shared memory). This physical information, along with empirical optimization data, is encoded in the rule base. During conversion, the model not only performs the above syntax adaptation but also actively analyzes the BLOCK_M and BLOCK_N parameters in the standard code. If the rule base indicates that the optimal block size combination to fully utilize the 64 computing units and ensure efficient data block resident in the L1 cache is (256, 64), rather than the standard (128, 128), the model will adjust these two parameters accordingly. Subsequently, the model collaboratively optimizes the entire execution structure: recalculating the grid dimensions to accommodate the new block size (e.g., from (M / / 128, N / / 128) to (M / / 256, N / / 64)), and ensuring that the total number of threads in each thread block (256*64) conforms to the constraints on thread block size in the rule base (e.g., within 1024). Furthermore, the model adjusts the shared memory usage strategy based on the memory hierarchy of chip B, for example, splitting data blocks loaded all at once in the original scheme into multiple loads to match shared memory capacity limitations.

[0070] This embodiment illustrates the two-layered effect of hardware characteristic constraint rules, demonstrating how rules drive the code conversion model to achieve precise hardware adaptation. The first layer directly circumvents hardware-unsupported syntax features. Through an "identification-refactoring" mechanism, code segments that might lead to compilation failures or runtime errors are replaced with hardware-compatible equivalents, fundamentally ensuring the executability of the generated code. The second, deeper layer involves collaborative optimization based on the hardware physical architecture. The rule base quantifies the chip's computing and storage resources into optimization constraints, guiding the model to systematically adjust the parallel execution structure of operators. This ensures that the generated target code not only "runs" but also "runs efficiently," with its execution organization deeply matched to the underlying hardware characteristics. Therefore, this embodiment not only improves the stability and success rate of code generation but also directly lays a crucial architectural foundation for the final performance of the generated operators.

[0071] S240. Compile and verify the target operator code, and output the verified target operator code.

[0072] According to the present invention, an automatic multi-chip operator generation method is provided, which compiles and verifies target operator code, including: performing just-in-time compilation of the target operator code to generate a computational kernel; executing the kernel to obtain initial output results and initial performance profile data; performing numerical correctness alignment tests between the initial output results and reference results; if the test fails, analyzing the failure mode and generating a first correction instruction, which guides the revision of relevant application programming interface mapping rules or hardware characteristic constraint rules in the hardware rule base; if the test passes, comparing the initial performance profile data with the performance target; if the performance target is not achieved, generating a second correction instruction based on bottleneck analysis in the initial performance profile data, which guides the optimization of parallel structures in the standard operator code or performance tuning rules in the hardware rule base; updating the hardware rule base or triggering the regeneration or conversion of the standard operator code according to the first or second correction instruction, and starting a new round of iteration based on the updated information until a target operator code that has passed verification and meets the performance target is generated.

[0073] Specifically, the target operator code is compiled just-in-time to generate a computational kernel that can be executed on the target coprocessor. After the kernel is executed, the system collects the initial output tensor C and initial performance profile data. C is then compared element-wise with the reference results calculated using a standard numerical library, and the maximum relative error is calculated.

[0074] If a test fails (e.g., the error exceeds a preset threshold), the system automatically analyzes the failure mode, identifies errors concentrated at certain boundary locations, and generates a first correction instruction. This instruction explicitly states that the problem may originate from an application programming interface mapping rule in the hardware rule base (e.g., an incorrect mapping of an internal function for a certain boundary handling condition) or a hardware characteristic constraint rule (e.g., incorrect handling of memory access behavior under certain boundary conditions). Based on this instruction, the system revises the corresponding rule.

[0075] If the test passes, the initial performance profile data (such as the actual number of floating-point operations per second) is compared with the performance target (the expected peak performance percentage). If the performance target is not met, the system performs bottleneck analysis based on the profile data and finds that the main bottleneck is shared memory access conflicts. Subsequently, a second correction instruction is generated, which suggests optimizing the parallel structure in the standard operator code (e.g., adjusting the arrangement of data in shared memory to reduce volume conflicts) or supplementing / correcting the performance tuning rules in the hardware rule base (e.g., adding optimization constraints for the shared memory volume access mode of this coprocessor).

[0076] Based on the first or second correction instruction, the system selectively updates the hardware rule base, or, if necessary, triggers the logic generation layer to regenerate the standard operator code (for algorithm-level modifications), before handing it over to the chip adaptation layer for conversion. The entire process begins a new round of iterations based on the updated information, recompiling, executing, testing, and comparing, until the final generated target operator code passes both the numerical correctness test and meets the preset performance targets.

[0077] This embodiment demonstrates, through a detailed description of an automated iterative loop incorporating conditional judgments and multi-path feedback, how the compilation and verification layer transcends simple correctness checks to become a proactive performance optimization and knowledge correction engine. It transforms the one-off "generate-test" process into a dynamic system capable of analyzing failure modes, identifying performance bottlenecks, generating and executing correction instructions for rule bases or standard code, and iteratively optimizing until the target is met. This significantly improves the final quality and reliability of the generated operators, continuously correcting errors and suboptimal decisions introduced during early generation or adaptation processes through closed-loop feedback. It also enables the automated and systematic accumulation of engineering knowledge, feeding back hardware adaptation experience and performance optimization strategies discovered in each iteration in the form of structured rules and solidifying them into the hardware rule base. This allows the system to continuously learn and evolve when facing new chips or operators, thereby maintaining and improving the overall efficiency and intelligence level of its multi-chip operator automatic generation in the long term.

[0078] The automatic multi-chip operator generation device provided by the present invention is described below. The automatic multi-chip operator generation device described below and the automatic multi-chip operator generation method described above can be referred to in correspondence.

[0079] like Figure 3 The image shows an automatic multi-chip operator generation device provided by the present invention, comprising: The requirement acquisition module 310 is used to acquire the operator requirement description, which includes at least the mathematical definition, input and output tensor specifications, data type, parallel scale assumptions and performance targets. The standard operator generation module 320 is used to generate standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware in response to the operator requirement description. The target operator generation module 330 is used to convert standard operator code into target operator code adapted to the target chip based on the hardware rule base of the target chip; wherein, the hardware rule base includes at least application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; The compilation and verification module 340 is used to compile and verify the target operator code and output the target operator code that has passed the verification.

[0080] Specifically, the functions of each module in the user account management system provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0081] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a multi-chip operator automatic generation method. This method includes: obtaining an operator requirement description, which at least includes mathematical definitions, input / output tensor specifications, data types, parallel scale assumptions, and performance targets; generating standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware, in response to the operator requirement description; converting the standard operator code into target operator code adapted to the target chip based on the target chip's hardware rule base; wherein the hardware rule base at least includes application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; compiling and verifying the target operator code, and outputting the verified target operator code.

[0082] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-chip operator automatic generation method provided by the above methods. The method includes: obtaining an operator requirement description, which at least includes mathematical definitions, input / output tensor specifications, data types, parallel scale assumptions, and performance targets; generating standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware in response to the operator requirement description; converting the standard operator code into target operator code adapted to the target chip based on the hardware rule base of the target chip; wherein the hardware rule base at least includes application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; compiling and verifying the target operator code, and outputting the verified target operator code.

[0084] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the multi-chip operator automatic generation method provided by the above methods. The method includes: obtaining an operator requirement description, which at least includes mathematical definitions, input / output tensor specifications, data types, parallel scale assumptions, and performance targets; generating standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware in response to the operator requirement description; converting the standard operator code into target operator code adapted to the target chip based on a hardware rule base of the target chip; wherein the hardware rule base includes at least application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; compiling and verifying the target operator code, and outputting the verified target operator code.

[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically generating multi-chip operators, characterized in that, include: Obtain the operator requirement description, which includes at least the mathematical definition, input and output tensor specifications, data type, parallel scale assumptions, and performance targets; In response to the operator requirement description, standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware is generated; Based on the hardware rule base of the target chip, the standard operator code is converted into target operator code adapted to the target chip; wherein, the hardware rule base includes at least application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; The target operator code is compiled and verified, and the verified target operator code is output.

2. The method for automatically generating multi-chip operators according to claim 1, characterized in that, The generation of standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware includes: The operator requirement description is input into the large language model; Based on the large language model, standard Triton operator code that follows Triton syntax and does not use target chip proprietary instructions is generated.

3. The method for automatically generating multi-chip operators according to claim 1, characterized in that, The hardware rule base based on the target chip converts the standard operator code into target operator code adapted to the target chip, including: Obtain the hardware rule base corresponding to the target chip; The standard operator code and the hardware rule base are input into the code conversion model; Based on the code conversion model, the standard operator code is automatically rewritten according to the hardware rule base to generate the target operator code.

4. The method for automatically generating multi-chip operators according to claim 3, characterized in that, The step of automatically rewriting the standard operator code according to the hardware rule base to generate the target operator code includes: Based on the application programming interface mapping rules, the general memory operation instructions and parallel built-in functions not supported by the target chip in the standard operator code are replaced with the equivalent hardware memory access instructions and internal functions supported by the target chip, respectively. Based on the hardware characteristic constraint rules, identify the syntax features in the standard operator code that are not supported on the target chip, and perform avoidance processing or replace them with equivalent implementations; and insert necessary boundary checks or data rearrangement logic according to the memory access characteristics of the target chip. Based on the aforementioned performance and engineering experience rules, the parallel block parameters and thread block sizes in the standard operator code are numerically adjusted to adapt to the computing unit and memory hierarchy of the target chip.

5. The method for automatically generating multi-chip operators according to claim 1, characterized in that, The hardware rule base also includes performance tuning rules, which include parallel dimension constraints, memory access alignment requirements, and computation unit configuration experience data for the target chip.

6. The method for automatically generating multi-chip operators according to claim 1, characterized in that, The compilation and verification of the target operator code includes: The target operator code is compiled in just-in-time to generate a computational kernel; the kernel is executed to obtain initial output results and initial performance profile data. The initial output result is compared with the reference result for numerical correctness alignment test; if the test fails, the failure mode is analyzed and a first correction instruction is generated, which is used to guide the revision of the relevant application programming interface mapping rules or hardware characteristic constraint rules in the hardware rule base; If the test passes, the initial performance profile data is compared with the performance target; if the performance target is not achieved, a second correction instruction is generated based on the bottleneck analysis in the initial performance profile data. The second correction instruction is used to guide the optimization of the parallel structure in the standard operator code or the performance tuning rules in the hardware rule base. According to the first correction instruction or the second correction instruction, the hardware rule base is updated or the standard operator code is regenerated or transformed, and a new round of iteration begins based on the updated information until target operator code that has passed verification and meets the performance target is generated.

7. A multi-chip operator automatic generation device, characterized in that, include: The requirement acquisition module is used to acquire operator requirement descriptions, which include at least mathematical definitions, input and output tensor specifications, data types, parallel scale assumptions, and performance targets. The standard operator generation module is used to generate standard operator code that conforms to a preset intermediate representation specification and is independent of specific hardware in response to the operator requirement description. The target operator generation module is used to convert the standard operator code into target operator code adapted to the target chip based on the hardware rule base of the target chip; wherein, the hardware rule base includes at least application programming interface mapping rules, hardware characteristic constraint rules, and performance and engineering experience rules; The compilation and verification module is used to compile and verify the target operator code and output the target operator code that has passed the verification.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multi-chip operator automatic generation method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-chip operator automatic generation method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-chip operator automatic generation method as described in any one of claims 1 to 6.

Images