A heterogeneous computing power adaptive compilation method and system for large models

Abstract

This invention provides a heterogeneous computing power adaptive compilation method for large models. The method includes: a user inputting a pre-trained large model through a system interface; a system front-end conversion module parsing and converting the model's computation graph into an intermediate representation based on a Unified Operator Description Language (UDL); a system hardware awareness module automatically detecting and extracting hardware feature fingerprints of at least one target hardware; and automatically generating optimization adaptation rules for at least one target hardware based on the UDL intermediate representation and the hardware feature fingerprints. The parameterization is based on the specific parameters of the hardware feature fingerprints and the optimization attribute tags carried in the UDL intermediate representation. Multiple back-end code generation and deployment are also included. The beneficial effects of this invention are: it enables intelligent compilation based on hardware features, significantly improving the deployment efficiency and operational performance of large models in complex heterogeneous computing power clusters.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence computing technology, and in particular to a heterogeneous computing power adaptive compilation method and system for large models. Background Technology

[0002] With the rapid growth in the scale of artificial intelligence models, especially large language models, single types of computing hardware can no longer meet their enormous computing power demands. Utilizing heterogeneous computing platforms composed of acceleration chips with different architectures such as GPUs, NPUs, and ASICs has become an inevitable choice to improve computing efficiency and reduce deployment costs.

[0003] However, hardware diversity brings significant fragmentation challenges. Chips from different manufacturers have drastically different instruction sets, memory architectures, computational precision, and dedicated programming environments (such as NVIDIA CUDA and Huawei CANN). This means that deploying the same AI model on different hardware requires rewriting a large amount of low-level operator code and optimization rules for each hardware, resulting in an exponentially increasing workload for adaptation and maintenance. To address this challenge, current mainstream solutions adopt a compilation technology approach centered on a unified intermediate representation (IR) (such as TVM and MLIR). The core idea is to convert models from different deep learning frameworks (such as PyTorch and TensorFlow) into a hardware-independent intermediate representation (IR) (such as ONNX). The compiler then analyzes and optimizes based on this intermediate representation IR, and finally generates the target code through different backend code generators. Open-source compiler frameworks such as TVM and MLIR are representative of this approach. By introducing multi-level IRs and automatic scheduling search (AutoTVM), they have achieved cross-hardware deployment to some extent.

[0004] However, this solution still has significant drawbacks when dealing with heterogeneous adaptation of large models: hardware adaptation heavily relies on manually written scheduling templates and optimization rules. Whenever a new hardware or operator needs to be supported, hardware experts must perform a large amount of manual coding and debugging, resulting in long adaptation cycles, high costs, and difficulty in maintenance and expansion. Essentially, it is still a "manual operation" mode and cannot achieve rapid and automatic adaptation to new hardware. Summary of the Invention

[0005] To address the problems in existing technologies, this invention provides a heterogeneous computing power adaptive compilation method for large models. By designing a unified operator description language (UDL), the model computation graph is decomposed into atomic operators. The system can automatically extract the instruction set and memory level features of the target chip and dynamically generate adaptation rules and optimization code (such as generating PTX for NVIDIA hardware and CANN operators for Huawei Ascend). This significantly reduces the workload of manual adaptation and achieves a leap from static compilation that relies on manual rules to intelligent compilation based on hardware features. This significantly improves the deployment efficiency and running performance of large models in complex heterogeneous computing power clusters and solves the problem in existing technologies where hardware adaptation highly depends on manually written scheduling templates and optimization rules.

[0006] The present invention provides a heterogeneous computing power adaptive compilation method for large models, comprising the following steps:

[0007] Step 1, Model Input and UDL Conversion: The user inputs a pre-trained large model through the system interface. The system front-end conversion module parses the computation graph of the model and converts it into an intermediate representation based on the Unified Operator Description Language (UDL).

[0008] Step 2, hardware feature fingerprint extraction: The system hardware perception module automatically detects and extracts the hardware feature fingerprint of at least one target hardware. The hardware feature fingerprint includes the instruction set architecture, memory hierarchy, and number of computing units of the target hardware.

[0009] Step 3, Adaptive rule generation: Based on the Unified Operator Description Language (UDL) intermediate representation and hardware feature fingerprint, the system automatically generates optimized adaptation rules for at least one target hardware by matching and parameterizing the rules through a pre-built rule template library. The parameterization is based on the specific parameters of the hardware feature fingerprint and the optimization attribute tags carried in the UDL intermediate representation.

[0010] Step 4, Multi-backend code generation and deployment: The multi-backend code generator receives the optimized Unified Operator Description Language (UDL) subgraph and the corresponding hardware-specific rules, and compiles the UDL intermediate representation into target code that can be executed on at least one target hardware according to the hardware-specific rules.

[0011] The present invention is further improved by including the following steps in step 3, where matching and parameterized filling are performed using a pre-set rule template library:

[0012] Step 301: Match the corresponding parameterized rule template from the rule template library based on the operator type in the intermediate representation of the Unified Operator Description Language (UDL) and the hardware architecture type in the hardware feature fingerprint.

[0013] Step 302: Based on the memory level parameters and computing unit parameters in the hardware feature fingerprint, and combined with the attribute tags in the intermediate representation of the Unified Operator Description Language (UDL), the undetermined parameters in the parameterized rule template are calculated and filled through the cost model to form candidate adaptation rules.

[0014] Step 303: Use the performance prediction model to evaluate the candidate code generated by the candidate adaptation rules. If the performance is not up to standard, adjust the pending parameters and regenerate the candidate adaptation rules for iterative optimization.

[0015] The present invention is further improved in that, in step 1, the Unified Operator Description Language (UDL) is a declarative language, and the data structure used to describe atomic operators includes at least operators, input and output tensor descriptions, and attribute label fields for carrying optimization intentions. The attribute label fields are used to provide optimization constraints and suggestions for parameterization in step 3.

[0016] The present invention is further improved in that, in step 1, the attribute label field includes at least one of the following: a layout label for indicating the data arrangement format, a precision label for indicating the calculation precision, a block label for indicating the cyclic block preference, and a communication awareness label for triggering network-aware optimization.

[0017] The present invention is further improved in that, in step 2, the hardware feature fingerprint of the target hardware also includes the special instruction set information and bandwidth information of the target hardware. When there are multiple target hardware and they form a heterogeneous cluster, the hardware feature fingerprint further includes the interconnection bandwidth and communication delay matrix between hardware within the cluster.

[0018] The present invention is further improved in that, in step 3, when the operator in the intermediate representation of the Unified Operator Description Language (UDL) is marked as communication-aware, the process of generating optimization adaptation rules also includes combining the interconnection bandwidth and communication delay matrix between hardware within the cluster, and synchronously calculating the computational overhead and communication overhead in the cost model to generate distributed operator partitioning and scheduling rules that make computation and communication overlap.

[0019] The present invention is further improved in that, in step 4, the operators in the Unified Operator Description Language (UDL) intermediate representation are mapped to the native instructions or operator calls corresponding to the target hardware, and PTX / CUDA code is generated for NVIDIA GPUs, or code that calls the CANN computing engine is generated for Huawei Ascend NPUs.

[0020] This invention also provides a system for implementing the above-mentioned heterogeneous computing power adaptive compilation method for large models, comprising:

[0021] The front-end conversion module is used for model input and UDL conversion. The user inputs a pre-trained large model through the system interface, and the system's front-end conversion module parses the computation graph of the model and converts it into an intermediate representation based on the Unified Operator Description Language (UDL).

[0022] The hardware awareness module is used for hardware feature fingerprint extraction. The system hardware awareness module automatically detects and extracts the hardware feature fingerprint of at least one target hardware. The hardware feature fingerprint includes the instruction set architecture, memory hierarchy and number of computing units of the target hardware.

[0023] An adaptive rule generation engine is used for adaptive rule generation. Based on the Unified Operator Description Language (UDL) intermediate representation and hardware feature fingerprint, it automatically generates optimized adaptation rules for at least one target hardware by matching and parameterizing the rules through a pre-built rule template library.

[0024] The multi-backend code generator is used for multi-backend code generation and deployment. The multi-backend code generator receives the optimized Unified Operator Description Language (UDL) subgraph and the corresponding hardware-specific rules, and compiles the UDL intermediate representation into target code that can be executed on at least one target hardware according to the hardware-specific rules.

[0025] The beneficial effects of this invention are as follows: This invention provides a heterogeneous computing power adaptive compilation method for large models. By designing a unified operator description language (UDL), the model computation graph is decomposed into atomic operators. The system can automatically extract the instruction set and memory level features of the target chip, dynamically generate adaptation rules and optimized code (such as generating PTX for NVIDIA hardware and CANN operators for Huawei Ascend), significantly reducing the workload of manual adaptation. It realizes a leap from static compilation relying on manual rules to intelligent compilation based on hardware features, significantly improving the deployment efficiency and running performance of large models in complex heterogeneous computing power clusters; it greatly improves efficiency, reducing the adaptation work of new hardware from expert research on a monthly basis to a much shorter timeframe. The automatic analysis and generation of short code via toolchains greatly accelerates the technology deployment process; it lowers the technical threshold and reduces reliance on scarce hardware expert experience, enabling ordinary developers to easily deploy large models on various heterogeneous chips, thus improving the technology's universality; it enhances system scalability by providing a standardized automatic access paradigm for new AI acceleration chips, helping to break down hardware ecosystem barriers and promote the formation of an open heterogeneous computing environment; and it achieves performance optimization by automatically combining specific hardware features and optimization intentions during compilation to generate deeply optimized target code, ensuring the running performance of large models on heterogeneous platforms and solving the problem of existing technologies where hardware adaptation highly relies on manually written scheduling templates and optimization rules. Attached Figure Description

[0026] Figure 1This is a flowchart of a heterogeneous computing power adaptive compilation method for large models according to the present invention. Detailed Implementation

[0027] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0028] Please see Figure 1 The present invention provides a heterogeneous computing power adaptive compilation method for large models, comprising the following steps:

[0029] Step 1, Model Input and UDL Conversion: The user inputs a pre-trained large model through the system interface. The system's front-end conversion module parses the computation graph of the model and converts it into an intermediate representation based on the Unified Operator Description Language (UDL). The Unified Operator Description Language (UDL) is a declarative language. The data structure used to describe atomic operators includes at least operators, input and output tensor descriptions, and attribute label fields to carry optimization intentions. The attribute label fields are used to provide optimization constraints and suggestions for parameterization in Step 3. The attribute label fields include at least one of the following: layout label indicating data arrangement format, precision label indicating computational precision, block label indicating cyclic block preference, and communication-aware label to trigger network-aware optimization.

[0030] Step 2, hardware feature fingerprint extraction: The system hardware perception module automatically detects and extracts the hardware feature fingerprint of at least one target hardware. The hardware feature fingerprint includes the instruction set architecture, memory hierarchy, and number of computing units of the target hardware. The hardware feature fingerprint of the target hardware also includes the special instruction set information and bandwidth information of the target hardware. When there are multiple target hardware and they form a heterogeneous cluster, the hardware feature fingerprint further includes the interconnection bandwidth and communication delay matrix between hardware within the cluster.

[0031] Step 3, Adaptive Rule Generation: Based on the Unified Operator Description Language (UDL) intermediate representation and hardware feature fingerprints, matching and parameterization are performed using a pre-built rule template library to automatically generate optimized adaptation rules for at least one target hardware. The parameterization is based on the specific parameters of the hardware feature fingerprints and the optimization attribute labels carried in the UDL intermediate representation. When an operator in the UDL intermediate representation is marked as communication-aware, the process of generating optimized adaptation rules also includes combining the interconnection bandwidth and communication delay matrix between hardware within the cluster to simultaneously calculate computational and communication overhead in the cost model, thereby generating distributed operator partitioning and scheduling rules that overlap computation and communication.

[0032] Step 4, Multi-backend code generation and deployment: The multi-backend code generator receives the optimized Unified Operator Description Language (UDL) subgraph and the corresponding hardware-specific rules, and compiles the UDL intermediate representation into target code that can be executed on at least one target hardware according to the hardware-specific rules. Specifically, the operators in the UDL intermediate representation are mapped to the native instructions or operator calls corresponding to the target hardware, and PTX / CUDA code is generated for NVIDIA GPUs, or code that calls the CANN computing engine is generated for Huawei Ascend NPUs.

[0033] Please see Figure 1 In step 3, the matching and parameterized filling using a pre-set rule template library also includes the following steps:

[0034] Step 301: Match the corresponding parameterized rule template from the rule template library based on the operator type in the intermediate representation of the Unified Operator Description Language (UDL) and the hardware architecture type in the hardware feature fingerprint.

[0035] Step 302: Based on the memory level parameters and computing unit parameters in the hardware feature fingerprint, and combined with the attribute tags in the intermediate representation of the Unified Operator Description Language (UDL), the undetermined parameters in the parameterized rule template are calculated and filled through the cost model to form candidate adaptation rules.

[0036] Step 303: Use the performance prediction model to evaluate the candidate code generated by the candidate adaptation rules. If the performance is not up to standard, adjust the pending parameters and regenerate the candidate adaptation rules for iterative optimization.

[0037] In this embodiment, the present invention provides a heterogeneous computing power adaptive compilation method for large models, which can be deployed in a single compilation system. The following example illustrates the implementation process of this method in detail using a scenario of deploying a Transformer model with billions of parameters on a cloud computing cluster that simultaneously includes NVIDIA GPUs and Huawei Ascend NPUs.

[0038] Step 1: Model Input and UDL Transformation. The user inputs a pre-trained large model (e.g., a model saved using the PyTorch framework) through the system interface. The system's front-end transformation module parses the model's computation graph and converts it into an intermediate representation based on the Unified Operator Description Language (UDL). For example, a matrix multiplication (MatMul) operator is transformed into a UDL structure containing the operator "MatMul", the shape and data type of the input and output tensors, and attribute labels (e.g., precision: fp16, layout: default, communication_aware: true).

[0039] Step 2: Hardware Feature Fingerprint Extraction. The system hardware awareness module automatically detects available hardware in the current cluster: an NVIDIA H100 GPU and a Huawei Ascend 910 NPU. The module reads the hardware information of both and forms a structured feature fingerprint. For the NVIDIA H100 GPU, the extracted fingerprint information is as follows: json { “arch_type”:“NVIDIA_Hopper”, “compute_units”: 144, “memory_hierarchy”: { “register_size”:256, “shared_memory_size”: “256KB”, “l2_cache_size”: “60MB”}, “special_instructions”: [“TensorCore_fp16”], “bandwidth”: {“GPU_GPU”: “900GB / s”}} For the Ascend 910 NPU, information such as the number of cores in its Da Vinci architecture, on-chip buffer size, and specific instruction set is extracted. Meanwhile, the cluster management module provides network bandwidth and latency matrices between GPUs and NPUs, as well as between multiple GPUs.

[0040] Step 3: Adaptive Rule Generation. This is the core of the invention. The adaptive rule generation engine receives the intermediate UDL representation and hardware feature fingerprint. Taking the MatMul operator marked with `communication_aware: true` as an example:

[0041] Template matching: Based on the operator type "MatMul" and hardware "arch_type", find the initial parameterized templates suitable for "MatMul on NVIDIA_Hopper" and "MatMul on Ascend" from the rule base.

[0042] Parameterized Filling: For the GPU, based on the H100's shared_memory_size (256KB) and TensorCore_fp16 instruction set, and combined with the precision:fp16 tag in the UDL, the cost model calculates the optimal cyclic block size (e.g., BM=128, BN=128, BK=32) and TensorCore usage strategy, and fills it into the template. For the NPU, based on the Ascend's on-chip memory size and compute unit array, different block parameters and instruction scheduling strategies are filled. Network-Aware Coordination: Since the operators are tagged with communication awareness, the engine calls the network-aware optimization submodule. This module evaluates different operator partitioning strategies (e.g., partitioning at the batch dimension or the model dimension) based on the cluster bandwidth latency matrix. The cost model simultaneously calculates the computation time after partitioning and the required cross-card communication time, aiming to select a partitioning scheme that allows communication to be computationally hidden. Finally, a comprehensive adaptation rule containing the specific partitioning dimension and the insertion position of communication primitives is generated.

[0043] Validation and Iteration: After generating the initial rules, a lightweight performance prediction model based on historical data is used for evaluation. If the predicted performance does not reach the threshold (e.g., below 90% of the manually optimized version), a fine-tuning process is triggered, performing a small-scale search on parameters such as block size and updating the rules.

[0044] Step 4: Multi-Backend Code Generation and Deployment. The multi-backend code generator receives the optimized UDL subgraph (which may have been partitioned according to rules) and the corresponding hardware-specific rules. For the GPU portion, it calls the CUDA compiler chain to generate CUDA kernel code (or PTX intermediate code) that utilizes TensorCore and is optimized for chunking and memory loading, based on the rules. For the NPU portion, it generates code that calls the Huawei CANN graph engine API, or directly generates an offline model (OM) file conforming to the Da Vinci architecture. Finally, the system generates an executable deployment package containing code optimized for both GPU and NPU, working collaboratively (processing the partitioned data), thus achieving efficient deployment of large models on heterogeneous clusters.

[0045] Please see Figure 1 The method for implementing the above-mentioned heterogeneous computing power adaptive compilation method for large models includes:

[0046] The front-end conversion module is used for model input and UDL conversion. The user inputs a pre-trained large model through the system interface, and the system's front-end conversion module parses the computation graph of the model and converts it into an intermediate representation based on the Unified Operator Description Language (UDL).

[0047] The hardware awareness module is used for hardware feature fingerprint extraction. The system hardware awareness module automatically detects and extracts the hardware feature fingerprint of at least one target hardware. The hardware feature fingerprint includes the instruction set architecture, memory hierarchy and number of computing units of the target hardware.

[0048] An adaptive rule generation engine is used for adaptive rule generation. Based on the Unified Operator Description Language (UDL) intermediate representation and hardware feature fingerprint, it automatically generates optimized adaptation rules for at least one target hardware by matching and parameterizing the rules through a pre-built rule template library.

[0049] A multi-backend code generator is used for multi-backend code generation and deployment. The generator receives an optimized Unified Operator Description Language (UDL) subgraph and corresponding hardware-specific rules, and compiles the UDL intermediate representation into target code executable on at least one target hardware based on these rules. In this embodiment, a novel hardware-independent UDL layer is designed. This layer decomposes and describes large model computation graphs from different frameworks (such as PyTorch and TensorFlow) into a series of well-defined and semantically clear atomic operators (such as ADD and CONV). This is similar to a "universal computation contract," ensuring the uniformity of upper-layer applications. Unlike existing solutions (such as directly using ONNX or framework-specific IR), UDL is designed for automatic adaptation across heterogeneous hardware. Its syntax and structure contain meta-information that facilitates subsequent automatic analysis of hardware characteristics and network-aware optimization.

[0050] The adaptive compilation engine, the "intelligent brain" of this invention, comprises two key sub-modules: a hardware awareness and automatic rule generation module. This module proactively detects key parameters of the target chip, such as its instruction set architecture, memory hierarchy, and number of computing units. Based on these parameters and a pre-built meta-rule library, the system automatically derives and generates mapping and scheduling rules from UDL operators to the optimal native instructions for that hardware (such as CUDA cores or Ascend CANN operators), replacing the traditional manual coding method. The multi-backend dynamic code generator receives optimized UDL computation graphs and targeted adaptation rules from the adaptive compilation engine, generating high-performance target code for different hardware backends. For example, for NVIDIA GPUs, it calls the CUDA toolchain to generate PTX intermediate code and optimized CUDA kernels; for Huawei Ascend NPUs, it generates code that calls its CANN (Compute Architecture for Neural Networks) graph engine to leverage its hardware-level optimizations such as operator fusion and memory reuse.

[0051] As can be seen from the above, the beneficial effects of this invention are as follows: This invention provides a heterogeneous computing power adaptive compilation method for large models. By designing a unified operator description language (UDL), the model computation graph is decomposed into atomic operators. The system can automatically extract the instruction set and memory level features of the target chip, dynamically generate adaptation rules and optimized code (such as generating PTX for NVIDIA hardware and CANN operators for Huawei Ascend), significantly reducing the workload of manual adaptation. It realizes a leap from static compilation relying on manual rules to intelligent compilation based on hardware features, significantly improving the deployment efficiency and running performance of large models in complex heterogeneous computing power clusters; it greatly improves efficiency, reducing the adaptation work of new hardware from expert work on a monthly basis. The process of streamlining and automatically generating large models through toolchains significantly accelerates the technology's implementation; it lowers the technical threshold and reduces reliance on scarce hardware expert experience, enabling ordinary developers to easily deploy large models on various heterogeneous chips, thus improving the technology's versatility; it enhances system scalability by providing a standardized automatic access paradigm for new AI acceleration chips, helping to break down hardware ecosystem barriers and promote the formation of an open heterogeneous computing environment; and it achieves performance optimization by automatically combining specific hardware features and optimization intentions during compilation to generate deeply optimized target code, ensuring the performance of large models on heterogeneous platforms and solving the problem of existing technologies where hardware adaptation heavily relies on manually written scheduling templates and optimization rules.

[0052] The specific embodiments described above are preferred embodiments of the present invention and are not intended to limit the specific scope of the present invention. The scope of the present invention includes, but is not limited to, these specific embodiments. All equivalent changes made in accordance with the present invention are within the protection scope of the present invention.

Claims

1. A heterogeneous computing power adaptive compilation method for large models, comprising the following steps: Step 1, Model Input and UDL Conversion: The user inputs a pre-trained large model through the system interface. The system's front-end conversion module parses the computation graph of the model and converts it into an intermediate representation based on the Unified Operator Description Language (UDL). The Unified Operator Description Language (UDL) is a declarative language. The data structure used to describe atomic operators includes at least operators, input and output tensor descriptions, and attribute label fields to carry optimization intentions. The attribute label fields are used to provide optimization constraints and suggestions for parameterization in Step 3. Step 2, hardware feature fingerprint extraction: The system hardware perception module automatically detects and extracts the hardware feature fingerprint of at least one target hardware. The hardware feature fingerprint includes the instruction set architecture, memory hierarchy, and number of computing units of the target hardware. Step 3, Adaptive rule generation, based on the Unified Operator Description Language (UDL) intermediate representation and hardware feature fingerprints, automatically generates optimized adaptation rules for at least one target hardware through matching and parameterized filling using a pre-built rule template library; wherein, The basis for parameterized filling includes the specific parameters of the hardware feature fingerprint and the optimization attribute labels carried in the Unified Operator Description Language (UDL) intermediate representation. When the operators in the UDL intermediate representation are marked as communication-aware, the process of generating optimization adaptation rules also includes combining the interconnection bandwidth and communication delay matrix between hardware in the cluster, and synchronously calculating the computational overhead and communication overhead in the cost model to generate distributed operator partitioning and scheduling rules that overlap computation and communication. Step 4, Multi-backend code generation and deployment: The multi-backend code generator receives the optimized Unified Operator Description Language (UDL) subgraph and the corresponding hardware-specific rules, and compiles the UDL intermediate representation into target code that can be executed on at least one target hardware according to the hardware-specific rules. In step 3, the matching and parameterized filling using a pre-set rule template library also includes the following steps: Step 301: Match the corresponding parameterized rule template from the rule template library based on the operator type in the intermediate representation of the Unified Operator Description Language (UDL) and the hardware architecture type in the hardware feature fingerprint. Step 302: Based on the memory level parameters and computing unit parameters in the hardware feature fingerprint, and combined with the attribute tags in the intermediate representation of the Unified Operator Description Language (UDL), the undetermined parameters in the parameterized rule template are calculated and filled through the cost model to form candidate adaptation rules. Step 303: Use the performance prediction model to evaluate the candidate code generated by the candidate adaptation rules. If the performance is not up to standard, adjust the pending parameters and regenerate the candidate adaptation rules for iterative optimization.

2. The heterogeneous computing power adaptive compilation method for large models as described in claim 1, characterized in that: In step 1, the attribute label field includes at least one of the following: a layout label for indicating data arrangement format, a precision label for indicating calculation precision, a block label for indicating cyclic block preference, and a communication awareness label for triggering network-aware optimization.

3. The heterogeneous computing power adaptive compilation method for large models as described in claim 2, characterized in that: In step 2, the hardware feature fingerprint of the target hardware also includes the target hardware's special instruction set information and bandwidth information. When there are multiple target hardware devices that form a heterogeneous cluster, the hardware feature fingerprint further includes the interconnection bandwidth and communication delay matrix between hardware devices within the cluster.

4. The heterogeneous computing power adaptive compilation method for large models as described in claim 3, characterized in that: In step 4, the operators in the Unified Operator Description Language (UDL) intermediate representation are mapped to the native instructions or operator calls corresponding to the target hardware, and PTX / CUDA code is generated for NVIDIA GPUs, or code that calls the CANN computing engine is generated for Huawei Ascend NPUs.

5. A heterogeneous computing power adaptive compilation system for large models, used to implement the heterogeneous computing power adaptive compilation method for large models as described in any one of claims 1-4, characterized in that, include: The front-end conversion module is used for model input and UDL conversion. The user inputs a pre-trained large model through the system interface, and the system's front-end conversion module parses the computation graph of the model and converts it into an intermediate representation based on the Unified Operator Description Language (UDL). The hardware awareness module is used for hardware feature fingerprint extraction. The system hardware awareness module automatically detects and extracts the hardware feature fingerprint of at least one target hardware. The hardware feature fingerprint includes the instruction set architecture, memory hierarchy and number of computing units of the target hardware. An adaptive rule generation engine is used for adaptive rule generation. Based on the Unified Operator Description Language (UDL) intermediate representation and hardware feature fingerprint, it automatically generates optimized adaptation rules for at least one target hardware by matching and parameterizing the rules through a pre-built rule template library. The multi-backend code generator is used for multi-backend code generation and deployment. The multi-backend code generator receives the optimized Unified Operator Description Language (UDL) subgraph and the corresponding hardware-specific rules, and compiles the UDL intermediate representation into target code that can be executed on at least one target hardware according to the hardware-specific rules.

Images