Programming framework adaptation method, device, storage medium and program product

Abstract

Embodiments of the present disclosure provide a programming framework adaptation method and device, a storage medium and a program product, relating to the technical field of chip programming. The method comprises: obtaining first programming framework corresponding to a first chip and target chip information of a second chip; the first programming framework comprises an initial compiler and first programming framework data; performing adaptation processing on the initial compiler according to the target chip information to obtain a target compiler meeting a first preset condition; performing adaptation processing on a second type of kernel function in the first programming framework data based on the target compiler and the target chip information to obtain a candidate kernel function set meeting a second preset condition; adjusting the kernel functions in the candidate kernel function set based on a preset kernel function optimization target to generate a target kernel function set meeting the preset kernel function optimization target; and generating a second programming framework of the second chip according to the target compiler and the target kernel function set. Thus, the efficiency and reliability of programming framework adaptation are improved.

Description

Technical Field

[0001] This disclosure relates to the field of chip programming technology, and more specifically, to a programming framework adaptation method, device, storage medium, and program product. Background Technology

[0002] Graphics Processing Units (GPUs) and other chips have become core hardware in complex scenarios such as Artificial Intelligence (AI) and scientific computing. Programming frameworks for GPUs are typically deeply coupled with specific GPU architectures. When a new GPU architecture requires the use of an existing programming framework, adaptation and migration are necessary. However, the efficiency and reliability of programming framework adaptation in related technologies are relatively low. Summary of the Invention

[0003] One objective of this disclosure is to provide a new technical solution regarding programming framework adaptation.

[0004] According to a first aspect of the present disclosure, a programming framework adaptation method is provided, the method comprising: Obtain the first programming framework corresponding to the first chip and the target chip information of the second chip; wherein the chip architecture of the second chip and the first chip are different, the first programming framework includes an initial compiler adapted to the first chip and first programming framework data, and the first programming framework data includes a first type kernel function and a second type kernel function; The initial compiler is adapted based on the target chip information to obtain a target compiler that meets the first preset conditions; wherein, the first preset conditions indicate that the target compiler compiles the first type of kernel function to obtain an executable file for the second chip; Based on the target compiler and the target chip information, the second type of kernel functions in the first programming framework data are adapted to obtain a set of candidate kernel functions that meet the second preset conditions; wherein, the second preset conditions indicate that the kernel functions in the candidate kernel function set pass the test after executing the preset kernel function test cases based on the second chip; Based on a preset kernel function optimization objective, the kernel functions in the candidate kernel function set are adjusted to generate a target kernel function set that meets the preset kernel function optimization objective; Based on the target compiler and the target kernel function set, a second programming framework corresponding to the second chip is generated.

[0005] Optionally, the complexity of the second type of kernel function is higher than that of the first type of kernel function; wherein: The first type of kernel function includes atomic kernel functions, which do not call other kernel functions; The second type of kernel function includes one or more of the following: element-wise kernel function, reduction kernel function, normalized exponential transformation kernel function, and matrix multiplication kernel function.

[0006] Optionally, the preset kernel function test cases include multiple test case sets executed in a preset order, with the later test case set being executed if the earlier test case set passes the test.

[0007] Optionally, the plurality of test case sets includes a set of first to fifth test cases executed in a preset order, wherein: The first set of test cases includes smoke test cases used to verify the correctness of compilation and numerical values; The second set of test cases includes shape scan test cases used to verify the correctness of generalization adaptation under multi-scale input. The third set of test cases includes numerical stability test cases used to verify numerical robustness under extreme input scenarios; The fourth set of test cases includes deterministic test cases used to verify the reproducibility of the calculation results in multiple tests; The fifth set of test cases includes boundary condition test cases used to verify the correctness of non-standard input processing.

[0008] Optionally, the kernel function optimization objective includes ensuring that the measured throughput of the kernel functions in the target kernel function set on the second chip is greater than a preset throughput threshold, wherein the preset throughput threshold is a throughput threshold set based on the theoretical peak computing power of the second chip.

[0009] Optionally, generating a second programming framework corresponding to the second chip based on the target compiler and the target kernel function set includes: A target model is generated based on the target compiler and the target kernel function set, and the target model is then inferred and verified. If the target model inference verification passes, the second programming framework is generated based on the target compiler and the target kernel function set.

[0010] Optionally, generating a second programming framework corresponding to the second chip based on the target compiler and the target kernel function set further includes: If the target model inference verification fails, the target compiler and / or the target kernel function set are adjusted according to the reason for the failure. A new target model is generated based on the adjusted target compiler and / or target kernel function set, and inference verification is performed on the new target model until the target model inference verification passes.

[0011] Optionally, generating the second programming framework based on the target compiler and the target kernel function set includes: Regression testing was performed based on the target compiler and the target kernel function set. If the regression test passes, the second programming framework is generated based on the target compiler and the target kernel function set.

[0012] Optionally, the first programming framework data further includes multiple initial operators implemented based on the kernel functions of the first chip; the method further includes: When adapting the second type of kernel function or generating a second programming framework corresponding to the second chip, if there are missing operators on the second chip, the missing operators are adapted based on a preset operator adaptation strategy to obtain a target operator corresponding to the second chip. The missing operators include operators that cannot be executed or operators that do not exist among multiple initial operators. The preset operator adaptation strategy includes multiple candidate adaptation strategies sorted by intrusion level from low to high. The candidate adaptation strategies with low intrusion level are used for adaptation processing with priority over the candidate adaptation strategies with high intrusion level.

[0013] Optionally, the preset operator adaptation strategy includes first to fourth adaptation strategies sorted by intrusion level from low to high, wherein: The first adaptation strategy is to adapt operators based on domain-specific language text; The second adaptation strategy is to adapt operators based on compiler-stage extensions; The third adaptation strategy is to adapt operators based on compiler language extensions; The fourth adaptation strategy is to adapt operators based on the native programming model of the second chip.

[0014] Optionally, the programming framework adaptation method is triggered by a user and / or an intelligent agent to execute or exit; the method further includes: During the execution of the programming framework adaptation method, if an exit instruction is received from the user and / or the agent to exit the programming framework adaptation method, an archive file corresponding to the programming framework adaptation method is generated; the archive file includes a progress file for recording the programming framework adaptation progress and / or a log file for recording programming framework adaptation process data. If a start instruction is received from the user and / or the agent to trigger the execution of the programming framework adaptation method and an archive file corresponding to the programming framework adaptation method is obtained, the programming framework adaptation progress is restored based on the archive file, and the programming framework adaptation method is continued to be executed based on the restored programming framework adaptation progress.

[0015] According to a second aspect of the present disclosure, an electronic device is provided, including a memory and a processor, the memory being configured to store computer instructions, and the processor being configured to invoke the computer instructions from the memory to perform the method described in the first aspect.

[0016] According to a third aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, the computer program implementing the method described in the first aspect when executed by a processor.

[0017] According to a fourth aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0018] Based on the programming framework adaptation method provided in this disclosure, the programming framework adaptation process is divided into several orderly stages such as compiler adaptation, kernel function migration, performance optimization, and framework generation. Clear quality gate conditions are set for each stage, which enables subsequent stages to be built on the basis that the preceding stages have been verified to meet the corresponding conditions or goals, thereby improving the efficiency and reliability of programming framework adaptation.

[0019] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0021] Figure 1 This is a schematic diagram of an application scenario that can be used to implement the programming framework adaptation method of the embodiments of this disclosure.

[0022] Figure 2 This is a flowchart illustrating a programming framework adaptation method provided in an embodiment of this disclosure.

[0023] Figure 3 This is a flowchart illustrating another programming framework adaptation method provided in this embodiment.

[0024] Figure 4This is a schematic diagram of the architecture of a programming framework adaptation system provided in an embodiment of this disclosure.

[0025] Figure 5 This is a flowchart illustrating another programming framework adaptation method provided in this embodiment.

[0026] Figure 6 This is a schematic diagram of the interaction process of a programming framework adaptation method provided in an embodiment of this disclosure.

[0027] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0028] Various exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0029] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit this disclosure or its application or use. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus may be considered part of the specification.

[0030] It should be noted that similar labels and letters in the following figures indicate similar items, so once an item is defined in one figure, it need not be discussed further in subsequent figures.

[0031] This disclosure relates to a technical solution for programming framework adaptation. Figure 1 This is a schematic diagram illustrating an application scenario where the technical solutions provided in the embodiments of this disclosure can be applied. For example... Figure 1 As shown, this application scenario may include a programming platform 100, an operation terminal 200, and a target chip 300.

[0032] The programming platform 100 can be used for programming framework adaptation, for example, it can execute the programming framework adaptation method provided in the embodiments of this disclosure. The programming framework adaptation method can be executed by an AI agent and / or human engineer deployed on the programming platform to adapt and migrate a first programming framework corresponding to a first chip to a second programming framework corresponding to a second chip. The first chip and the second chip can have different chip architectures. Figure 1The target chip 300 shown can be a target chip for programming framework adaptation (e.g., a second chip). The target chip 300 can be any chip, such as a Graphics Processing Unit (GPU), General-Purpose Graphics Processing Unit (GPGPU), Neural-network Processing Unit (NPU), Tensor Processing Unit (TPU), or Central Processing Unit (CPU). In some examples, the programming platform may include a processor, on which the programming framework adaptation method is executed. The processor may include at least one of a CPU, GPU, GPGPU, NPU, and TPU. Exemplarily, the programming platform may also include memory, which may include at least one of Static Random Access Memory (SRAM), Read-Only Memory (ROM), or Erasable Programmable Read-Only Memory (EPROM). The memory can be used to store a computer program that, when executed by the processor, controls the processor to execute the programming framework adaptation method of this disclosure embodiment.

[0033] In some examples, the programming platform can provide a programming framework for a target chip. This framework can be a Domain-Specific Language (DSL) programming framework corresponding to a specific type of chip architecture design. For example, the first programming framework for a first chip could be a Triton or Tensor Virtual Machine (TVM) programming framework, and the second programming framework for a second chip can be an adaptation of the first programming framework to support programming for the second chip. This programming framework can be a software system deployed on a computer, providing a compiler and programming framework data. This programming framework data can include a set of kernel functions (also called a kernel function library) and / or a set of operators (also called an operator library). The kernel function set can include various types of kernel functions, and the operators in the operator set can be implemented based on kernel functions. This programming framework provides a standardized programming interface and a rich library of pre-built operators to support the construction and execution of deep learning or high-performance computing tasks. The framework can transform upper-level algorithm logic into corresponding computational instructions that the target chip can recognize through the underlying Hardware Abstraction Layer (HAL), thereby enabling the scheduling, management and optimization of the target chip's computing resources.

[0034] The operating terminal 200 and the programming platform 100 can establish a connection through an interface device or a communication device. The user can transmit programming framework-related files or other information (such as operation instructions) to the programming platform via the operating terminal. The programming platform can read the user-inputted files or other information and execute the programming framework adaptation method of this embodiment. The user can also manage the execution process of the programming framework adaptation method on the programming platform through the operating terminal; for example, the user can start, pause, stop, or exit the execution process of the programming framework adaptation method.

[0035] exist Figure 1 In the application scenario shown, the programming platform 100, the operation terminal 200, and the target chip 300 are illustrated as different physical devices. It should be noted that the programming platform 100 and the operation terminal 200 can be integrated into the same physical device, and the target chip 300 and the programming platform 100 can also be integrated into the same physical device; this disclosure does not limit this aspect.

[0036] based on Figure 1The application scenario illustrated illustrates that when a new GPU architecture (e.g., a second chip) needs to use an existing programming framework (e.g., the first programming framework of the first chip), adaptation and migration of the programming framework are required to obtain a second programming framework adapted to the new GPU architecture. For example, mainstream AI programming frameworks (such as Triton, TVM, and other language-specific programming frameworks) are deeply coupled with specific GPU architectures. When a new GPU architecture (e.g., a domestically produced GPGPU) needs to access these frameworks, it faces the challenge of full-stack adaptation, including compiler backend adaptation, kernel function migration, and performance tuning. Related technologies include compiler backend rewriting, IR cross-architecture redirection, compile-time execution mode selection, and automatic operator tuning. While these methods can solve single-point problems such as compiler intermediate representation redirection and automatic operator tuning, they result in a significant amount of ineffective work and confusion during the programming framework adaptation process, affecting the efficiency and reliability of the adaptation.

[0037] Furthermore, the programming framework adaptation solutions in related technologies also suffer from the following problems: a lack of a systematic methodology covering the entire process from compiler adaptation and kernel function migration to performance optimization; engineering teams often lack clear phase divisions and quality standards for each phase during actual migration; related technologies attempt to optimize performance before kernel functions are running correctly, or attempt to migrate complex operators when the compiler backend is incomplete, leading to a confusion between migration and optimization, resulting in a large amount of ineffective work and chaotic dependencies; in handling operator gaps, related technologies either implement all solutions by modifying the compiler or bypass them entirely through external kernel functions, resulting in the loss of the compilation optimization advantages of domain-specific language frameworks and excessively high development costs; the correctness verification of related technologies usually only checks the numerical correctness under fixed shapes, lacking a hierarchical verification mechanism, causing correctness issues to be exposed only in the later stages of the project, significantly increasing repair costs; since programming framework migration is usually a long-term project lasting several weeks or even months, human engineers or agents in related technologies find it difficult to determine the current stage, the range of kernel functions that have been migrated, and the list of operators with remaining gaps after the project is interrupted, affecting the continuity and manageability of the migration work.

[0038] This disclosure provides a new technical solution for programming framework adaptation. By dividing the programming framework adaptation process into several orderly stages such as compiler adaptation, kernel function migration, performance optimization, and framework generation, and setting clear quality gating conditions for each stage, subsequent stages can be built on the basis that the preceding stages have verified to meet the corresponding conditions or goals, thereby solving all or part of the above problems.

[0039] The following combination Figure 1 The illustrated application environment illustrates the specific implementation of the embodiments of this disclosure.

[0040] Figure 2 This is a flowchart illustrating a programming framework adaptation method provided in an embodiment of this disclosure. This programming framework adaptation method can be... Figure 1 The programming platform shown can be used for execution, or it can be executed by any electronic device. For example... Figure 2 As shown, the programming framework adaptation method of this embodiment may include the following steps S210 to S250.

[0041] Step S210: Obtain the first programming framework corresponding to the first chip and the target chip information of the second chip.

[0042] The second chip and the first chip can have different chip architectures. This chip architecture can include the underlying logic blueprint and core design specifications for chip design and operation. For example, it can include information such as the chip's instruction set architecture (ISA) and microarchitecture. The instruction set architecture serves as a contract between the chip's hardware and software, determining the basic set of machine instructions (such as x86, ARM, RISC-V) that the chip can recognize and execute. The microarchitecture can include aspects such as pipeline depth, parallel execution strategies, hierarchical organization of multi-level caches, and memory models. Based on this chip architecture, the methods by which the chip "processes information" and "works collaboratively" can be determined.

[0043] The programming framework in this embodiment may include a compiler and programming framework data. The compiler can compile the source code developed by the user or agent based on the programming framework data to obtain a chip-executable file (e.g., a binary file). The source code developed by the user or agent may be source code based on a domain-specific language, and the programming framework may be a domain-specific language-based programming framework. The programming framework data may include framework source code or other programming aids, such as domain-specific language (DSL) framework source code, which may include various kernel functions and / or operators. For example, the programming framework data may include a set of kernel functions (also called a kernel function library) and / or a set of operators (also called an operator library), etc. The kernel function set may include multiple types of kernel functions, and the operators in the operator set may be implemented based on kernel functions.

[0044] In some examples, the first programming framework may include an initial compiler and first programming framework data adapted to the first chip, and the second programming framework may include a target compiler and second programming framework data adapted to the second chip. The chip architecture of the first chip can be a currently widely used chip architecture, and the first programming framework corresponding to this first chip can be a language-specific programming framework such as Triton or TVM. The chip architecture of the second chip can be a newly developed chip architecture, requiring adaptation and migration of the first programming framework to obtain a second programming framework corresponding to the second chip.

[0045] In some examples, the first programming framework data may include a first type of kernel function and a second type of kernel function.

[0046] For example, the complexity of the second type of kernel function can be higher than that of the first type of kernel function.

[0047] For example, this first type of kernel function may include atomic kernel functions that do not call other kernel functions. This first type of kernel function may also be called a simple kernel function, and may include, for example, vector addition functions, vector subtraction functions, and other atomic kernel functions that do not call other kernel functions.

[0048] For example, the second type of kernel function may include one or more of the following: element-wise kernel function, reduction kernel function, normalized exponential transformation (Softmax) kernel function, and matrix multiplication kernel function. This second type of kernel function may also be called a core kernel function. Exemplarily, the second type of kernel function may include: a kernel function constructed by calling a first type of kernel function, or a kernel function that, although not calling a first type of kernel function, has a higher complexity than a first type of kernel function.

[0049] It should be noted that the kernel functions included in the first programming framework data may also include other kernel functions besides the first type and the second type. For example, the first type of kernel function is a simple kernel function (or an atomic kernel function), the second type of kernel function is a core kernel function, and other kernel functions (such as non-core peripheral kernel functions) may be called third type kernel functions. For example, the first programming framework may include an initial set of kernel functions for implementing the operators supported by the first chip, and this initial set of kernel functions may include one or more of the first type, second type, and third type kernel functions mentioned above.

[0050] For example, the second type of kernel function can be a kernel function that is adapted after the first type of kernel function has been adapted. In this way, by dividing kernel functions into two categories according to their complexity and determining the order of adaptation, it is possible to first complete the compilation verification of simple kernel functions after the compiler backend has been verified to be usable, and then gradually promote the migration and adaptation of core kernel functions. This helps to reduce the risk of dependency confusion during the migration process and improve the orderliness and reliability of the adaptation.

[0051] In some examples, the target chip information may include one or more of the following: architecture characteristics, instruction set information, memory hierarchy information, etc., of the second chip. The second chip can be any chip with a different architecture than the first chip, such as a domestically produced general-purpose graphics processing unit (GPGPU). This target chip information can be obtained by reading the hardware specification document of the second chip, querying the driver interface of the second chip, or directly detecting the hardware capabilities of the second chip; this embodiment does not limit this method. Optionally, the target chip information may also include the hardware environment information of the second chip, which can be used for programming framework adaptation and testing.

[0052] In this way, by obtaining the first programming framework and target chip information, the necessary input data can be provided for subsequent compiler adaptation and kernel function migration, which helps to improve the relevance and accuracy of programming framework adaptation.

[0053] Step S220: Adapt the initial compiler according to the target chip information to obtain a target compiler that meets the first preset conditions.

[0054] The first preset condition can be used to characterize the executable file of the second chip obtained after the target compiler compiles the first type of kernel function.

[0055] This step S220 can also be called the compiler basic adaptation step or the compiler basic adaptation phase.

[0056] For example, adapting the initial compiler can involve adapting the compiler backend. The compiler backend is a core component of the compiler, used to convert the hardware-independent intermediate representation (IR) generated by the frontend into machine code that the target chip (such as a GPU) can directly execute. This can include low-level adaptation work such as instruction selection, register allocation, memory access optimization, and thread scheduling tailored to the target architecture, serving as a crucial bridge between high-level code and hardware execution. In cross-architecture migration, by adjusting the backend logic (such as modifying the downscaling channel of the MLIR dialect layer and adapting to the instruction set and memory model of the new GPU), the initial compiler for the first chip can be adapted to obtain the target compiler for the second chip. Based on this target compiler, the source code can be compiled to generate executable files (e.g., binary files) for the second chip.

[0057] In some examples, a compiler backend rewriting approach can be used to adapt the initial compiler, starting from the Multi-Level Intermediate Representation (MLIR) dialect layer, to reimplement a lowering pass for the second-chip architecture. For instance, at the MLIR dialect layer, a memory operation lowering pass is reimplemented, reducing high-level memory operations to operations corresponding to the second-chip instruction set; a thread mapping pass is reimplemented, mapping parallel loop structures to the thread hierarchy of the second chip; synchronization primitive adaptation is reimplemented, mapping the synchronization semantics of the source architecture to the corresponding synchronization instructions of the second chip; the register allocation strategy is adjusted according to the register file specifications of the second chip; and finally, the second-chip driver interface is interfaced, enabling the generated binary file to be correctly submitted and executed through the second-chip driver's loading and execution interface. The high-level memory operations can include shared memory allocation, global memory access, etc., and the thread hierarchy can include thread blocks, thread bundles, threads, etc. In this way, the lowering pass rewriting approach can deeply adapt to the characteristics of the second-chip architecture, resulting in higher code generation quality, greater performance potential, and suitability for scenarios where the first and second chip architectures differ significantly.

[0058] In other examples, an architecture-independent intermediate representation (IR) cross-architecture redirection method can be used to adapt the initial compiler. For instance, the initial compiler's compilation pipeline can be reused to compile domain-specific language code to an architecture-independent low-level virtual machine intermediate representation (LLVM IR) layer. Then, a multi-level intermediate representation text transformation pipeline converts this low-level virtual machine intermediate representation into an equivalent intermediate representation compilable by the second chip. For example, this text transformation may include replacing first-chip-specific built-in functions with equivalent calls corresponding to the second chip, and mapping address space annotations of the first chip to corresponding address space identifiers of the second chip. The first-chip-specific built-in functions may include thread synchronization primitives, thread indexing functions, atomic operation functions, etc., and the address space annotations may include shared memory identifiers, global memory identifiers, etc. After the text transformation is completed, the transformed intermediate representation is sent to the compilation backend of the second chip to generate an executable binary file for the second chip, thereby satisfying the first preset condition. In this way, executable code can be generated quickly through intermediate representation redirection, which has a low implementation cost and is suitable for scenarios where the differences between the instruction sets of the first chip and the second chip are relatively controllable.

[0059] In this embodiment, the IR cross-architecture redirection method can be used first to quickly verify the basic feasibility of the compiler backend. If the IR cross-architecture redirection method cannot produce an executable binary file for the second chip, the compiler backend rewriting method is switched to gradually reduce the number of channels until an executable binary file is produced.

[0060] Step S230: Based on the target compiler and target chip information, the second type of kernel function in the first programming framework data is adapted to obtain a set of candidate kernel functions that meet the second preset conditions.

[0061] The second preset condition can be used to characterize the test result of the kernel function in the candidate kernel function set after executing the preset kernel function test case based on the second chip as passing.

[0062] This step S230 can also be called the core kernel function adaptation step or the core kernel function adaptation stage, and the core kernel function may include the second type of kernel function.

[0063] In some examples, the adaptation process for the second type of kernel function may include: selecting an execution mode on the second chip for each second type of kernel function; performing domain-specific language adaptation on the kernel function based on the selected execution mode; executing preset kernel function test cases on the adapted kernel function to verify its correctness on the second chip, thereby obtaining a set of candidate kernel functions that meet the second preset conditions.

[0064] In some examples, the preset kernel function test cases include a set of multiple test cases to be executed in a preset order, and the later test cases can be executed if the earlier test cases pass the test.

[0065] In this way, by executing test cases in a hierarchical and orderly manner, basic correctness issues can be quickly identified in the early stages. This avoids wasting resources on executing subsequent complex tests before the basic issues are resolved, which helps improve the efficiency of correctness verification and reduce the cost of fixing problems that are exposed later.

[0066] For example, the plurality of test case sets includes a first to a fifth set of test cases executed in a preset order. The first to fifth test case sets are described below.

[0067] The first set of test cases may include smoke test cases for verifying compilation and numerical correctness. For example, the smoke test case may use a minimum input shape to verify that the kernel function can successfully compile and produce substantially correct numerical results. The minimum input shape may be a one-dimensional array of vector length 128, or a two-dimensional array of matrix size 64×64. For example, the detection targets of the smoke test case may include: minimum shape and substantially correct numerical results. The timeout for the smoke test case may be set to 10 seconds; if the test cannot be completed within 10 seconds, it is considered a test failure.

[0068] The second set of test cases may include shape scan test cases for verifying the correctness of generalization adaptation under multi-scale inputs. For example, the shape scan test cases can use input shapes of different scales (small, medium, large, etc.) to verify the kernel function's generalization ability to different input sizes. For instance, for a matrix multiplication kernel function, a small-scale input could be a 128×128 matrix, a medium-scale input could be a 1024×1024 matrix, and a large-scale input could be a 4096×4096 matrix. By verifying the correctness of the kernel function at different scales, the kernel function can be correctly executed when faced with inputs of different sizes in practical applications. For example, the detection targets of this shape scan test case may include: multi-scale (small / medium / large), shape generalization. The timeout duration of this shape scan test case can be set to 60 seconds; if the test cannot be completed within 60 seconds, it is considered a test failure.

[0069] The third set of test cases may include numerical stability test cases for verifying numerical robustness under extreme input scenarios. For example, these numerical stability test cases can execute kernel functions based on extreme inputs such as zero, maximum, minimum, mixed sign, or denormalized numbers to verify numerical robustness under extreme input scenarios. For instance, for a normalized exponential transform kernel function, an all-zero vector, a vector containing the maximum floating-point value, a vector containing the minimum floating-point value, or a vector containing mixed positive and negative values ​​can be used as input to verify whether the kernel function produces overflow, underflow, or Not a Number (NaN) results under these extreme conditions. For example, the detection targets of these numerical stability test cases may include: zero, maximum / minimum, mixed sign, and denormalization. The timeout for these numerical stability test cases can be set to 60 seconds; if the test cannot be completed within 60 seconds, it is considered a test failure.

[0070] The fourth set of test cases may include deterministic test cases for verifying the reproducibility of the calculation results across multiple tests. For example, these deterministic test cases may repeatedly run the kernel function to verify the reproducibility of the calculation results. Reproducibility can be judged based on a preset numerical tolerance standard. If the error of multiple calculation results is within the preset numerical tolerance standard, then the reproducibility standard is met, i.e., the deterministic test passes. This numerical tolerance standard may include relative tolerance (rtol) and absolute tolerance (atol). Different types of data correspond to different tolerance standards. For example, for 32-bit floating-point (FP32) data types, the relative tolerance and absolute tolerance can be set to 1e-5, achieving standard precision; for 16-bit floating-point (FP16) data types, the relative tolerance and absolute tolerance can be set to 1e-3, achieving half precision; for 16-bit brain floating-point (BF16) data types, the relative tolerance and absolute tolerance can be set to 1e-3, achieving 1e-2; for 8-bit integer (INT8) data types, the relative tolerance and absolute tolerance can be set to 0, achieving exact matching; for matrix multiplication accumulation data types, the relative tolerance and absolute tolerance can be set to 1e-2. Through deterministic verification, the kernel function can produce consistent outputs under the same input, avoiding instability caused by differences in the order of floating-point operations. For example, the detection objective of this deterministic test case may include: consistency of results across 5 runs. The timeout for this deterministic test case can be set to 120 seconds; if the test cannot be completed within 120 seconds, it is considered a test failure.

[0071] The fifth set of test cases may include boundary condition test cases for verifying the correctness of non-standard input processing. For example, these boundary condition test cases can execute a kernel function with empty-dimensional input, non-contiguous memory layout input, broadcast input, or non-power-2 size input to verify the correctness of non-standard input processing. For instance, an empty-dimensional input can be a matrix of shape (0, 128); a non-contiguous memory layout input can be a matrix after transpose, where its memory layout is no longer contiguous; a broadcast input can be automatic dimension expansion when performing element-wise operations between a one-dimensional vector and a two-dimensional matrix; a non-power-2 size input can be a matrix of shape 1000×1000, rather than the more common 1024×1024. Boundary condition verification ensures that the kernel function will not crash or produce incorrect results when processing various non-standard inputs. For example, the detection targets of these boundary condition test cases may include: empty dimension, non-contiguous, broadcast, non-power-2, etc. The timeout for these boundary condition test cases can be set to 60 seconds; if the test cannot be completed within 60 seconds, it is considered a test failure.

[0072] The aforementioned pre-defined kernel function test cases can also be called five-level hierarchical correctness verification. By covering five different types of defects—compilation correctness, shape generalization, numerical stability, determinism, and boundary conditions—the system can capture different types of correctness issues at different stages. It detects kernel function correctness issues progressively according to the severity of the defects. The earlier the problem is discovered, the lower the cost of fixing it. This helps to discover and fix defects at an earlier stage, reduces the probability of problems being exposed in the later integration testing stage, and improves the traceability of kernel function migration quality.

[0073] In some examples, performance tests can be performed only after all five levels of validation have passed. This avoids wasting performance benchmarking resources on incorrect kernel functions and improves the effectiveness of performance tuning efforts.

[0074] In other examples, the adaptation of second-type kernel functions can also be carried out in the following way: For each second-type kernel function, firstly, the optimal execution mode is selected for it by statically analyzing its characteristics (such as memory access patterns, computational density, data dependencies, etc.); then, based on the selected execution mode, the target compiler is called to compile and adapt the kernel function; if, during the adaptation process, it is found that an operator that a kernel function depends on is missing or cannot be executed on the second chip, a preset operator adaptation strategy can be triggered to adapt the operator, obtaining the target operator corresponding to the second chip, and then the kernel function is re-adapted based on the target operator; finally, the preset kernel function test cases are executed on the adapted kernel function, and after verification, the kernel function is added to the candidate kernel function set. In this way, by systematically adapting and hierarchically verifying the correctness of second-type kernel functions, compatibility issues of kernel functions on the second chip can be effectively identified and resolved, which helps to improve the correctness and completeness of the candidate kernel function set.

[0075] Step S240: Adjust the kernel functions in the candidate kernel function set based on the preset kernel function optimization objective to generate a target kernel function set that meets the preset kernel function optimization objective.

[0076] This step S240 can also be called the performance optimization step or the performance optimization phase.

[0077] In some examples, adjusting kernel functions within a candidate kernel function set can include one or more hardware-aware tuning techniques such as block size search, thread configuration optimization, or memory access pattern adjustment. For instance, block size search can find the optimal parameter combination for performance on the second chip by traversing different block parameters; thread configuration optimization can fully utilize the computing units of the second chip by adjusting the number of threads in each thread block and the grid layout of the thread blocks; memory access pattern adjustment can reduce memory access latency and improve data throughput by optimizing shared memory usage strategies and adjusting memory access merging.

[0078] In some examples, the kernel function optimization objective may include ensuring that the measured throughput of kernel functions in the target kernel function set on the second chip exceeds a preset throughput threshold. This preset throughput threshold can be any pre-defined value.

[0079] For example, the preset throughput threshold can be a throughput threshold set based on the theoretical peak computing power of the second chip. This theoretical peak computing power can be calculated based on the hardware specifications of the second chip, such as core frequency, number of computing units, and number of floating-point operations per cycle. In some examples, the preset throughput threshold can be set as a preset percentage (e.g., 60%) of the theoretical peak computing power of the second chip. The measured throughput can be obtained by performing benchmark tests on the optimized kernel function. This benchmark test can include running the kernel function multiple times on the second chip, recording the execution time of each run, and calculating the throughput based on the computational load of the kernel function. In some examples, experimental logs can be used to record data such as throughput, latency, and peak computing power utilization for each optimization experiment. Experimental logs support data tracking during the optimization process, facilitating the analysis of the effects of different optimization strategies.

[0080] In this way, by linking the performance gating conditions to the theoretical peak computing power of the target chip, the performance of the kernel function on the target chip can be objectively quantified, avoiding the need to proceed to subsequent steps when the performance is not at an acceptable level. This can improve the efficiency of programming framework adaptation and the reliability of the final generated second programming framework.

[0081] Step S250: Generate a second programming framework corresponding to the second chip based on the target compiler and the target kernel function set.

[0082] For example, the second programming framework may include a software system supporting programming to the second chip. For instance, the second programming framework may include a target compiler and second programming framework data, which may include a set of target kernel functions, a set of target operators, and / or programming interface documentation, etc., which may be generated based on the set of target kernel functions. Users or AI agents can use this second programming framework to build and execute deep learning or high-performance computing tasks on the second chip.

[0083] In some examples, the process of generating a second programming framework may include integrating and packaging the target compiler and the target kernel function set to generate a programming framework software system (i.e., the second programming framework) that supports programming for the second chip.

[0084] It should be noted that the execution order of steps S220 to S250 in this embodiment can adopt a strict gating condition sequential execution strategy. That is, the subsequent steps will only be executed after the output result of the previous step meets the preset conditions (i.e., the gating conditions). Otherwise, if the output result of a certain step does not meet the preset conditions, the output result of that step needs to be further adjusted or optimized until the preset conditions are met. For example, the conditions for executing step S230 may include that the target compiler output in step S220 meets the first preset condition, that is, the target compiler can obtain a second chip executable file after compiling the first type of kernel function. If the target compiler meets the first preset condition, step S230 will be executed to adapt the second type of kernel function in the first programming framework data based on the target compiler and target chip information. Otherwise, if the target compiler does not meet the first preset condition, the target compiler needs to be further adjusted or optimized until a target compiler that meets the first preset condition is obtained before step S230 will be executed.

[0085] Furthermore, the conditions for executing step S240 may include the candidate kernel function set output in step S230 meeting the second preset condition, that is, the test result of the kernel function in the candidate kernel function set after executing the preset kernel function test case based on the second chip is passed. If the candidate kernel function set meets the second preset condition, step S240 is executed again to adjust the kernel function in the candidate kernel function set based on the preset kernel function optimization target; otherwise, if the candidate kernel function set does not meet the second preset condition, the candidate kernel function set needs to be further adjusted or optimized until a candidate kernel function set that meets the second preset condition is obtained, and then step S240 is executed again.

[0086] Furthermore, the conditions for executing step S250 may include that the target kernel function set output in step S240 meets the preset kernel function optimization target. For example, if the measured throughput of the kernel functions in the target kernel function set on the second chip is greater than the preset throughput threshold, then step S250 is executed again when the target kernel function set meets the preset kernel function optimization target to generate the second programming framework corresponding to the second chip. Otherwise, if the target kernel function set does not meet the preset kernel function optimization target, the target kernel function set needs to be further adjusted or optimized until a target kernel function set that meets the preset kernel function optimization target is obtained before step S250 is executed.

[0087] By employing the methods described in steps S210 to S250, the programming framework adaptation process is divided into several orderly stages, such as compiler adaptation, kernel function migration, performance optimization, and framework generation. Clear quality gating conditions are set for each stage, ensuring that subsequent stages are built upon the verified compliance with the corresponding conditions or objectives of the preceding stages. This improves the efficiency and reliability of programming framework adaptation.

[0088] Figure 3 This is a flowchart illustrating another programming framework adaptation method provided in this embodiment. Figure 3 As shown, step S250 above, which generates a second programming framework corresponding to the second chip based on the target compiler and the target kernel function set, may include the following steps S251 and / or S252.

[0089] Step S251, Ecological integration step.

[0090] For example, this step S251 can also be called the ecosystem integration stage. For instance, a second programming framework can be generated through ecosystem integration verification.

[0091] The ecosystem integration verification can include verification of at least one of attention mechanisms (such as FlashAttention), fusion operators, and end-to-end models. Ecosystem integration verification enables at least one complete model compiled based on the target compiler and target kernel function to achieve correct end-to-end inference.

[0092] In some examples, a target model can be generated based on the target compiler and the target kernel function set, and the target model can be inferred and verified; if the target model inference verification is successful, a second programming framework can be generated based on the target compiler and the target kernel function set.

[0093] For example, the target model can be a deep learning model, such as a Transformer model containing an attention mechanism operator, or a convolutional neural network containing a fusion operator. This target model can be constructed using an application programming interface provided by a second programming framework, calling kernel functions from the target kernel function set to implement the various operators in the model. For instance, the attention mechanism in the Transformer model can be implemented using the FlashAttention operator, which can call matrix multiplication kernel functions, Softmax kernel functions, etc., from the target kernel function set. As another example, the fusion operator can be implemented by calling first-type kernel functions and / or second-type kernel functions.

[0094] In this example, inference verification of the target model may include running the complete forward inference process of the target model on the second chip, comparing the inference result with the inference result of a reference platform (e.g., the first chip), and verifying the correctness of the end-to-end inference result. This inference verification may include preparing test input data, performing forward propagation of the target model on the second chip, obtaining the output result, and numerically comparing the output result with the output result obtained by running the same model on the reference platform. In some examples, this numerical comparison may use a preset numerical tolerance standard, which may include relative tolerance (rtol) and absolute tolerance (atol). The specific implementation of these standards can be found in the descriptions of the foregoing embodiments of this disclosure, and will not be repeated here.

[0095] If the target model inference verification passes, a second programming framework can be generated based on the target compiler and target kernel function set. For example, the target compiler, target kernel function set, and related application programming interface documentation can be packaged into a release package for the second programming framework. Passing this inference verification indicates that at least one complete model has correct end-to-end inference, meaning the model output is consistent with the reference platform's output within a tolerable range.

[0096] In this way, end-to-end model inference verification can verify the collaborative correctness of the target compiler and the target kernel function set at the system level. This helps the second programming framework support actual deep learning tasks and avoids situations where individual kernel functions pass verification but problems arise during system integration.

[0097] Optionally, if the target model inference verification fails, the target compiler and / or target kernel function set are adjusted according to the reason for the failure, a new target model is generated based on the adjusted target compiler and / or target kernel function set, and inference verification is performed on the new target model until the target model inference verification passes.

[0098] For example, the failure reason may include compiler backend issues, kernel function implementation issues, missing operator issues, or performance timeout issues. If the failure reason is a compiler backend issue, such as the compiler-generated binary file producing incorrect results under certain input shapes, the downscaling channel or code generation logic of the target compiler can be adjusted. If the failure reason is a kernel function implementation issue, such as a kernel function's numerical precision not meeting requirements under certain data types or input shapes, the implementation of that kernel function can be optimized or repaired. If the failure reason is a missing operator or execution error, an operator adaptation strategy can be triggered to adapt the operator. If the failure reason is a timeout caused by a performance issue, performance tuning of the corresponding kernel function can be re-executed (e.g., step S240).

[0099] For example, after adjusting the target compiler and / or target kernel function set, a new target model can be generated based on the adjusted target compiler and / or target kernel function set, and the new target model can be inferred and verified. This process can be executed iteratively until the target model inference verification passes. In some examples, a maximum iteration limit can be set; if the verification still fails after exceeding this limit, failure information can be recorded and manual analysis can be performed.

[0100] In this way, through the iterative repair mechanism, problems that arise in end-to-end inference can be systematically resolved, avoiding the interruption of the entire migration process due to a single operator or kernel function problem, and improving the robustness and completeness of programming framework adaptation.

[0101] Step S252, Production Ready Step.

[0102] For example, this step S252 can also be referred to as the production-ready phase. For instance, a second programming framework can be generated through production regression testing.

[0103] In some examples, regression testing can be performed based on the target compiler and the target kernel function set; if the regression test passes, a second programming framework can be generated based on the target compiler and the target kernel function set.

[0104] For example, regression testing may include re-performing correctness verification on all migrated kernel functions in the second programming framework to ensure that previously verified kernel functions remain correct in the final version. This correctness verification may include the aforementioned pre-defined kernel function test cases (e.g., five-level hierarchical correctness verification), namely, smoke tests, shape scan tests, numerical stability tests, deterministic tests, and boundary condition tests. Regression testing can detect whether new defects have been introduced during subsequent development, such as optimization of one kernel function leading to performance degradation or correctness issues in other kernel functions.

[0105] In some examples, regression testing may also include establishing a Continuous Integration (CI) and Continuous Deployment (CD) pipeline to automate the execution of regression test suites. This CI and CD pipeline can be automatically triggered upon code commit, executing the compilation, testing, and deployment processes. For example, when a developer commits modifications to the target compiler or target kernel function set, the CI pipeline can automatically compile the modified code, execute the regression test suite on a secondary chip, and generate a test report. If all tests pass, the pipeline can automatically merge the changes into the main branch and generate a new release package. If tests fail, the pipeline can prevent the merge and notify the developer.

[0106] For example, the second programming framework generated upon passing regression testing can be a stable release package. This stable release package may include a target compiler and second programming framework data. The target compiler may be, for example, an executable file of a target compiler, and the second programming framework data may include library files of the target kernel function set, operator libraries, application programming interface documentation, and usage examples. This stable release package can be provided to users for developing and running deep learning or high-performance computing tasks on a second chip.

[0107] In this way, regression testing ensures the stability and reliability of the final released second programming framework, avoids known issues in the production environment, and improves users' trust in the second programming framework and their user experience.

[0108] In some examples, step S252 can be executed after step S251 has been successfully executed, or it can be executed separately (i.e., after step S240). This embodiment does not limit this.

[0109] In some examples, model inference verification and regression testing can be optional steps. In step S250, if the kernel functions in the target kernel function set cover all the operators required by the second programming framework, and each kernel function has been verified through preset kernel function test cases (e.g., five-level hierarchical correctness verification) and performance optimization, then the second programming framework can be directly generated based on the target compiler and the target kernel function set without additional model inference verification and regression testing steps. This improves the efficiency of programming framework adaptation.

[0110] It should be noted that, Figure 2 and Figure 3 The illustrated embodiment provides a programming framework adaptation method based on Phased Gated Migration (PGMR). This method breaks down the adaptation migration of the programming framework from a first chip (e.g., the source GPU) to a second chip (e.g., the target GPU) into P ordered stages (e.g., P=4 or 5). Each stage has a clearly defined goal, deliverables, and quality gating. The initiation of stage p+1 is contingent upon passing the gating of stage p. Skipping between stages is strictly prohibited to ensure that subsequent work is built upon a validated foundation. Figure 3 In the illustrated embodiment, each stage of the staged gating migration described above can be executed sequentially in the order of "compiler basics (step S220), core kernel functions (step S230), performance optimization (step S240), ecosystem integration (step S251), and production readiness (step S252)," forming a progressive path with increasing benefits and decreasing risks. This allows subsequent stages to be built on the basis of previous stages having verified compliance with corresponding conditions or goals, thereby improving the overall efficiency and reliability of programming framework adaptation.

[0111] In some embodiments of this disclosure, the aforementioned first programming framework data may further include multiple initial operators implemented based on the kernel functions of the first chip, such as an initial operator set (also known as an initial operator library). In this programming framework adaptation method, a corresponding operator adaptation strategy may also be provided when an operator gap is identified. For example, when adapting a second type of kernel function (e.g., step S230) or generating a second programming framework corresponding to the second chip (e.g., step S250), if there are missing operators on the second chip, the missing operators can be adapted based on a preset operator adaptation strategy to obtain a target operator corresponding to the second chip. The missing operator may include operators that cannot be executed or operators that do not exist among the multiple initial operators.

[0112] In this example, the identification of missing operators can occur at two points: The first point could be during the adaptation process for the second type of kernel function. If a kernel function fails to compile or execute on the second chip, the specific missing operator name and type can be located by analyzing compilation or runtime error information, thus identifying the operator as a missing operator. The second point could be during the generation of the code corresponding to the second programming framework, such as during the ecosystem integration phase. If end-to-end model inference fails due to a missing operator or execution error, the missing operator can be located by analyzing the inference error log.

[0113] In some examples, identified operator gaps can be recorded in an operator coverage matrix. This matrix may contain fields such as operator name, status, highest correctness level, peak utilization (pct_peak), and notes. The status can take one of the following values: MISSING, STRATEGY_A (first adaptation strategy adopted), STRATEGY_B (second adaptation strategy adopted), STRATEGY_C (third adaptation strategy adopted), STRATEGY_D (fourth adaptation strategy adopted), or VERIFIED. The highest correctness level can be an integer, for example, an integer between 0 and 5. This operator coverage matrix can be used to track the adaptation progress of each target operator, allowing engineers or agents to understand the operator coverage in the current migration state.

[0114] In some examples, the preset operator adaptation strategy may include multiple candidate adaptation strategies ranked from low to high intrusion. Candidate adaptation strategies with low intrusion can be prioritized for adaptation processing over those with high intrusion. This intrusion level can characterize the extent of modification to the original code structure of the first programming framework and / or the maintenance cost of synchronizing with the upstream domain-specific language framework version. For example, the upstream domain-specific language framework version may include the official backbone code repository or open-source project version of DSL programming frameworks such as Triton or TVM. For each missing operator, the strategy can be evaluated step-by-step, starting with the strategy with the lowest intrusion level, and the lowest intrusion level strategy that can solve the problem can be selected for adaptation processing. This preset operator adaptation strategy can be called the Minimum-Invasion Operator Adaptation strategy. When the programming framework on the second chip lacks support for certain operators, K adaptation strategies can be provided from low to high intrusion. For each missing operator, the lowest intrusion level strategy that can solve the problem is selected. The lower the intrusion level, the lower the implementation cost, the easier the synchronization with the upstream, and the higher the maintainability. Where K can be an integer greater than 1, for example, K can be 4.

[0115] In this way, by selecting the adaptation strategy with the lowest level of intrusion for each missing operator, the modification to the original code structure of the first programming framework can be minimized while meeting functional requirements, reducing the maintenance cost of synchronizing with the upstream framework version, and helping to improve the long-term maintainability of the second programming framework.

[0116] In some examples, the preset operator adaptation strategy may include first to fourth adaptation strategies ordered by the degree of intrusion from low to high, i.e., the degree of intrusion from low to high is: first adaptation strategy < second adaptation strategy < third adaptation strategy < fourth adaptation strategy. The first to fourth adaptation strategies are described below.

[0117] The first adaptation strategy can be based on domain-specific language text for operator adaptation. This strategy can implement missing operators using existing combinations of domain-specific language primitives without modifying the compiler or underlying code, resulting in minimal intrusion and zero upstream synchronization cost. Applicable scenarios for this strategy include situations where the missing operator can be expressed using combinations of existing domain-specific language primitives. For example, if the missing operator is an element-wise multiply-add fusion operator, it can be implemented by calling a combination of element-wise multiplication and addition primitives provided by the domain-specific language, without modifying any compiler code. This first adaptation strategy can also be called the DSL pure implementation strategy or Strategy A.

[0118] The second adaptation strategy can be an operator adaptation strategy based on compiler-stage extensions. This strategy extends the compiler optimization pass in the Triton GPU Intermediate Representation (TTGIR) layer of the compiler, adding specific scheduling policies or pattern matching rules for missing operators. This approach has low invasiveness and low upstream synchronization costs. Applicable scenarios for this second adaptation strategy include situations where missing operators require TTGIR-level scheduling optimizations or pattern matching, but do not require adding new intermediate representation operators. For example, operators requiring specific memory access patterns or thread configurations can be implemented by adding corresponding compiler optimization passes in the Triton GPU Intermediate Representation layer. This second adaptation strategy can also be called the compiler pass extension strategy or Strategy B.

[0119] The third adaptation strategy can be an operator adaptation strategy based on compiler language extensions. This strategy adds custom operator definitions to the Multi-Level Intermediate Representation (MLIR) dialect layer, providing dedicated intermediate representations for operators involving second-chip-specific hardware semantics. The intrusion level is moderate, and the upstream synchronization cost is also moderate. Applicable scenarios for this third adaptation strategy include situations where missing operators involve second-chip-specific hardware semantics and cannot be expressed using existing intermediate representation operators. These second-chip-specific hardware semantics can include dedicated coprocessor instructions, special buffer types, etc. For example, if the second chip has a dedicated matrix multiplication acceleration unit, and the calling semantics of this unit are incompatible with existing MLIR operators, a corresponding custom operator can be added to the MLIR dialect layer, and a reduced path from the operator to the second-chip instructions can be implemented. This third adaptation strategy can also be called the new dialect operator strategy or Strategy C.

[0120] The fourth adaptation strategy can be based on the native programming model of the second chip for operator adaptation. This fourth adaptation strategy can completely bypass the compilation pipeline of the domain-specific language and directly use the native programming model provided by the second chip manufacturer to implement the operator. It has the highest degree of intrusion and high upstream synchronization costs. The applicable scenarios for this fourth adaptation strategy can include situations where the missing operator needs to call the native programming model of the second chip (e.g., the target GPU). For example, the missing operator needs to completely bypass the compilation pipeline of the domain-specific language, or the performance requirements of the missing operator are extremely high, requiring direct control of the underlying hardware resources of the second chip. This fourth adaptation strategy can also be called the external kernel function strategy or strategy D. Optionally, this fourth adaptation strategy can only be selected when the first three strategies are not feasible.

[0121] In some examples, for any of the above adaptation strategies, the adapted target operator can be verified through preset kernel function test cases (e.g., five-level hierarchical correctness verification). After successful verification, the state of the target operator in the operator coverage matrix can be updated to the verified state. For example, for the kernel function that implements the target operator, the aforementioned preset kernel function test cases can be executed based on the second chip. Only if the test result is passed is the target operator determined to be usable, for example, by updating the state of the target operator in the operator coverage matrix to the verified state.

[0122] In some examples, a target adaptation strategy can be determined from the first to fourth adaptation strategies among the aforementioned preset operator adaptation strategies based on missing operator descriptions, compilation error information, and hardware capability descriptions, and the implementation path of the target adaptation strategy can be output. For example, if the operator can be implemented using existing domain-specific language primitive combinations, the first adaptation strategy (DSL pure implementation) can be used to adapt the operator. If the operator requires compiler-level scheduling optimization, the second adaptation strategy (compiler pass extension) can be used to extend the compiler. If the operator involves hardware semantics specific to the second chip, the third adaptation strategy (new dialect operator) can be used to add operator definitions at the MLIR dialect layer. If the operator needs to completely bypass the domain-specific language compilation pipeline, the fourth adaptation strategy (external kernel function) can be used to directly call the native programming model of the second chip.

[0123] In this way, by using a four-level adaptation strategy that sorts the intrusion levels from low to high, the feasible solution with the least intrusion can be selected for each missing operator. This ensures the correctness of the operator's function while maximizing code compatibility with the upstream domain's specific language framework and reducing the long-term maintenance cost of the second programming framework.

[0124] In some embodiments of this disclosure, the above-described programming framework adaptation method can be triggered or exited by a user (e.g., a programming engineer) and / or an intelligent agent. The method may further include: during the execution of the programming framework adaptation method, if an exit instruction triggered by a user and / or an intelligent agent to exit the programming framework adaptation method is received, an archive file corresponding to the programming framework adaptation method is generated; the archive file may include a progress file for recording the progress of the programming framework adaptation and / or a log file for recording data from the programming framework adaptation process.

[0125] In some examples, the progress file can be a migration_progress.md file, which records the migration progress status. This progress status can include one or more of the following: the current stage, the status within the stage, active kernel functions (e.g., kernel functions currently being adapted), a list of kernel functions that have completed migration, identified operator gaps, the current best speedup, the total number of experiments executed, current blocking items, the next step plan, and the last update time. This progress file can be written at the end of each session. Optionally, the progress file can be stored in a structured text format or other arbitrary format to facilitate users or agents quickly understanding the current status and next action plan of the migration project.

[0126] In some examples, the log file can be a results.tsv file, which records experimental record data for each experiment. This experimental record data may include one or more of the following: experiment number, experiment label, kernel function type, throughput, latency, theoretical peak utilization, relative baseline speedup, correctness results, correctness level for failures, peak memory usage, and experiment description. This log file can be appended after each experiment. Optionally, the log file can be stored in a structured text format or other arbitrary format to facilitate subsequent data analysis and performance comparison.

[0127] In some examples, if a user and / or agent triggers a start command to initiate the execution of the programming framework adaptation method and obtains the archive file corresponding to the programming framework adaptation method, the programming framework adaptation progress can be restored based on the archive file, and the programming framework adaptation method can continue to be executed based on the restored programming framework adaptation progress.

[0128] For example, the method for resuming the programming framework adaptation progress may include: first, checking if the progress file exists, and if it exists, reading the current stage and status information; second, checking the experimental log file, reading the most recent experimental records to confirm the continuity of the experiment; and finally, if the progress file shows an incomplete stage, resuming directly from that stage, skipping the initial interaction, and continuing to execute the programming framework adaptation method.

[0129] In some examples, progress files and experiment log files can provide complementary persistence mechanisms. Progress files can be geared towards engineering decisions, providing a snapshot of the current migration state and the next course of action; experiment log files can be geared towards data analysis, providing a complete record of data for each experiment. Either file can independently reconstruct the session state, providing dual fault tolerance.

[0130] In this way, the cross-session state persistence mechanism enables long-term migration projects to be carried out in stages, and each recovery does not require manual recall of the current state. This helps to improve the continuity and manageability of programming framework adaptation, and allows migration projects that take weeks to months to proceed in an orderly manner.

[0131] Figure 4 This is a schematic diagram of the architecture of a programming framework adaptation system provided in an embodiment of this disclosure. Figure 4 As shown, the programming framework adaptation system can include an adaptation migration process consisting of five phases, from phase 0 to phase 4. It provides a set of cross-cutting concern modules throughout the entire adaptation migration lifecycle. This set of cross-cutting concern modules can include one or more of the following: a hierarchical correctness gating module, an operator adaptation strategy module, and a state persistence module. Specifically, the hierarchical correctness gating module can be a mandatory gating judgment for each kernel function; the operator adaptation strategy module can be used for gap operator filling; and the state persistence module can be used throughout the entire process, recording the session state throughout the execution of the programming framework adaptation method. The session state can represent the execution phase and / or execution state of the programming framework adaptation method.

[0132] The programming framework adaptation architecture in this embodiment divides the programming framework adaptation and migration process into five ordered stages, from stage 0 to stage 4. Each stage has a clear goal and gating conditions. The set of cross-cutting concern modules (hierarchical correctness gating module, operator adaptation strategy module, and state persistence module) runs through the entire adaptation and migration lifecycle. The five stages are strictly ordered; for example, stage 1 does not start before the gating of stage 0 is passed. The five stages are described below.

[0133] Phase 0, compiler basic adaptation. The goal of Phase 0 can be the successful compilation of a first-type kernel function (simple kernel function). Its gating conditions can include compiling to generate an executable and valid binary file. The gating conditions of Phase 0 can be called the first preset conditions. The specific implementation of Phase 0 can be referred to in the specific description of step S220 and related steps in the foregoing embodiments of this disclosure, and will not be repeated here.

[0134] Phase 1: Core kernel function adaptation. The goal of Phase 1 can be element-wise / reduction / Softmax / matrix multiplication adaptation, and its gating conditions can include the hierarchical correctness of the core kernel function satisfying preset conditions. This core kernel function can also be called a second type kernel function, and the gating conditions of Phase 1 can be called second preset conditions. The specific implementation of Phase 1 can be referred to the specific description of step S230 and related steps in the foregoing embodiments of this disclosure, and will not be repeated here.

[0135] Phase 2, Performance Optimization. The goal of Phase 2 can be hardware-aware tuning and performance analysis. Performance analysis may include Performance Functional Counter (PFC) analysis, with gating conditions based on a preset throughput threshold. For example, the measured throughput of the kernel function on the second chip may exceed the preset throughput threshold, which could be 60% of the theoretical peak. The gating condition of Phase 2 can be called the preset kernel function optimization target. The specific implementation of Phase 2 can be found in the detailed description of step S240 and related steps in the foregoing embodiments of this disclosure, and will not be repeated here.

[0136] Phase 3, Ecosystem Integration. The goal of Phase 3 may include at least one of attention mechanisms, fusion operators, and end-to-end models, and its gating condition may include passing the inference verification of the target model. The specific implementation of Phase 3 can be found in the detailed description of step S251 and related steps in the foregoing embodiments of this disclosure, and will not be repeated here.

[0137] Phase 4, Production Ready. The goal of Phase 4 may include at least one of Continuous Integration (CI), Continuous Deployment (CD), Regression Testing, and Upstream Tracing. Its gating conditions may include passing regression tests and stable release. The specific implementation of Phase 4 can be found in the detailed description of step S252 and related steps in the foregoing embodiments of this disclosure, and will not be repeated here.

[0138] The input to the programming framework adaptation system may include the target chip information of the second chip (e.g., the hardware environment of the target GPU) and the first programming framework (e.g., the compiler and the first programming framework data including the DSL framework source code), and the output may include the stage progress status and / or gating determination results.

[0139] By adopting this programming framework adaptation system, the process of adapting the programming framework is divided into several orderly stages, such as compiler adaptation, kernel function migration, performance optimization, ecosystem integration, and production readiness. Each stage is given clear quality gating conditions, which ensures that subsequent stages are built on the basis of previous stages that have been verified to meet the corresponding conditions or goals. This improves the efficiency and reliability of programming framework adaptation.

[0140] Figure 5 This is a flowchart illustrating another programming framework adaptation method provided in this embodiment. Figure 5 As shown, the programming framework adaptation method may include the following steps S510 to S590.

[0141] Step S510: Begin the adaptation and migration.

[0142] Step S520: Determine if an archive file exists.

[0143] If an archive file for the programming framework adaptation method exists, proceed to step S530; otherwise, if no archive file exists, proceed to step S540 and subsequent steps, i.e., start execution from stage 0. The archive file may include a progress file for recording the programming framework adaptation progress and / or a log file for recording data from the programming framework adaptation process.

[0144] Step S530: Restore to the recorded target stage and continue executing the programming framework adaptation method from the target stage.

[0145] For example, when a progress file is detected, the system reads the context information saved in the file (such as the identifier of the completed stage, the current software and hardware status, etc.). Then, the corresponding checkpoint environment is loaded, and the previous adaptation work is no longer repeated. Instead, the subsequent compilation or migration operations continue directly from the recorded breakpoint, thereby improving overall efficiency.

[0146] Step S540, compiler basic adaptation (stage 0).

[0147] For example, in this step, the core objective of stage 0 is compiler basic adaptation. The specific execution logic may include checking whether the compiled binary file is valid (i.e., whether it is an executable file for the second chip) through gate 0. If not, it is determined to be FAIL (invalid), and the repair process can be entered. The compiler basic adaptation problem is solved by repairing the compiler backend (IR redirection / dialect extension), thereby re-performing compiler basic adaptation. If yes, it is determined to be PASS (file valid), and the progress of this stage can be solidified and saved (i.e., the progress file is saved), and then stage 1 is entered.

[0148] Step S550, Core kernel function adaptation (Phase 1).

[0149] For example, core kernel functions such as element-wise, reduction, softmax, and matrix multiplication can be adapted sequentially. Each kernel function can be migrated, including steps such as mode selection, DSL adaptation, and semantic mapping. After the migration is completed, hierarchical correctness verification is performed. Then, all kernel functions are checked for verification through gate 1. If not, it is judged as FAIL. If there are gaps in the operators, the operator adaptation strategy selection needs to be triggered to supplement the missing operators. If so, it is judged as PASS, the progress is saved, and the performance optimization of stage 2 is entered.

[0150] Step S560, Performance Optimization (Phase 2).

[0151] For example, hierarchical tuning and PFC analysis can be performed to deeply analyze performance bottlenecks and use performance function counters to guide optimization directions. Further, gating 2 checks whether the performance meets the standards. If not, it is judged as FAIL and tuning is iterated again; if yes, it is judged as PASS, the progress is saved, and the process enters stage 3, ecosystem integration.

[0152] Step S570, Ecosystem Integration (Phase 3).

[0153] For example, validation can be performed in real-world business scenarios with complex operators / models to complete ecosystem integration. Hierarchical correctness verification is performed to improve the stability of end-to-end logic. Furthermore, gating 3 can be used to check if the model inference is correct; if not, it is judged as FAIL. If there are gaps in the operators, an operator adaptation strategy selection needs to be triggered to supplement the missing operators; if yes, it is judged as PASS, the progress is saved, and the process proceeds to stage 4, production readiness.

[0154] Step S580, Production Ready (Stage 4).

[0155] For example, comprehensive regression testing can be performed to verify industrial-grade reliability. Gating 4 confirms whether the regression test passes; if it does, it is considered PASS, and the adaptation and migration process can be completed; if not, it can revert to the previous stage (e.g., stage 3) for repair.

[0156] Step S590, adaptation and migration complete.

[0157] It should be noted that the specific implementation of each step in this embodiment can be referred to the description in the foregoing embodiments of this disclosure, and will not be repeated here.

[0158] This programming framework adaptation method divides the adaptation process into several orderly stages, including compiler adaptation, kernel function migration, performance optimization, ecosystem integration, and production readiness. It sets clear quality gating conditions for each stage, which ensures that subsequent stages are built on the basis of previous stages that have been verified to meet the corresponding conditions or goals. This improves the efficiency and reliability of programming framework adaptation.

[0159] Figure 6 This is a schematic diagram of the interaction flow of a programming framework adaptation method provided in an embodiment of this disclosure. For example... Figure 6 As shown, the programming framework adaptation system includes a migration engine, a compiler backend adaptation module, a hierarchical correctness gating module, an operator adaptation strategy module, and a state persistence module. Users and / or AI agents can trigger the programming framework adaptation system to execute a programming framework adaptation method, which may include the following steps S610 to S660.

[0160] Step S610: Start the adaptation migration.

[0161] In this step, the user and / or AI agent can input a start command to initiate the programming framework adaptation method.

[0162] Step S620: Check the progress file. If there is no record, start from stage 0.

[0163] In this step, the migration engine can read the progress file from the state persistence module. If the state persistence module has no record (no progress file), it starts from stage 0; otherwise, if a progress file is read, it can restore to the recorded target stage (e.g., stage 1 or stage 2) based on the progress file and continue executing the programming framework adaptation method from the target stage.

[0164] Step S630, compiler basic adaptation (stage 0).

[0165] In this step, the migration engine can call the compiler backend adaptation module to attempt to compile simple kernel functions (such as vector addition) and check the compilation results. If the compilation fails, an IR redirection repair operation can be performed. After the repair is completed, the compilation is re-executed. If the compilation is successful, it means that stage 0 has passed (PASS). The progress status of stage 0 can be recorded through the state persistence module.

[0166] Step S640, Core kernel function adaptation (Phase 1).

[0167] In this step, the adaptation of each core kernel function can be performed cyclically (covering element-wise, reduction, softmax, and matrix multiplication types). During kernel function adaptation, a mode selection is first performed (e.g., General mode, Tensor mode, or Tensor Core mode), followed by DSL adaptation, then hierarchical correctness verification, and the verification results at each level are recorded. If a level of correctness verification fails and there is an operator gap, operator adaptation is performed. For example, a target adaptation strategy can be selected from multiple strategies through the operator adaptation strategy module, and the target adaptation strategy is executed. After that, hierarchical correctness verification is performed again. If all levels of correctness verification pass successfully, the coverage matrix is ​​updated. When all hierarchical correctness verifications of the core kernel functions have passed, the state persistence module can record the state of Phase 1 passing (PASS).

[0168] Step S650, performance optimization (Phase 2).

[0169] In this step, the performance optimization of each kernel function can be performed in a loop. During optimization, hierarchical tuning can be performed according to Amdahl influence (e.g., Tier 1-6), and experimental records can be appended to the log file (results.tsv). If the correctness verification is passed after performance optimization, the status of PASS in Phase 2 is recorded, indicating that the current adaptation scheme has achieved the expected performance indicators on the target chip (e.g., the second chip).

[0170] Step S660, session ends.

[0171] In this step, the final progress can be written to the progress file (migration_progress.md) and a session end marker (session_end) can be appended to the log file (results.tsv) to mark the official completion of this adaptation and migration task. All intermediate states and final results have been persistently saved.

[0172] Thus, through Figure 6 The method shown implements the complete interactive flow from initiating adaptation migration to the end of the session. The migration engine coordinates the compiler backend adaptation module, the hierarchical correctness gating module (5 levels of correctness gating), the operator adaptation strategy module, and the state persistence module. The loop structure in Phase 1 demonstrates the complete process of each kernel function undergoing mode selection, adaptation, verification, and possible gap filling and re-verification.

[0173] use Figure 6 The programming framework adaptation method shown divides the adaptation process into several orderly stages, such as compiler adaptation, kernel function migration, performance optimization, ecosystem integration, and production readiness. It sets clear quality gating conditions for each stage, which enables subsequent stages to be built on the basis of the previous stages having verified compliance with the corresponding conditions or goals, thereby improving the efficiency and reliability of programming framework adaptation.

[0174] This disclosure also provides an electronic device, such as... Figure 7As shown, the electronic device 700 may include a memory 710 and a processor 720. The memory can be used to store computer instructions, and the processor can be used to retrieve computer instructions from the memory to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. The processor may be one or more, and these processors may execute instructions individually or jointly. Similarly, the memory may be one or more, and these memories may store the aforementioned computer instructions individually or jointly. The processor of the electronic device may be a CPU, GPU, GPGPU, NPU, TPU, or a chip in the form of an Application Specific Integrated Circuit (ASIC), System on Chip (SOC), Field Programmable Gate Array (FPGA), Programmable Logic Array (PLA), etc. This embodiment does not limit this. Optionally, the electronic device 700 may be... Figure 1 The programming platform 100 shown.

[0175] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a temporary storage medium.

[0176] This disclosure also provides a computer program product that may include a computer program that, when executed by a processor, can implement all or part of the steps of any of the methods in the foregoing embodiments of this disclosure.

[0177] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and apparatuses according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, unit, or part of a circuit. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that embodiments of the present disclosure may include some or all of the functions marked in the multiple blocks in the drawings, and may also include other functions not shown in the blocks in the drawings. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in hardware that performs the specified function or action, or in a combination of dedicated hardware and computer instructions. Unless otherwise specified, implementation in hardware, implementation in software, and implementation in a combination of software and hardware may be equivalent.

[0179] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A programming framework adaptation method, characterized in that, The method includes: Obtain the first programming framework corresponding to the first chip and the target chip information of the second chip; wherein the chip architecture of the second chip and the first chip are different, the first programming framework includes an initial compiler adapted to the first chip and first programming framework data, and the first programming framework data includes a first type kernel function and a second type kernel function; The initial compiler is adapted based on the target chip information to obtain a target compiler that meets the first preset conditions; wherein, the first preset conditions indicate that the target compiler compiles the first type of kernel function to obtain an executable file for the second chip; Based on the target compiler and the target chip information, the second type of kernel functions in the first programming framework data are adapted to obtain a set of candidate kernel functions that meet the second preset conditions; wherein, the second preset conditions indicate that the kernel functions in the candidate kernel function set pass the test after executing the preset kernel function test cases based on the second chip; Based on a preset kernel function optimization objective, the kernel functions in the candidate kernel function set are adjusted to generate a target kernel function set that meets the preset kernel function optimization objective; Based on the target compiler and the target kernel function set, a second programming framework corresponding to the second chip is generated; The first programming framework data also includes multiple initial operators implemented based on the kernel functions of the first chip; the method further includes: When adapting the second type of kernel function or generating a second programming framework corresponding to the second chip, if there are missing operators on the second chip, the missing operators are adapted based on a preset operator adaptation strategy to obtain a target operator corresponding to the second chip. The missing operators include operators that cannot be executed or operators that do not exist among multiple initial operators. The preset operator adaptation strategy includes multiple candidate adaptation strategies sorted by intrusion level from low to high. The candidate adaptation strategies with low intrusion level are used for adaptation processing with priority over the candidate adaptation strategies with high intrusion level.

2. The method according to claim 1, characterized in that, The complexity of the second type of kernel function is higher than that of the first type of kernel function; where: The first type of kernel function includes atomic kernel functions, which do not call other kernel functions; The second type of kernel function includes one or more of the following: element-wise kernel function, reduction kernel function, normalized exponential transformation kernel function, and matrix multiplication kernel function.

3. The method according to claim 1, characterized in that, The preset kernel function test cases include multiple test case sets executed in a preset order, with the test case set that is later in the order being executed if the test case set that is earlier in the order passes the test.

4. The method according to claim 3, characterized in that, The multiple test case sets include a first to a fifth set of test cases executed in a preset order, wherein: The first set of test cases includes smoke test cases used to verify the correctness of compilation and numerical values; The second set of test cases includes shape scan test cases used to verify the correctness of generalization adaptation under multi-scale input. The third set of test cases includes numerical stability test cases used to verify numerical robustness under extreme input scenarios; The fourth set of test cases includes deterministic test cases used to verify the reproducibility of the calculation results in multiple tests; The fifth set of test cases includes boundary condition test cases used to verify the correctness of non-standard input processing.

5. The method according to claim 1, characterized in that, The kernel function optimization objective includes ensuring that the measured throughput of the kernel functions in the target kernel function set on the second chip is greater than a preset throughput threshold, whereby the preset throughput threshold is a throughput threshold set based on the theoretical peak computing power of the second chip.

6. The method according to claim 1, characterized in that, The step of generating a second programming framework corresponding to the second chip based on the target compiler and the target kernel function set includes: A target model is generated based on the target compiler and the target kernel function set, and the target model is then inferred and verified. If the target model inference verification passes, the second programming framework is generated based on the target compiler and the target kernel function set.

7. The method according to claim 6, characterized in that, The step of generating a second programming framework corresponding to the second chip based on the target compiler and the target kernel function set further includes: If the target model inference verification fails, the target compiler and / or the target kernel function set are adjusted according to the reason for the failure. A new target model is generated based on the adjusted target compiler and / or target kernel function set, and inference verification is performed on the new target model until the target model inference verification passes.

8. The method according to claim 6, characterized in that, The generation of the second programming framework based on the target compiler and the target kernel function set includes: Regression testing was performed based on the target compiler and the target kernel function set. If the regression test passes, the second programming framework is generated based on the target compiler and the target kernel function set.

9. The method according to claim 1, characterized in that, The preset operator adaptation strategy includes first to fourth adaptation strategies sorted by intrusion level from low to high, wherein: The first adaptation strategy is to adapt operators based on domain-specific language text; The second adaptation strategy is to adapt operators based on compiler-stage extensions; The third adaptation strategy is to adapt operators based on compiler language extensions; The fourth adaptation strategy is to adapt operators based on the native programming model of the second chip.

10. The method according to any one of claims 1 to 9, characterized in that, The programming framework adaptation method is triggered by a user and / or an agent to execute or exit; the method further includes: During the execution of the programming framework adaptation method, if an exit instruction is received from the user and / or the agent to exit the programming framework adaptation method, an archive file corresponding to the programming framework adaptation method is generated; the archive file includes a progress file for recording the programming framework adaptation progress and / or a log file for recording programming framework adaptation process data. If a start instruction is received from the user and / or the agent to trigger the execution of the programming framework adaptation method and an archive file corresponding to the programming framework adaptation method is obtained, the programming framework adaptation progress is restored based on the archive file, and the programming framework adaptation method is continued to be executed based on the restored programming framework adaptation progress.

11. An electronic device, characterized in that, The method includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method of any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 10.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 10.

Images