A method, device and terminal device for cross-platform migration of artificial intelligence programs

Abstract

The present application is applicable to the technical field of program migration, and provides a method, apparatus and terminal device for cross-platform migration of an artificial intelligence program, the method comprising: obtaining code data of an AI program to be migrated on a source development platform and platform configuration information of a target development platform, wherein the computing architecture of the target development platform is a CANN platform; when the code data is source code data, extracting and converting multiple CUDA API call interfaces in the source code data according to the platform configuration information of the CANN platform to obtain multiple first API call interfaces; when the code data is PTX data, parsing and translating the PTX data according to the platform configuration information of the CANN platform to obtain an IR instruction set; controlling the CANN acceleration engine to execute the first API call interface or the IR instruction set to migrate the AI ​​program to be migrated from the source development platform to the CANN platform, significantly simplifying the migration process and improving the migration efficiency.

Description

Technical Field

[0001] The present application belongs to the technical field of program migration, and in particular, relates to a method, apparatus and terminal device for cross-platform migration of artificial intelligence programs. Background Art

[0002] At present, with the rapid development of deep learning technology, the scale and computing requirements of artificial intelligence (AI) models continue to increase. Graphics Processing Units (GPUs) play an important role in this process, and a large number of AI programs rely on GPU platforms to run. As a hardware accelerator specially optimized for AI computing, Neural Processing Units (NPUs) have significant advantages in AI reasoning and training tasks, especially in low power consumption and high concurrent computing. Therefore, NPUs have gradually become an important technical route that cannot be ignored in the field of AI.

[0003] However, the NPU ecosystem is still in its early stages of development. Although it has certain advantages in computing power, its software and hardware ecosystem is not yet mature and there is a large gap with the GPU ecosystem. How to enable developers to more conveniently port AI programs from the GPU platform to the NPU platform has become an urgent problem to be solved.

[0004] Due to the complexity of the NPU internal ecosystem, the NPU architecture is quite different from the GPU. Although there are many technical solutions for migrating AI programs from the GPU platform to the NPU platform, there are generally many problems at the level of static code analysis: (1) Because NPU has different frameworks such as pytorch_npu and MindSpore, users can only choose one of the frameworks to migrate GPU programs. Each framework provides an independent migration tool. Although these tools can convert through simple Python code layer function calls, due to the large differences between frameworks, it is impossible to provide a unified solution. Developers need to learn specific frameworks and adjust and optimize based on framework characteristics. In addition, the migration process usually requires manual forward alignment and backward precision alignment, which increases the difficulty of development and learning cost. (2) Most of the current migration tools only perform function call conversion at the code level, which can cover limited scenarios and is prone to conversion errors. Developers still need manual intervention to ensure the correct execution of the forward reasoning process. In addition, these migration tools usually require manual confirmation and adjustment of variable precision to ensure that the data precision during backward propagation is consistent with the GPU program, which increases the workload and complexity of the development process. (3) pytorch_npu provides a pure PyTorch migration tool. A program migration solution that generates a standard computation graph by parsing the PyTorch code, and maps it to one or more corresponding Ascend operators at the operator granularity. Finally, these operators complete the program migration by calling the functionally equivalent Ascend operators through AscendCL, thereby achieving migration. For codes containing custom CUDA kernels, pytorch_npu cannot handle them. When users rewrite the CUDA kernels and embed them through methods such as CuPy, the difficulty of migration increases significantly. Therefore, the above problems limit the efficient migration of AI programs from GPU platforms to NPU platforms. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus and terminal device for cross-platform migration of artificial intelligence programs, which can solve the problems of complex process and high development cost of cross-platform migration of artificial intelligence programs due to differences in existing framework-level migration tools, manual intervention or only supporting static code parsing.

[0006] In a first aspect, an embodiment of the present application provides a method for cross-platform migration of an artificial intelligence program, the method comprising: Obtain code data of the AI ​​program to be migrated on the source development platform and platform configuration information of the target development platform, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data and intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; When the code data of the AI ​​program to be migrated is the source code data, extracting and converting multiple general parallel computing architecture development CUDA API call interfaces in the source code data according to the platform configuration information of the CANN platform to obtain multiple first API call interfaces applicable to the CANN platform; When the code data of the AI ​​program to be migrated is the PTX data, the PTX data is parsed and translated according to the platform configuration information of the CANN platform to obtain an intermediate representation IR instruction set applicable to the CANN platform; On the target development platform, the CANN acceleration engine is controlled to execute the first API call interface or the IR data to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

[0007] In a possible implementation manner of the first aspect, when the code data of the AI ​​program to be migrated is the source code data, according to the platform configuration information of the CANN platform, multiple general parallel computing architecture development CUDA API call interfaces in the source code data are extracted and converted to obtain multiple first API call interfaces applicable to the CANN platform, including: Extracting the plurality of CUDA API calling interfaces in the source code data to obtain a plurality of second API calling interfaces; According to the platform configuration information of the CANN platform and the preset mapping table, the plurality of the second API calling interfaces are converted to obtain the plurality of the first API calling interfaces applicable to the CANN platform; wherein the preset mapping table is used to characterize the mapping relationship between the CUDA API calling interface and the CANN API calling interface.

[0008] In a possible implementation manner of the first aspect, extracting the multiple CUDA API calling interfaces in the source code data to obtain multiple second API calling interfaces includes: Through the performance analyzer or tool extension library of the deep learning framework PyTorch, the multiple CUDA API calling interfaces of the AI ​​program to be migrated during runtime are captured, and the kernel function information of the CUDA API calling interface is parsed to obtain the multiple second API calling interfaces.

[0009] In a possible implementation manner of the first aspect, converting the plurality of second API calling interfaces according to the platform configuration information of the CANN platform and a preset mapping table to obtain the plurality of first API calling interfaces applicable to the CANN platform includes: Parsing the kernel function information of the second API call interface to obtain multiple parsing results; wherein the kernel function information includes function signature, memory operation, calculation logic and thread scheduling; According to the plurality of analysis results and the preset mapping table, the second API calling interface is mapped to obtain the first API calling interface corresponding to the second API calling interface.

[0010] In a possible implementation manner of the first aspect, the kernel function information of the second API call interface is parsed to obtain multiple parsing results, including: Using regular expressions, extract the signature and body code of the global function in the second API call interface, and obtain the input and output variables and their data types of the second API call interface; Parsing the data interaction between the host and the device corresponding to the source development platform to obtain the memory operation information of the second API calling interface; Analyze the core mathematical operations and data processing in the second API call interface to obtain the key calculation mode of the second API call interface; Capture the parallel computing index logic and related loop control in the second API calling interface to obtain the parallel computing granularity and scope of the second API calling interface.

[0011] In a possible implementation manner of the first aspect, parsing and translating the PTX data according to the platform configuration information of the CANN platform to obtain an intermediate representation IR instruction set applicable to the CANN platform includes: Determine, according to the platform configuration information of the CANN platform, a PTX translator corresponding to the platform configuration information; Through the PTX translator, multiple key information in the PTX data is extracted, and the multiple key information is converted into the IR instruction set; wherein the key information includes thread scheduling, memory access instructions, calculation instructions and branch control instructions.

[0012] In a possible implementation manner of the first aspect, after obtaining the multiple first API call interfaces applicable to the CANN platform, the method further includes: According to the first API calling interface, program development is performed on the software development tool CANN SDK to obtain a corresponding user program; On the target development platform, the CANN acceleration engine is controlled to execute the user program to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

[0013] In a possible implementation manner of the first aspect, when the code data is the source code data, the method further includes: The source code data is compiled into the PTX data by a compiler, so as to obtain the IR instruction set applicable to the CANN platform according to the PTX data.

[0014] In a second aspect, an embodiment of the present application provides an artificial intelligence program cross-platform migration device, the device comprising: An acquisition module is used to acquire code data of the artificial intelligence AI program to be migrated on the source development platform and platform configuration information of the target development platform, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data and intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; A first conversion module is used to extract and convert multiple general parallel computing architecture development CUDA API call interfaces in the source code data according to the platform configuration information of the CANN platform when the code data of the AI ​​program to be migrated is the source code data, so as to obtain multiple first API call interfaces applicable to the CANN platform; A second conversion module is used to parse and translate the PTX data according to the platform configuration information of the CANN platform when the code data of the AI ​​program to be migrated is the PTX data, so as to obtain an intermediate representation IR instruction set applicable to the CANN platform; An execution module is used to control the CANN acceleration engine to execute the first API call interface or the IR instruction set on the target development platform to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods for cross-platform migration of artificial intelligence programs when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cross-platform migration method of the artificial intelligence program described in any one of the above items is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the cross-platform migration method of an artificial intelligence program as described in any one of the first aspects above.

[0018] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The cross-platform migration method of an artificial intelligence program provided in an embodiment of the present application obtains the code data of the artificial intelligence AI program to be migrated on the source development platform and the platform configuration information of the target development platform, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data, intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; when the code data of the AI ​​program to be migrated is source code data, according to the platform configuration information of the CANN platform, multiple general parallel computing architecture development CUDA API call interfaces in the source code data are extracted and converted to obtain multiple first API call interfaces applicable to the CANN platform; when the code data of the AI ​​program to be migrated is PTX data, according to the platform configuration information of the CANN platform, the PTX data is parsed and translated to obtain an intermediate representation IR instruction set applicable to the CANN platform; then, on the target development platform, the CANN acceleration engine is controlled to execute the first API call interface or the IR instruction set to migrate the AI ​​program to be migrated from the source development platform to the CANN platform. It can be seen that this application realizes the cross-platform migration of AI programs by automatically extracting the CUDA API call interface in the AI ​​program source code data and converting it into the CANN API call interface or directly parsing the PTX data of the AI ​​program and translating it into the IR instruction set. The CANN acceleration engine then executes the CANN API call interface or the IR instruction set. This migration method effectively avoids the defects of existing tools such as the need for manual code modification and high learning costs, shields users from complex underlying details, and eliminates the need for users to manually adjust the code, thereby significantly simplifying the migration process, reducing development costs, and improving the efficiency of cross-platform migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is a flowchart of a method for cross-platform migration of an artificial intelligence program provided by an embodiment of the present application; Figure 2 This is a flowchart of migrating an AI program from a GPU platform to an NPU platform provided by an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of an artificial intelligence program cross-platform migration device provided by an embodiment of the present application; Figure 4It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0022] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0023] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0024] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0025] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0027] First, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0028] Graphics Processing Uint (GPU), also known as visual processor, is a microprocessor specially used to perform image computing on personal computers, workstations and some mobile devices. It is mainly used in games, virtual reality, deep learning, autonomous driving and other fields.

[0029] The neural network processing unit NPU is a hardware processor specially designed to accelerate neural network operations. It is mainly used in artificial intelligence (AI) tasks related to deep learning, such as image recognition, speech recognition, big data processing, etc.

[0030] CANN platform (Compute Architecture for Neural Networks) is a heterogeneous computing architecture used to optimize the execution efficiency of AI tasks on NPU. CANN platform can provide a rich tool chain and library, such as programming interface, compiler, operator library, etc., to support the whole process from model development, training to reasoning, and support developers to efficiently deploy AI models on NPU hardware to achieve low power consumption, high performance reasoning and training.

[0031] CUDA (Compute Unified Device Architecture) is the abbreviation of Compute Unified Device Architecture, which is a parallel computing platform and programming model.

[0032] SDK (Software Development Kit) is the abbreviation of software development kit.

[0033] CUDA SDK is a software development kit developed for CUDA. It provides a series of sample source codes, utilities and white papers, covering a wide range of application scenarios, which can help developers quickly get started with CUDA programming and understand the features and performance optimization techniques of CUDA.

[0034] The above is a brief introduction to the nouns involved in the embodiments of the present application, which will not be repeated below.

[0035] This application example provides an example of a method for migrating an artificial intelligence program across platforms. Figure 1 , Figure 1 This is a flowchart of a method for cross-platform migration of an artificial intelligence program provided in an embodiment of the present application. As an example but not a limitation, the method can be applied to or run in an electronic device. The method includes: S11. Obtain code data of the AI ​​program to be migrated on the source development platform and platform configuration information of the target development platform, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data and intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; S12. When the code data of the AI ​​program to be migrated is source code data, extract and convert multiple CUDA API call interfaces developed by a general parallel computing architecture in the source code data according to the platform configuration information of the CANN platform to obtain multiple first API call interfaces applicable to the CANN platform; S13. When the code data of the AI ​​program to be migrated is PTX data, the PTX data is parsed and translated according to the platform configuration information of the CANN platform to obtain an intermediate representation IR instruction set applicable to the CANN platform; S14. On the target development platform, control the CANN acceleration engine to execute the first API call interface or IR instruction set to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

[0036] In this application example, the source development platform is the platform where the AI ​​program to be migrated was originally developed and run. The target development platform is the new platform to which the AI ​​program to be migrated needs to be migrated. In this application example, the migration of the AI ​​program to be migrated from the GPU platform (source development platform) to the NPU platform (target development platform) is used as an example for explanation. The computing architecture of the target development platform is a platform using the heterogeneous computing architecture CANN.

[0037] The code data of the AI ​​program to be migrated is the code of the AI ​​program that needs to be migrated between different platforms. The code data can be the source code data of the program or the intermediate code PTX data of the program. Among them, the source code data is the source code used to guide computers or other programmable devices to perform specific tasks, that is, a text file written by developers in a programming language, which contains the original expression of the program. The intermediate code (Parallel Thread Exection, PTX) data is a low-level intermediate code with an assembly-like syntax that can describe the underlying behaviors such as thread scheduling, memory operations, and computing logic. It is mainly used for parallel programming and underlying optimization of AI programs to be migrated.

[0038] Platform configuration information refers to a series of hardware specifications and software environment configuration parameters required for the target development platform when developing and deploying applications, such as computing architecture, development tools, processor performance, application parameters, operating environment, etc. Platform configuration information will serve as an important reference during program migration to ensure that the program can run correctly and efficiently on the target platform.

[0039] In the example of this application, the CUDA API calling interface is a series of programming interfaces provided by the general parallel computing architecture for developers to call, so that developers can make full use of the source development platform for parallel computing. The first API calling interface is a calling interface suitable for the CANN platform obtained by converting the CUDA API calling interface extracted from the source code data on the source development platform, so as to apply the first API calling interface on the CANN platform. Among them, the first API calling interface can perform the same functions on the CANN platform as the CUDA API calling interface on the source development platform.

[0040] In this application example, the Intermediate Representation (IR) instruction set is an intermediate form generated during the compilation of PTX data, which is between the source code and the target machine code. The IR instruction set provides an abstract representation method and an independent intermediate form that can facilitate compiler optimization and cross-platform compilation. The IR instruction set usually contains a series of instructions for representing program structure and operations, which are lower than the source code but higher than the machine code at an abstract level, such as arithmetic operations, logical operations, memory access, control flow operations, etc. Due to the independence of the IR instruction set, it is convenient for the compiler to operate on the code at different optimization stages while maintaining enough low-level details to ensure that the generated machine code can be executed efficiently, thereby achieving cross-platform compilation.

[0041] In this application example, the CANN acceleration engine is the hardware and software component responsible for executing AI program computing tasks in the CANN platform. By scheduling the CANN acceleration engine, it is possible to ensure that the AI ​​program to be migrated runs efficiently on the CANN platform. The execution process of the CANN acceleration engine is exactly the same as the operation of the native program on the CANN platform to ensure the compatibility and optimal performance of the program.

[0042] Specifically, in the process of migrating the AI ​​program to be migrated on the source development platform to the target development platform, first, it is necessary to judge the code data type of the AI ​​program to be migrated, and confirm the platform configuration information of the target development platform; then, if the code data of the AI ​​program to be migrated is source code data, the CUDA API call interface in the source code data is extracted, and translated into an API call interface suitable for the target development platform (i.e., the CANN platform), i.e., the first API call interface, and then, on the target development platform, the CANN acceleration engine is controlled to execute the API call interface so that it can run normally on the CANN platform, thereby realizing the migration of the AI ​​program to be migrated from the source development platform to the CANN platform; if the code data of the AI ​​program to be migrated is PTX data, the PTX data is parsed and translated to obtain the IR instruction set suitable for the CANN platform, and then, on the target development platform, the CANN acceleration engine is controlled to execute the IR instruction set so that it can run normally on the CANN platform, thereby realizing the migration of the AI ​​program to be migrated from the source development platform to the CANN platform.

[0043] It can be understood that in this embodiment, the code data of the artificial intelligence AI program to be migrated on the source development platform and the platform configuration information of the target development platform are obtained, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data, intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; when the code data of the AI ​​program to be migrated is source code data, according to the platform configuration information of the CANN platform, multiple general parallel computing architecture development CUDA API call interfaces in the source code data are extracted and converted to obtain multiple first API call interfaces applicable to the CANN platform; when the code data of the AI ​​program to be migrated is PTX data, according to the platform configuration information of the CANN platform, the PTX data is parsed and translated to obtain an intermediate representation IR instruction set applicable to the CANN platform; then, on the target development platform, the CANN acceleration engine is controlled to execute the first API call interface or the IR instruction set to migrate the AI ​​program to be migrated from the source development platform to the CANN platform. It can be seen that this application realizes the cross-platform migration of AI programs by automatically extracting the CUDA API call interface in the AI ​​program source code data and converting it into the CANN API call interface or directly parsing the PTX data of the AI ​​program and translating it into the IR instruction set. The CANN acceleration engine then executes the CANN API call interface or the IR instruction set. This migration method effectively avoids the defects of existing tools such as the need for manual code modification and high learning costs, shields users from complex underlying details, and eliminates the need for users to manually adjust the code, thereby significantly simplifying the migration process, reducing development costs, and improving the efficiency of cross-platform migration.

[0044] In a possible implementation, when the code data of the AI ​​program to be migrated is source code data, multiple CUDA API call interfaces for general parallel computing architecture development in the source code data are extracted and converted according to the platform configuration information of the CANN platform to obtain multiple first API call interfaces applicable to the CANN platform, including: Extracting multiple CUDA API call interfaces in the source code data to obtain multiple second API call interfaces; According to the platform configuration information of the CANN platform and the preset mapping table, the multiple second API calling interfaces are converted to obtain multiple first API calling interfaces applicable to the CANN platform; wherein the preset mapping table is used to characterize the mapping relationship between the CUDA API calling interface and the CANN API calling interface.

[0045] In the present application example, the preset mapping table is a predefined table for representing the correspondence between the CUDA API call interface and the CANN API call interface. The preset mapping table may contain a direct mapping between the CUDA API call interface and the CANN API call interface, or it may contain some conversion rules between the CUDA API call interface and the CANN API call interface. That is, in the present application example, the CUDA API call interface is an API interface applicable to the source development platform (such as the GPU platform), and the CANN API call interface is an API interface applicable to the target development platform (such as the NPU platform).

[0046] Specifically, when the code data of the AI ​​program to be migrated is source code data, the source code data is scanned to identify and extract all CUDA API call interfaces in the source code data, and these CUDA API call interfaces are used as the second API call interface. Afterwards, referring to the configuration information of the target development platform, the API call interface to be converted is determined from the second API call interface, and at the same time, the preset mapping table is used to convert the API call interface to be converted into the corresponding CANN API call interface applicable to the CANN platform, that is, the first API call interface, so that the conversion process can be carried out automatically and ensure that the converted API call interface can meet the hardware and software requirements of the target development platform.

[0047] It should be understood that this example can automatically convert the AI ​​program originally written for the source development platform into code suitable for the CANN platform, thereby realizing cross-platform migration of the AI ​​program.

[0048] In a possible implementation, multiple CUDA API call interfaces in the source code data are extracted to obtain multiple second API call interfaces, including: Through the performance analyzer or tool extension library of the deep learning framework PyTorch, multiple CUDA API call interfaces of the AI ​​program to be migrated during runtime are captured, and the kernel function information of the CUDA API call interface is parsed to obtain multiple second API call interfaces.

[0049] In the present application example, the present application example is mainly aimed at optimizing the ability to extract CUDA API call interfaces from Python programs in various scenarios, especially for the mixed use of CUDA API call interfaces in deep learning framework PyTorch programs, including high-level interfaces provided by torch.cuda and custom CUDA kernels embedded in cpp_extension or cupy. Using the torch.autograd.profiler or tool extension library NVTX tool of the deep learning framework PyTorch, all CUDA API call interfaces when the program is running can be captured, and the CUDA operations and their parameters corresponding to the CUDA API call interface can be identified by parsing the call stack, and the kernel function information of the CUDA API call interface can be parsed at the same time, thereby obtaining a second API call interface. Among them, the kernel function information is a function executed in parallel in CUDA. Generally, the kernel function information includes parameters such as function signature, memory operation, calculation logic, and thread scheduling.

[0050] It should be noted that, in addition, for the exposed CUDA source code, static code analysis tools (such as Clang AST or PyParsing) can be used to extract the definition and implementation of the kernel function of the CUDA API call interface and analyze its code logic and computing mode.

[0051] Compared with the currently commonly used methods for PyTorch programs, this extraction method has greater flexibility and adaptability, especially when dealing with scenarios that mix standard CUDA API and custom CUDA API kernels. It should be understood that this method can more comprehensively capture and analyze the CUDA API call interface in the program, thereby significantly improving the performance optimization and porting efficiency of complex computing programs.

[0052] In a possible implementation, according to the platform configuration information of the CANN platform and the preset mapping table, multiple second API call interfaces are converted to obtain multiple first API call interfaces applicable to the CANN platform, including: Parsing the kernel function information of the second API call interface to obtain multiple parsing results; wherein the kernel function information includes function signature, memory operation, calculation logic and thread scheduling; According to the multiple analysis results and the preset mapping table, the second API calling interface is mapped to obtain the first API calling interface corresponding to the second API calling interface.

[0053] Specifically, by parsing the kernel function information such as function signature, memory operation, calculation logic and thread scheduling of each second API call interface, the corresponding parsing result is obtained, and the parsing result contains detailed information of the kernel function. Afterwards, by parsing the parsing results and the preset mapping table, the CANN API calling interface equivalent to the second API calling interface, that is, the first API calling interface, is found. In this process, if the preset mapping table provides a direct mapping relationship, the equivalent CANN API calling interface can be directly found; if the preset mapping table provides conversion rules, these rules need to be applied to generate an equivalent CANN API calling interface. It should be understood that through the preset mapping table, the CUDA API calling interface is mapped to the corresponding CANN API calling interface to ensure that the AI ​​program to be migrated can run on the target development platform.

[0054] In a specific application example, the CUDA API call interface on the GPU platform can be correctly mapped to the corresponding CANN operator through a preset mapping table, and the corresponding CANN API call interface can be determined to ensure that the AI ​​program on the GPU platform can run seamlessly on the NPU platform.

[0055] Optionally, the kernel function information of the second API call interface is parsed to obtain multiple parsing results, including: Using regular expressions, extract the signature and body code of the global function in the second API call interface, and obtain the input and output variables and their data types of the second API call interface; Parse the data interaction between the host and the device corresponding to the source development platform to obtain the memory operation information of the second API call interface; Analyze the core mathematical operations and data processing in the second API call interface to obtain the key calculation mode of the second API call interface; Capture the parallel computing index logic and related loop control in the second API call interface to obtain the parallel computing granularity and scope of the second API call interface.

[0056] Specifically, the function signature is used as the basis for defining the entry parameters and return values ​​of the kernel function. The signature and main code of the global function are extracted using regular expressions, which can clarify the input and output variables and their data types, laying the foundation for the subsequent definition of the CANN call interface. At the memory operation level, the module parses the data interaction between the host (Host) and the device (Device), covering operations such as video memory allocation (such as cudaMalloc), data transmission (such as cudaMemcpy), and video memory release (such as cudaFree). It can clarify the memory operation information of the interface and ensure the efficiency and accuracy of data management. For the calculation logic, it mainly focuses on key calculation modes such as core mathematical operations (such as matrix multiplication) and data processing (such as normalization and activation functions), and improves performance and compatibility by replacing them with standard operators of the CANN platform. In terms of thread scheduling, by capturing and mapping CUDA's parallel computing index logic (such as blockIdx and threadIdx) and related loop control, the granularity and scope of parallel computing can be accurately controlled.

[0057] It should be understood that by comprehensively parsing the key elements of the kernel function of the second API call interface, including function signature, memory operation, calculation logic and thread scheduling, it can be ensured that the first API call interface can be accurately obtained, thereby accurately completing the migration between architectures.

[0058] In a possible implementation, the PTX data is parsed and translated according to the platform configuration information of the CANN platform to obtain an intermediate representation IR instruction set applicable to the CANN platform, including: According to the platform configuration information of the CANN platform, determine the PTX translator corresponding to the platform configuration information; Through the PTX translator, multiple key information in the PTX data is extracted and converted into an IR instruction set; wherein the key information includes thread scheduling, memory access instructions, calculation instructions and branch control instructions.

[0059] In this application example, a dedicated PTX translator is used to parse and convert PTX data into an IR instruction set that conforms to the CANN platform architecture. Specifically, when parsing and translating PTX data, it is necessary to determine the corresponding PTX translator. The PTX translator is selected based on the configuration information of the target development platform (CANN platform), such as supported hardware architectures, optimization options and other parameters. Different CANN platform configurations require different translators to process PTX data to ensure that an IR instruction set that matches the CANN platform is generated.

[0060] In the process of extracting and translating PTX data through the PTX translator, first of all, the following key information of PTX data needs to be extracted, namely, thread scheduling (such as tid.x, tid.y, blockIdx.x, etc.) for precise control of thread granularity; memory access instructions (such as ld.global, st.global) for managing the loading and storage of global memory; calculation instructions (such as add.f32 and mul.f32) for parsing basic arithmetic operations; branch control instructions (such as setp and bra) for processing conditional settings and branch jump logic. Afterwards, based on these extracted key information, the PTX data is converted into an IR instruction set. The IR instruction set is generated through a dedicated tool chain. For example, a computational graph can be built through a specific interface (scendCL interface), or a custom Tensor Boost Engine (TBE) operator can be used to directly map the logic in the PTX data to a TBE kernel.

[0061] Through the PTX translator, PTX data can be parsed and converted into IR instruction sets that comply with the CANN platform to ensure that AI programs run efficiently on the NPU platform.

[0062] like Figure 2 As shown, Figure 2 This is a flowchart of migrating an AI program from a GPU platform to an NPU platform provided in one embodiment of the present application. Figure 2 Two examples are given of migrating the AI ​​program to be migrated from the GPU platform to the NPU platform, namely, extracting and converting the CUDA API call interface in the source code data to obtain multiple CANN API call interfaces suitable for the CANN platform, or parsing and translating the PTX data to obtain the intermediate representation IR instruction set suitable for the CANN platform; then, controlling the CANN acceleration engine to execute the CANN API call interface or the IR instruction set to migrate the AI ​​program to be migrated from the GPU platform to the NPU platform.

[0063] Furthermore, after obtaining a plurality of first API call interfaces applicable to the CANN platform, the method further includes: According to the first API calling interface, program development is performed on the software development tool CANN SDK to obtain a corresponding user program; On the target development platform, the CANN acceleration engine is controlled to execute the user program to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

[0064] In this application example, the software development tool CANN SDK is a software development kit for the neural network architecture CANN. CANN SDK can provide a series of API interfaces, operator libraries, tools and documents to help developers develop and optimize AI programs efficiently on the CANN platform. The user program is an AI program developed in the CANN SDK according to the first API call interface, which contains the API calls and logic necessary for the AI ​​program to be migrated.

[0065] Alternatively, if Figure 2 As shown in , after obtaining multiple first API call interfaces applicable to the CANN platform, you can also use the software development tools and API interfaces provided by the CANN SDK to rewrite or adjust the code of the AI ​​program to be migrated according to the first API call interface obtained by the aforementioned conversion to obtain the corresponding user program. In this process, the architecture, algorithm, and data flow of the AI ​​program to be migrated may be modified to ensure that the developed user program can be correctly executed on the CANN platform. After that, the user program suitable for the CANN platform is compiled and deployed to the target development platform (ie, the CANN platform), and the AI ​​tasks in the user program are executed through the CANN acceleration engine.

[0066] It should be understood that by making a series of modifications and optimizations to the AI ​​program, a user program that can run correctly on the CANN platform is developed to achieve the migration of the AI ​​program to be migrated from the source development platform to the target development platform, and further ensure the running performance and results on the target development platform.

[0067] In a possible implementation, when the code data is source code data, the method further includes: The source code data is compiled into PTX data by a compiler, so as to obtain an IR instruction set applicable to the CANN platform according to the PTX data.

[0068] In this application example, the compiler is a tool used to convert source code into hardware-specific low-level intermediate code. The compiler can read source code data and compile it into corresponding PTX data.

[0069] Optionally, when the code data of the AI ​​program to be migrated is source code data, the source code data can be first compiled into PTX data by a compiler, and then the PTX data is parsed and translated to obtain an IR instruction set suitable for the CANN platform. Then, the CANN acceleration engine is controlled on the target development platform to execute the IR instruction set so that it can run normally on the CANN platform, thereby realizing the migration of the AI ​​program to be migrated from the source development platform to the CANN platform.

[0070] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0071] Corresponding to the above embodiment, a method for migrating an artificial intelligence program across platforms is provided. Figure 3 A schematic diagram of the structure of an artificial intelligence program cross-platform migration device provided in one embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0072] Reference Figure 3 The artificial intelligence program cross-platform migration device 3 of this embodiment includes: The acquisition module 31 is used to acquire the code data of the AI ​​program to be migrated on the source development platform and the platform configuration information of the target development platform, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data and intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; A first conversion module 32 is used to extract and convert multiple CUDA API call interfaces developed by a general parallel computing architecture in the source code data according to the platform configuration information of the CANN platform when the code data of the AI ​​program to be migrated is source code data, so as to obtain multiple first API call interfaces applicable to the CANN platform; The second conversion module 33 is used to parse and translate the PTX data according to the platform configuration information of the CANN platform when the code data of the AI ​​program to be migrated is PTX data, so as to obtain an intermediate representation IR instruction set applicable to the CANN platform; The execution module 34 is used to control the CANN acceleration engine to execute the first API call interface or IR instruction set on the target development platform to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

[0073] It can be understood that the artificial intelligence program cross-platform migration device 3 provided in this embodiment obtains the code data of the artificial intelligence AI program to be migrated on the source development platform and the platform configuration information of the target development platform through the acquisition module 31, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data, intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; then, when the code data of the AI ​​program to be migrated is source code data, the first conversion module 32 extracts and converts multiple general parallel computing architecture development CUDA API call interfaces in the source code data according to the platform configuration information of the CANN platform to obtain multiple first API call interfaces applicable to the CANN platform; when the code data of the AI ​​program to be migrated is PTX data, the second conversion module 33 parses and translates the PTX data according to the platform configuration information of the CANN platform to obtain an intermediate representation IR instruction set applicable to the CANN platform; then, the execution module 34 controls the CANN acceleration engine to execute the first API call interface or IR instruction set on the target development platform to migrate the AI ​​program to be migrated from the source development platform to the CANN platform. It can be seen that this application realizes the cross-platform migration of AI programs by automatically extracting the CUDA API call interface in the AI ​​program source code data and converting it into the CANN API call interface or directly parsing the PTX data of the AI ​​program and translating it into the IR instruction set. The CANN acceleration engine then executes the CANN API call interface or the IR instruction set. This migration method effectively avoids the defects of existing tools such as the need for manual code modification and high learning costs, shields users from complex underlying details, and eliminates the need for users to manually adjust the code, thereby significantly simplifying the migration process, reducing development costs, and improving the efficiency of cross-platform migration.

[0074] Furthermore, the first conversion module 32 includes: An extraction unit, used for extracting multiple CUDA API call interfaces in the source code data to obtain multiple second API call interfaces; The conversion unit is used to convert multiple second API call interfaces according to the platform configuration information of the CANN platform and a preset mapping table to obtain multiple first API call interfaces applicable to the CANN platform; wherein the preset mapping table is used to characterize the mapping relationship between the CUDA API call interface and the CANN API call interface.

[0075] Furthermore, the extraction unit is specifically used for: Through the performance analyzer or tool extension library of the deep learning framework PyTorch, multiple CUDA API call interfaces of the AI ​​program to be migrated during runtime are captured, and the kernel function information of the CUDA API call interface is parsed to obtain multiple second API call interfaces.

[0076] Furthermore, the conversion unit comprises: A parsing subunit, used to parse the kernel function information of the second API call interface to obtain multiple parsing results; wherein the kernel function information includes function signature, memory operation, calculation logic and thread scheduling; The mapping subunit is used to map the second API calling interface according to the multiple parsing results and the preset mapping table to obtain the first API calling interface corresponding to the second API calling interface.

[0077] Furthermore, the parsing subunit is specifically used for: Using regular expressions, extract the signature and body code of the global function in the second API call interface, and obtain the input and output variables and their data types of the second API call interface; Parse the data interaction between the host and the device corresponding to the source development platform to obtain the memory operation information of the second API call interface; Analyze the core mathematical operations and data processing in the second API call interface to obtain the key calculation mode of the second API call interface; Capture the parallel computing index logic and related loop control in the second API call interface to obtain the parallel computing granularity and scope of the second API call interface.

[0078] Furthermore, the second conversion module 33 includes: The translation determination unit is used to determine the PTX translator corresponding to the platform configuration information according to the platform configuration information of the CANN platform; The translation extraction unit is used to extract multiple key information in the PTX data through the PTX translator, and convert the multiple key information into an IR instruction set; wherein the key information includes thread scheduling, memory access instructions, calculation instructions and branch control instructions.

[0079] Furthermore, the artificial intelligence program cross-platform migration device 3 also includes: A program development subunit, configured to develop a program on a software development tool CANN SDK according to the first API calling interface to obtain a corresponding user program; The first execution subunit is used to control the CANN acceleration engine to execute the user program on the target development platform to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

[0080] Furthermore, the artificial intelligence program cross-platform migration device 3 also includes: The compiling module is used to compile the source code data into PTX data through a compiler, so as to obtain an IR instruction set applicable to the CANN platform according to the PTX data.

[0081] It should be noted that the information interaction, execution process and other contents between the modules in the above-mentioned artificial intelligence program cross-platform migration device 3 are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0082] The present application also provides a terminal device, such as Figure 4 As shown, Figure 4 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. Figure 4 The terminal device 4 of this embodiment includes: a memory 41, a processor 42, and a computer program stored in the memory 41 and executable on the processor 42. When the processor 42 executes the computer program, the steps in any one of the above-mentioned embodiments of the method for cross-platform migration of artificial intelligence programs are implemented.

[0083] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0084] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0086] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0087] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0088] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for cross-platform migration of artificial intelligence programs, characterized in that: include: Obtain code data of the AI ​​program to be migrated on the source development platform and platform configuration information of the target development platform, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data and intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; When the code data of the AI ​​program to be migrated is the source code data, extracting and converting multiple general parallel computing architecture development CUDA API call interfaces in the source code data according to the platform configuration information of the CANN platform to obtain multiple first API call interfaces applicable to the CANN platform; When the code data of the AI ​​program to be migrated is the PTX data, the PTX data is parsed and translated according to the platform configuration information of the CANN platform to obtain an intermediate representation IR instruction set applicable to the CANN platform; On the target development platform, the CANN acceleration engine is controlled to execute the first API call interface or the IR instruction set to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

2. The cross-platform migration method of artificial intelligence program according to claim 1, characterized in that: When the code data of the AI ​​program to be migrated is the source code data, according to the platform configuration information of the CANN platform, multiple general parallel computing architecture development CUDA API call interfaces in the source code data are extracted and converted to obtain multiple first API call interfaces applicable to the CANN platform, including: Extracting the plurality of CUDA API calling interfaces in the source code data to obtain a plurality of second API calling interfaces; According to the platform configuration information of the CANN platform and the preset mapping table, the plurality of the second API calling interfaces are converted to obtain the plurality of the first API calling interfaces applicable to the CANN platform; wherein the preset mapping table is used to characterize the mapping relationship between the CUDA API calling interface and the CANN API calling interface.

3. The cross-platform migration method of artificial intelligence program according to claim 2, characterized in that: The extracting the plurality of CUDA API calling interfaces in the source code data to obtain a plurality of second API calling interfaces includes: Through the performance analyzer or tool extension library of the deep learning framework PyTorch, the multiple CUDA API calling interfaces of the AI ​​program to be migrated during runtime are captured, and the kernel function information of the CUDA API calling interface is parsed to obtain the multiple second API calling interfaces.

4. The cross-platform migration method of artificial intelligence program according to claim 3, characterized in that: The converting the plurality of second API calling interfaces according to the platform configuration information of the CANN platform and the preset mapping table to obtain the plurality of first API calling interfaces applicable to the CANN platform includes: Parsing the kernel function information of the second API call interface to obtain multiple parsing results; wherein the kernel function information includes function signature, memory operation, calculation logic and thread scheduling; According to the plurality of analysis results and the preset mapping table, the second API calling interface is mapped to obtain the first API calling interface corresponding to the second API calling interface.

5. The cross-platform migration method of artificial intelligence program according to claim 4, characterized in that: The kernel function information of the second API call interface is parsed to obtain multiple parsing results, including: Using regular expressions, extract the signature and body code of the global function in the second API call interface, and obtain the input and output variables and their data types of the second API call interface; Parsing the data interaction between the host and the device corresponding to the source development platform to obtain the memory operation information of the second API calling interface; Analyze the core mathematical operations and data processing in the second API call interface to obtain the key calculation mode of the second API call interface; Capture the parallel computing index logic and related loop control in the second API calling interface to obtain the parallel computing granularity and scope of the second API calling interface.

6. The cross-platform migration method of artificial intelligence program according to claim 1, characterized in that: The step of parsing and translating the PTX data according to the platform configuration information of the CANN platform to obtain an intermediate representation IR instruction set applicable to the CANN platform includes: Determine, according to the platform configuration information of the CANN platform, a PTX translator corresponding to the platform configuration information; Through the PTX translator, multiple key information in the PTX data is extracted, and the multiple key information is converted into the IR instruction set; wherein the key information includes thread scheduling, memory access instructions, calculation instructions and branch control instructions.

7. The cross-platform migration method of artificial intelligence program according to claim 1, characterized in that: After obtaining a plurality of first API call interfaces applicable to the CANN platform, the method further includes: According to the first API calling interface, program development is performed on the software development tool CANN SDK to obtain a corresponding user program; On the target development platform, the CANN acceleration engine is controlled to execute the user program to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

8. The cross-platform migration method of artificial intelligence program according to claim 1, characterized in that: When the code data is the source code data, the method further includes: The source code data is compiled into the PTX data by a compiler, so as to obtain the IR instruction set applicable to the CANN platform according to the PTX data.

9. An artificial intelligence program cross-platform migration device, characterized in that: include: An acquisition module is used to acquire code data of the artificial intelligence AI program to be migrated on the source development platform and platform configuration information of the target development platform, wherein the code data of the AI ​​program to be migrated includes at least one of the following: source code data and intermediate code PTX data; the computing architecture of the target development platform is a heterogeneous computing architecture CANN platform; A first conversion module is used to extract and convert multiple general parallel computing architecture development CUDA API call interfaces in the source code data according to the platform configuration information of the CANN platform when the code data of the AI ​​program to be migrated is the source code data, so as to obtain multiple first API call interfaces applicable to the CANN platform; A second conversion module is used to parse and translate the PTX data according to the platform configuration information of the CANN platform when the code data of the AI ​​program to be migrated is the PTX data, so as to obtain an intermediate representation IR instruction set applicable to the CANN platform; An execution module is used to control the CANN acceleration engine to execute the first API call interface or the IR instruction set on the target development platform to migrate the AI ​​program to be migrated from the source development platform to the CANN platform.

10. A terminal device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images