Operator creation method and device, equipment and storage medium

By inputting operator information by users, generating operator configuration information and writing definition files, creating operator projects automatically, solving the problem of high difficulty in developing custom operators, improving efficiency and simplifying the process.

CN119002877APending Publication Date: 2024-11-22CHINA CONSTRUCTION BANK +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411091025.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In the existing technology, the development of customized operators requires high technical requirements, and it is difficult for ordinary users to create complex neural network operators, with high thresholds and complex development processes.

Method used

Provides an operator creation method, generate operator configuration information by inputting operator information by users, identifying and writing operator definition files, creating operator projects, and modifying logic in the development environment to generate custom operators, reducing development Difficulty and threshold.

Benefits of technology

It realizes the automated creation of custom operators, improves operator creation efficiency, reduces users' dependence on syntax and commands, and simplifies the development process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002877A_ABST
    Figure CN119002877A_ABST
Patent Text Reader

Abstract

The invention provides an operator creation method and device, equipment and a storage medium. Relates to the technical field of deep learning. The method comprises the steps of generating operator configuration information according to operator information input by a user under the condition that an operator development environment is installed; operator information identification is conducted on the operator configuration information, user-defined operator information in the operator configuration information is extracted, and an operator definition file is written with the operator information as content; creating an operator project, marking an operator definition file path and a pre-configured operator related information path in the operator project, and modifying operator logic in the operator project; and compiling and running the modified operator project, searching an operator definition file and operator related information based on an operator definition file path and a pre-configured operator related information path, and executing modified operator logic based on the operator definition file and the operator related information to create a user-defined operator. According to the scheme, automatic creation of the custom operator can be completed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to an operator creation method, apparatus, device and storage medium. Background Art

[0002] Operators refer to some algorithm nodes that perform specific functions in a neural network. For example, in a CNN network, a convolution node belongs to a convolution operator. A neural network is composed of layers of operators. However, when using the framework to build a network, it is basically like calling the operator interface like building blocks. In order to improve efficiency, a complex neural network can be created as an interface for calling, that is, it is necessary to create a custom operator.

[0003] Currently, developers need to have relevant custom operator development capabilities, but custom operator development requires very high technical requirements. For the custom operator development process, developers need to master the corresponding syntax and commands, and write some functions and other content in the development process. For ordinary users, creating custom operators is difficult and has a high threshold. Summary of the invention

[0004] The present application provides an operator creation method, apparatus, device and storage medium, which can complete the automatic creation of custom operators according to operator information input by a user, reduce the difficulty of operator creation, and improve the efficiency of operator creation.

[0005] In a first aspect, the present application provides an operator creation method, comprising: when an operator development environment is installed, generating operator configuration information according to operator information input by a user; performing operator information identification on the operator configuration information, extracting operator information customized in the operator configuration information, and writing an operator definition file with the operator information as content; wherein the operator information is used to reflect expression information and input and output information of the customized operator; creating an operator project, marking the operator definition file path and a pre-configured operator-related information path in the operator project, and modifying the operator logic in the operator project; compiling and running the modified operator project, searching for the operator definition file and operator-related information based on the operator definition file path and the pre-configured operator-related information path, and executing the modified operator logic based on the operator definition file and the operator-related information to create a customized operator.

[0006] In a second aspect, the present application provides an operator creation device, including: an operator configuration information generation module, which is used to generate operator configuration information according to operator information input by a user when the operator development environment is installed; an operator definition file generation module, which is used to identify operator information of the operator configuration information, extract operator information customized in the operator configuration information, and write an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the custom operator; an operator project creation module, which is used to create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project; a custom operator creation module, which is used to compile and run the modified operator project, search for the operator definition file and operator related information based on the operator definition file path and the pre-configured operator related information path, and execute the modified operator logic to create a custom operator based on the operator definition file and the operator related information.

[0007] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the operator creation method as described in any embodiment of the present application.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the operator creation method described in any embodiment of the present application when executed.

[0009] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the operator creation method described in any embodiment of the present application.

[0010] The operator creation method, device, equipment and storage medium provided in the present application generate operator configuration information according to the operator information input by the user when the operator development environment is installed; perform operator information recognition on the operator configuration information, extract the operator information customized in the operator configuration information, and write an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator; create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project; compile and run the modified operator project, search for the operator definition file and operator related information based on the operator definition file path and the pre-configured operator related information path, and execute the modified operator logic to create a customized operator based on the operator definition file and the operator related information. The above scheme can complete the automatic creation of customized operators according to the operator information input by the user, reduce the difficulty of operator creation, and improve the efficiency of operator creation. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0012] Figure 1 The process of the operator creation method provided in the embodiment of the present application Figure 1 ;

[0013] Figure 2 The process of the operator creation method provided in the embodiment of the present application Figure 2 ;

[0014] Figure 3 The process of the operator creation method provided in the embodiment of the present application Figure 3 ;

[0015] Figure 4 The process of the operator creation method provided in the embodiment of the present application Figure 4 ;

[0016] Figure 5 A schematic diagram of the operator registration and calling process provided in the embodiment of the present application;

[0017] Figure 6 A flowchart of a specific implementation of the operator creation method provided in an embodiment of the present application;

[0018] Figure 7 A schematic diagram of the structure of an operator creation device provided in an embodiment of the present application;

[0019] Figure 8A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0020] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0021] The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. The acquisition, transmission, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0022] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0023] Terminology explanation:

[0024] Operator: Operator, referred to as OP, is the computing unit that makes up the deep learning algorithm. In the network model, the operator corresponds to the computing logic in the layer. For example, the convolution layer is an operator, and the weight summation process in the fully connected layer is an operator.

[0025] TVM: Tensor Virtual Machine, an open source deep learning compilation stack, connects deep learning models and backend hardware platforms through a unified intermediate expression stack, optimizes scheduling and execution plans through a unified structure, and can support CPU, GPU and specific accelerator platforms and languages.

[0026] Heterogeneous computing architecture platform: A computing system platform that combines computing units using different types of instruction sets and architectures. It supports multiple AI architectures, including MindSpore, PyTorch, TensorFlow, etc., and serves AI processors and programming, playing a key role in connecting the upper and lower levels. At the same time, it provides multi-level programming interfaces for diverse application scenarios, supporting users to quickly build AI applications and businesses.

[0027] AI Core architecture: Different from the traditional CPU (Central Processing Unit) and GPU (graphics processing unit) that support general computing, and also different from the dedicated chip ASIC (Application Specific Integrated Circuit) dedicated to a specific algorithm, the AI ​​Core architecture is essentially designed to adapt to common applications and algorithms in a specific field, and is usually called Domain Specific Architecture (DSA).

[0028] The embodiment of the present application also provides an operator creation method. Figure 1 The process of the operator creation method provided in the embodiment of the present application Figure 1 The embodiment of the present application may be applicable to the case of creating an operator. Typically, the embodiment of the present application may be applicable to the case of automatically creating a custom operator. The method may be executed by an operator creation device, which may be implemented in the form of hardware and / or software, and the operator creation device may be configured in an electronic device. Figure 1 As shown, the operator creation method includes:

[0029] S110: When the operator development environment is installed, generate operator configuration information according to the operator information input by the user.

[0030] Among them, the operator development environment is a software platform that supports the entire process of operator design to registration, and usually includes a series of tools, libraries and frameworks for simplifying the development process of operators. In an embodiment of the present application, before creating a custom operator, it is necessary to complete the installation of the operator development environment, and the installation path can be determined by the user. Among them, the operator development environment includes heterogeneous computing architecture platform software, Python, C++, and the operating environment corresponding to the above software. After the operator development environment is installed, the user can reference the process-level environment variable setting script provided by the combination package of the heterogeneous computing architecture platform software in the process to automatically complete the setting of the environment variables of the operator development environment. The operator configuration information contains the operator information necessary to create the operator, which is used to provide a basis for the subsequent operator creation process.

[0031] In the embodiment of the present application, when the operator development environment is installed, a visual interface will be provided to the user for the user to input the operator information of the custom operator that the user wants to create. Optionally, the visual interface can be a graphical user interface that allows the user to input the operator information by clicking and dragging, or it can be a command line interface that allows the user to input the operator information through command line parameters or interactive prompts.

[0032] In the embodiment of the present application, the operator information input by the user is related to the function that the operator wants to achieve by creating, and may include the operator name used to uniquely identify the operator, the input and output data types of the operator (such as tensors, lists, etc.), the specific type of data processed by the operator (such as integers, floating point numbers, strings, etc.), and the calculation logic used to describe the specific operations or calculation steps performed by the operator (which may be calculation expressions, text descriptions, pseudocodes, or flowcharts). After receiving the operator information input by the user through the visual interface, the operator configuration information can be generated according to the operator information input by the user, and stored in the corresponding file storage path, so that the operator configuration information can be quickly loaded and used in subsequent steps.

[0033] S120, performing operator information identification on the operator configuration information, extracting the operator information customized in the operator configuration information, and writing an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator.

[0034] Among them, the operator definition file is a file used to describe and define the behavior of operators in the deep learning framework, usually including the operator's interface definition, implementation logic, and any necessary metadata information. Generally speaking, the format of the operator definition file is JOSN format.

[0035] In the embodiment of the present application, after the operator configuration information is obtained from the file storage path corresponding to the operator configuration information, the operator configuration information is identified to parse the operator information such as the expression of the custom operator, the operator name, the number of inputs and outputs, the data type of the inputs and outputs, and any specific parameters related to the operator function that may be included in the operator configuration information, and the operator definition file is written according to the operator information. It should be noted that when the operator definition file is written according to the operator information in the operator configuration information, the generation of the operator definition function is also included.

[0036] In an embodiment of the present application, the operator configuration information is identified as operator information, the operator information customized in the operator configuration information is extracted, and an operator definition file is written with the operator information as content, including:

[0037] Identify the operator configuration information, extract the expression information and input-output information of the operator; and generate a shape reasoning function and a dimension verification function in the operator definition file according to the expression information and the input-output information.

[0038] Among them, the shape reasoning InterShape function and the dimension verification Verify function are operator definition functions. The shape reasoning function is used to automatically deduce the shape of the output tensor of the operator, and the dimension verification function is used to verify the internal relationship of the operator. For example, for multi-input operators, the data types of multiple inputs are verified to ensure that they remain consistent.

[0039] In an embodiment of the present application, the operator configuration information can be identified first, the expression information and input and output information of the operator can be extracted therefrom, and then the shape reasoning function and dimension verification function in the operator definition file can be generated according to the determined expression information and input and output information. Exemplarily, the operator configuration information is identified, and the expression information of the operator is obtained as C=A+B, including two inputs and one output. The data types supported by the two inputs and one output are both half-precision floating point float16 types, and the data arrangement format is NCHW format, requiring the shapes of the two inputs to be the same. At this time, the shape reasoning function can use the shapes of the two inputs of the operator as the input of the function, and when the shapes of the two inputs of the operator are the same, the shape of any input is output as the shape of the output of the operator. The dimension verification function can use the data type of the input of the operator and the data type of the output of the operator as input, and when the data type of the input of the operator and the data type of the output of the operator are inconsistent, the verification failure is output as the result.

[0040] In the embodiment of the present application, an operator definition file is written with the operator information as the content, including:

[0041] A class object defining a calling operator is generated through an operator definition function interface; and the class object is called to generate the operator definition file according to the operator information.

[0042] In an embodiment of the present application, the operator definition function interface can automatically generate an object OP of the type of calling operator Operator class. By calling the object OP of the calling operator Operator class, the generation of the shape reasoning function in the operator definition file can be realized, without the need for the user to master the relevant syntax and C++ commands to write the shape reasoning function. At the same time, in an embodiment of the present application, when generating a dimension verification function, it is only necessary to configure the dimension to be verified, and the user also does not need to master the relevant syntax and C++ commands to write the dimension verification function. Compared with the prior art, the difficulty and threshold of operator creation are greatly reduced.

[0043] S130: Create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project.

[0044] Among them, regarding the creation of an operator project, in an embodiment of the present application, after configuring the installation location of the heterogeneous computing architecture platform software, by executing a Python script, an operator project can be created based on the operator definition file and the configured operator-related information, without the need for the user to execute shell commands to create the operator project, thereby reducing the user's learning cost for the shell and improving the efficiency of operator creation. In an embodiment of the present application, an operator project is created, and the operator definition file path and the pre-configured operator-related information path are marked in the operator project, including:

[0045] Configure the operator parameter configuration information in the operator related information, wherein the operator parameter configuration information includes input parameter configuration information, output parameter configuration information, value range, operator attribute configuration information and operator definition function configuration information; configure the operator path configuration information in the operator related information, wherein the operator path configuration information includes the file storage path after the operator development environment is installed, the operator definition file path, the computing resource path used by the operator and the path where the generated file is located; create the operator project according to the operator definition file, the operator parameter configuration information and the operator path configuration information.

[0046] Among them, the input parameter configuration information includes the input parameter name, input parameter type, and data arrangement format. The output parameter configuration information includes the output parameter name, output parameter type, and data arrangement format. The operator attribute configuration information includes the name of the operator attribute parameter and the attribute parameter type. The value range includes the value range of the data arrangement format, the value range of the input parameter type, the value range of the output parameter type, and the value range of the attribute parameter type. Among them, the value range of the data arrangement format includes ND, NHWC, NCHW, HWCN, NC1HWC0, FRACTAL_Z, etc., and the value range of the input parameter type and the output parameter type includes half-precision floating point number float16, single-precision floating point number float32, eight-bit integer int8, sixteen-bit integer int16, thirty-two-bit integer int32, unsigned eight-bit integer uint8, unsigned sixteen-bit integer uint16, Boolean bool, etc. The value range of the attribute parameter type includes integer int, Boolean bool, floating point number float, string string, integer list list_int, floating point list list_float, etc. The operator definition function configuration information includes shape reasoning function configuration information and dimension verification function configuration information. The shape reasoning function configuration information defaults to the description information of the output tensor being the same as the description information of the input tensor. The dimension verification function configuration information defaults to verifying whether the data types of the input and output parameters of the operator are consistent. If they are inconsistent, failure is returned.

[0047] The file storage path after the operator development environment is installed is the file storage path after the heterogeneous computing architecture platform software is installed. The operator definition file path can be configured as an absolute path or a relative path, and the user needs to have read permission for this path. The path where the generated file is located can be configured as an absolute path or a relative path, and the user needs to have read permission for this path. If no configuration is performed, the file is generated by default in the current path of the Python script.

[0048] In an embodiment of the present application, after creating an operator project according to the operator definition file and the configured operator-related information by executing a Python script, it is also necessary to mark the operator definition file path and the pre-configured operator-related information path in the operator project, and modify the operator logic in the operator project according to the calculation logic in the operator configuration information so that the created operator meets user needs.

[0049] In the embodiment of the present application, the operator logic in the operator project is modified, including:

[0050] Search the core file in the directory of the operator project to determine the operator code framework generated in the core file; and modify the operator calculation function in the algorithm code framework according to the calculation logic of the operator.

[0051] The core file includes the core code of the operator, which is generated during the creation of the operator project and is stored in the file path under the operator project directory.

[0052] In the embodiment of the present application, the core file can be retrieved in the directory where the operator project is created, the operator code framework of the core code of the operator in the core file can be determined, and the operator calculation function used to implement the operator function in the operator code framework can be located. After that, according to the calculation logic of the operator in the expression information in the operator configuration information, the operator calculation function is modified to complete the modification of the operator logic in the operator project to meet the actual needs of the user.

[0053] S140: compile and run the modified operator project, search for the operator definition file and operator related information based on the operator definition file path and the pre-configured operator related information path, and execute the modified operator logic to create a custom operator based on the operator definition file and the operator related information.

[0054] In an embodiment of the present application, the automated build and compilation scripts can be searched in the operator project, the modified operator project can be compiled and run, the operator definition file and operator-related information can be searched based on the operator definition file path and the pre-configured operator-related information path, and the modified operator logic can be executed based on the operator definition file and the operator-related information to complete the creation of the custom operator.

[0055] In the embodiment of the present application, compiling and running the modified operator project includes:

[0056] Search for the automated build and compilation scripts in the directory of the operator project, and configure the environment variables required for operator compilation; execute the compilation command to compile the operator project; create an operator directory, generate a custom operator installation package in the operator directory, enter the operator directory, and install the custom operator installation package.

[0057] The automated build and compile script, namely the build.sh script, can be modified to configure the environment variables required for operator compilation. Specifically, the environment variable ASCEND_TENSOR__COMPILER_INCLUDE in build.sh can be configured to the path where the software header files of the heterogeneous computing architecture platform are located.

[0058] In the embodiment of the present application, the automatic construction and compilation script is searched in the directory of the operator project. After configuring the environment variables required for operator compilation, the compilation command is executed in the directory of the operator project to enter the operator project compilation. After the compilation is successful, an operator directory, namely the build_out directory, is created in the current directory, and a custom operator installation package custom_opp_ is generated in the build_ou directory.<target os> _<target architecture> .run, enter the build_out directory, execute the installation command of the custom operator installation package, and deploy the relevant files of the custom operator installation package to the operator library of the heterogeneous computing architecture platform software.

[0059] The scheme in the embodiment of the present application, when the operator development environment is installed, generates operator configuration information according to the operator information input by the user; performs operator information recognition on the operator configuration information, extracts the operator information customized in the operator configuration information, and writes an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator; creates an operator project, marks the operator definition file path and the pre-configured operator related information path in the operator project, and modifies the operator logic in the operator project; compiles and runs the modified operator project, searches for the operator definition file and the operator related information based on the operator definition file path and the pre-configured operator related information path, and executes the modified operator logic to create a customized operator based on the operator definition file and the operator related information. The above scheme can complete the automatic creation of customized operators according to the operator information input by the user, thereby improving the efficiency of operator creation.

[0060] Figure 2 The process of the operator creation method provided in the embodiment of the present application Figure 2 .like Figure 2 As shown, the operator creation method includes the following steps:

[0061] S210: When the operator development environment is installed, generate operator configuration information according to the operator information input by the user.

[0062] S220, perform operator information identification on the operator configuration information, extract the operator information customized in the operator configuration information, and write an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator.

[0063] S230: Create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project.

[0064] S240: compile and run the modified operator project, search for the operator definition file and operator related information based on the operator definition file path and the pre-configured operator related information path, and execute the modified operator logic to create a custom operator based on the operator definition file and the operator related information.

[0065] S250: Testing the custom operator based on the test case to test the performance of the custom operator.

[0066] Among them, the test case is a document written according to the test requirements and test design to guide the test execution and evaluate the test results, which clearly specifies the test objectives, test environment, test data, test steps and expected results. In the embodiment of the present application, in order to ensure the accuracy of the functions of the generated custom operator, it is necessary to test the created custom operator based on the test case.

[0067] In the embodiment of the present application, the custom operator is tested based on the test case, including:

[0068] Create a system test case definition file, and fill and modify the fields in the system test case definition file according to the operator information; generate a test case according to the system test case definition file through an operator testing tool, and execute the test case to verify the function of the custom operator.

[0069] To create a system test case definition file, you need to first define the structure of the system test case. This structure should contain all necessary fields to describe various aspects of the test case, such as the test case name, custom operator information, input data, expected output, etc. After that, you can create a system test case definition file based on the defined system test case structure and fill the system test case definition file with operator information.

[0070] In an embodiment of the present application, after the system test case definition file is created, the system test case definition file can be parsed by the operator testing tool, and the actual test input and verification logic can be constructed based on the information therein, a specific test case is generated, the function of the custom operator is verified, and the corresponding test results are fed back. For example, when the test passes, a prompt message about the success of the test is fed back; when the test fails, a prompt message about the failure of the test is fed back, and the reason for the test failure is indicated in the prompt message.

[0071] The scheme in the embodiment of the present application, when the operator development environment is installed, generates operator configuration information according to the operator information input by the user; identifies the operator configuration information, extracts the operator information customized in the operator configuration information, and writes the operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the custom operator; creates an operator project, marks the operator definition file path and the pre-configured operator related information path in the operator project, and modifies the operator logic in the operator project; compiles and runs the modified operator project, searches for the operator definition file and the operator related information based on the operator definition file path and the pre-configured operator related information path, and executes the modified operator logic to create a custom operator based on the operator definition file and the operator related information; tests the custom operator based on the test case to test the performance of the custom operator. The above scheme can complete the automatic creation of the custom operator according to the operator information input by the user, and improves the efficiency of operator creation. At the same time, after the custom operator is created, the custom operator is tested based on the test case to ensure the accuracy of the function of the created custom operator.

[0072] Figure 3 The process of the operator creation method provided in the embodiment of the present application Figure 3 .like Figure 3 As shown, the operator creation method includes the following steps:

[0073] S310: When the operator development environment is installed, generate operator configuration information according to the operator information input by the user.

[0074] S320, perform operator information identification on the operator configuration information, extract the operator information customized in the operator configuration information, and write an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator.

[0075] S330: Create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project.

[0076] S340: compile and run the modified operator project, search for the operator definition file and operator related information based on the operator definition file path and the pre-configured operator related information path, and execute the modified operator logic to create a custom operator based on the operator definition file and the operator related information.

[0077] S350: Adapt the custom operator to the heterogeneous environment so that the custom operator can be applicable to the heterogeneous environment.

[0078] Among them, the heterogeneous environment may involve different hardware architectures, such as CPU, GPU, FPGA, etc., and may also involve different software environments, such as different operating systems, deep learning framework versions, etc. Adapting the custom operator to the heterogeneous environment can enable the custom operator to run effectively in different hardware and software environments. In the embodiment of the present application, the heterogeneous environment adaptation of the custom operator mainly refers to the adaptation of the custom operator to the deep learning framework PyTorch.

[0079] In the embodiment of the present application, the custom operator is adapted to a heterogeneous environment, including:

[0080] Generate a source code format file corresponding to the heterogeneous environment according to the written code of the custom operator; download the heterogeneous environment source code, configure the mapping function of the custom operator, and add the source code format file to the heterogeneous environment source code; compile the heterogeneous environment source code with the source code format file added, and install the custom operator.

[0081] Among them, the source code format file corresponding to the heterogeneous environment is generated according to the writing code of the custom operator, and the writing code of the custom operator can be referenced to generate the source code format file of the custom operator suitable for the heterogeneous environment. Exemplarily, the source code format file corresponding to the heterogeneous environment PyTorch is a cpp file, then the source code format file corresponding to the heterogeneous environment is generated according to the writing code of the custom operator, and the content of the writing code of the custom operator can be referred to to generate a code file in the cpp file format with the same function as the writing code of the custom operator, so as to make it suitable for the heterogeneous environment PyTorch.

[0082] In an embodiment of the present application, after generating a source code format file corresponding to a heterogeneous environment, the source code of the heterogeneous environment is downloaded, and by searching or creating a mechanism for registering a custom operator in the source code of the heterogeneous environment, the mapping function of the custom operator is configured to ensure that the custom operator can be correctly identified and executed by the heterogeneous environment. Finally, the source code format file is added to the heterogeneous environment source code, the heterogeneous environment source code with the source code format file added is compiled, and the custom operator is installed. Optionally, after the custom operator is installed, the running effect of the custom operator in the heterogeneous environment can be tested based on the test case.

[0083] The scheme in the embodiment of the present application, when the operator development environment is installed, generates operator configuration information according to the operator information input by the user; identifies the operator configuration information, extracts the operator information customized in the operator configuration information, and writes the operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator; creates an operator project, marks the operator definition file path and the pre-configured operator related information path in the operator project, and modifies the operator logic in the operator project; compiles and runs the modified operator project, searches for the operator definition file and the operator related information based on the operator definition file path and the pre-configured operator related information path, and executes the modified operator logic to create a customized operator based on the operator definition file and the operator related information; and adapts the customized operator to a heterogeneous environment so that the customized operator can be applied to a heterogeneous environment. The above scheme can complete the automatic creation of customized operators according to the operator information input by the user, thereby improving the efficiency of operator creation. At the same time, custom operators are adapted to heterogeneous environments to ensure that they can run effectively in different hardware and software environments.

[0084] Figure 4 The process of the operator creation method provided in the embodiment of the present application Figure 4 .like Figure 4 As shown, the operator creation method includes the following steps:

[0085] S410: When the operator development environment is installed, generate operator configuration information according to the operator information input by the user.

[0086] S420: Perform operator information identification on the operator configuration information, extract the operator information customized in the operator configuration information, and write an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator.

[0087] S430: Create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project.

[0088] S440: Compile and run the modified operator project, search for the operator definition file and operator related information based on the operator definition file path and the pre-configured operator related information path, and execute the modified operator logic to create a custom operator based on the operator definition file and the operator related information.

[0089] S450: Load the custom operator file corresponding to the operating system from the directory of the operator library.

[0090] The operator library refers to the operator library of the heterogeneous computing architecture platform software, which contains the relevant files of the custom operator. In the embodiment of the present application, the operator prototype library management module loads the shared object so file of the custom operator corresponding to the operating system and architecture from the directory of the operator library of the heterogeneous computing architecture platform software.

[0091] S460. Register basic information, shape reasoning function, and dimension verification function in the operator database according to the information in the custom operator file.

[0092] In an embodiment of the present application, after loading the shared object so file of the custom operator, the operator prototype library management module can register the basic information, shape reasoning function and dimension verification function of the custom operator in the operator database OperatorFactory according to the information in the shared object so file of the custom operator, and use the operator type OpType as the key during registration.

[0093] S470, traverse each target operator node in the network model topology diagram, and each target operator node sends a target operator call request to the operator execution node.

[0094] The network model topology is obtained by the graph engine GE (Graph Engine) after receiving the original network model of the third-party framework and initializing it. The target operator call request includes a shape reasoning function call request and a dimension verification function call request.

[0095] In an embodiment of the present application, in the graph preparation stage, the graph engine GE sends a request to call a shape reasoning function and a dimension verification function to the network model topology graph Graph. After traversing each target operator node Node in the network model topology graph Graph, each target operator node Node sends a target operator call request including a shape reasoning function call request and a dimension verification function call request to the operator execution node OpDesc.

[0096] S480 . Obtain a shape reasoning function and a dimension verification function from an operator database through an operator execution node, and execute the shape reasoning function and the dimension verification function to obtain operator information of a target operator.

[0097] In an embodiment of the present application, after receiving a target operator call request, the operator execution node obtains the shape reasoning function and dimension verification function of the target operator from the operator database, and executes the shape reasoning function and dimension verification function to obtain the operator information of the target operator. Figure 5 A schematic diagram of the process of operator registration and calling is shown. In the figure, GE is the graph engine, Graph is the network model topology diagram, Node is the target operator node, OpDesc ​​is the operator execution node, OperatorFactory is the operator database, InterShape is the shape reasoning function, and Verify is the dimension verification function. The registration and calling process of the custom operator is shown in the figure.

[0098] The scheme in the embodiment of the present application, when the operator development environment is installed, generates operator configuration information according to the operator information input by the user; identifies the operator configuration information, and generates an operator definition file according to the operator information in the operator configuration information; creates an operator project according to the operator definition file and the configured operator related information, and modifies the operator logic in the operator project; compiles and runs the modified operator project to create a custom operator; loads the custom operator file corresponding to the operating system from the directory of the operator library; registers basic information, shape reasoning function registration, and dimension verification function registration in the operator database according to the information in the custom operator file; traverses each target operator node in the network model topology diagram, and each target operator node sends a target operator call request to the operator execution node; obtains the shape reasoning function and dimension verification function from the operator database through the operator execution node, and executes the shape reasoning function and dimension verification function to obtain the operator information of the target operator. The above scheme can complete the automatic creation of custom operators according to the operator information input by the user, reduces the difficulty of operator creation, and improves the efficiency of operator creation. At the same time, by registering custom operators, the flexibility of calling custom operators is improved.

[0099] This application embodiment provides a specific implementation of an operator creation method. Figure 6 This is a flowchart of a specific implementation method of the operator creation method provided in the embodiment of the present application, such as Figure 6 As shown, specifically:

[0100] Before developing custom operators, prepare the development environment, install CANN and configure related environments, and install Python and C++ related environments. The CANN package provides a process-level environment variable setting script for users to reference in the process to automatically complete the environment variable setting.

[0101] Analyze the operator information entered by the user:

[0102] Specify the mathematical expression of the operator;

[0103] Specify input and output: If an operator has two inputs and one output, the supported data types are all float16, the data arrangement format is NCHW, and the shapes of the two inputs are required to be the same;

[0104] Specify the name of the operator implementation file;

[0105] Specify the name of the operator implementation function;

[0106] Specify the operator type;

[0107] Clarify the implementation of the InferShape function: The operator definition function interface can automatically generate an object OP of the Operator class type. Developers can directly call the Operator class interface to implement the InferShape function;

[0108] Clarify the implementation of the verify function: The verify function mainly verifies the internal relationship between operators. For example, for multi-input operators, the dtype of multiple tensors needs to be consistent. In this case, the dtype of multiple inputs needs to be verified. In other cases, the dtype does not need to be verified.

[0109] Based on the analysis results of the operator information, an operator definition file is generated, which contains:

[0110] Operator name (op);

[0111] Configuration information of input parameter (input_desc): name (name), parameter type (format), data type arrangement, where the data type arrangement includes the following values: ND, NHWC, NCHW, HWCN, NC1HWC0, FRACTAL_Z, etc., and the parameter type includes the following values: float16 (fp16), float32 (fp32), int8, int16, int32, uint8, uint16, bfloat16 (bf16), bool, etc.;

[0112] Configuration information of output parameter (output_desc): name (name), parameter type (format), data type arrangement, where the data type arrangement includes the following values: ND, NHWC, NCHW, HWCN, NC1HWC0, FRACTAL_Z, etc. Parameter type includes the following values: float16 (fp16), float32 (fp32), int8, int16, int32, uint8, uint16, bfloat16 (bf16), bool, etc.;

[0113] Attribute (attr) configuration information: the name of the operator attribute parameter (name), parameter type (param_type), the type of the operator parameter includes the following values: int, bool, float, string, list_int, list_float, etc.;

[0114] Configuration information of the infershape function: By default, the description information of the output Tensor is the same as the description information of the input Tensor;

[0115] Verify function configuration information: By default, it verifies whether the data types of the operator's input and output parameters are consistent. If they are inconsistent, it returns failure.

[0116] Create an operator project:

[0117] Configure the operator parameter configuration information in the operator definition file;

[0118] Configure the file storage path after CANN software installation;

[0119] Configure the path of the operator definition file: it can be configured as an absolute path or a relative path. The user needs to have the read permission for this path.

[0120] Configure the computing resource path used by the operator;

[0121] Configure the path where the generated file is located: it can be configured as an absolute path or a relative path, and the user has read and write permissions. If not configured, it will be generated by default in the current path where the command is executed;

[0122] Create an operator project based on the operator definition file, operator parameter configuration information, and operator path configuration information.

[0123] Core code modification:

[0124] In the generated operator project directory, there are core files in tbe / impl / . Based on the analysis of the operator, the operator logic in the core code can be modified. The framework of the operator code can be determined in the tbe / impl / add_dsl.py file, and the operator calculation function in this file can be modified to modify the operator logic.

[0125] Operator compilation and deployment:

[0126] In the operator project directory, modify the build.sh script and configure the environment variables required for operator compilation. Configure the environment variable ASCEND_TENSOR__COMPILER_INCLUDE in build.sh to the path where the CANN software header file is located. After the modification is completed, execute the compile command in the operator project directory to compile the operator project. After successful compilation, a build_out directory will be created in the current directory, and a custom operator installation package will be generated in the build_out directory. Enter the build_out directory in the operator project directory and execute the installation command of the custom operator installation package to install the custom operator. After the command is executed successfully, the relevant files in the custom operator installation package are deployed to the CANN operator library.

[0127] ST test:

[0128] Create an ST test case definition file "*.json" to generate test cases. The main function of the ST test case is to use the AscendCL interface to load a single operator model file and execute it. After execution, a prompt message similar to "Case xxxx runsuccessfully." appears, indicating that the ST test case is generated and tested successfully.

[0129] PyTorch adaptation:

[0130] By analyzing the operator, write code to generate the pytorch mapping cpp file, download the pytorch source code, configure the custom operator mapping function in the op_plugin / config / v2r1 / op_plugin_functions.yaml file, and finally copy the generated cpp code to op_plugin / ops / base_ops / . Recompile pytorch, install and deploy the generated torch-npu package, and complete the pytorch adaptation.

[0131] Figure 7 This is a schematic diagram of the structure of the operator creation device provided in the embodiment of the present application. Figure 7As shown, the operator creation device includes: an operator configuration information generation module 510, an operator definition file generation module 520, an operator project creation module 530 and a custom operator creation module 540;

[0132] The operator configuration information generating module 510 is used to generate operator configuration information according to the operator information input by the user when the operator development environment is installed.

[0133] An operator definition file generation module 520 is used to identify the operator information of the operator configuration information, extract the operator information customized in the operator configuration information, and write an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator;

[0134] An operator project creation module 530 is used to create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project;

[0135] The custom operator creation module 540 is used to compile and run the modified operator project, search for the operator definition file and operator-related information based on the operator definition file path and the pre-configured operator-related information path, and execute the modified operator logic to create a custom operator based on the operator definition file and the operator-related information.

[0136] In some embodiments, the operator definition file generation module 520 performs operator information identification on the operator configuration information, extracts the operator information customized in the operator configuration information, and writes an operator definition file with the operator information as the content, specifically including: identifying the operator configuration information, extracting the expression information and input-output information of the operator; generating a shape reasoning function and a dimension verification function in the operator definition file according to the expression information and the input-output information.

[0137] In some embodiments, the operator definition file generation module 520 writes an operator definition file with the operator information as content, specifically including: generating a class object that defines a call operator through an operator definition function interface; calling the class object to generate the operator definition file according to the operator information.

[0138] In some embodiments, the operator project creation module 530 creates an operator project, and marks the operator definition file path and the pre-configured operator related information path in the operator project, specifically including: configuring the operator parameter configuration information in the operator related information, wherein the operator parameter configuration information includes input parameter configuration information, output parameter configuration information, value range, operator attribute configuration information and operator definition function configuration information; configuring the operator path configuration information in the operator related information, wherein the operator path configuration information includes the file storage path after the operator development environment is installed, the operator definition file path, the computing resource path used by the operator and the path where the generated file is located; creating the operator project according to the operator definition file, the operator parameter configuration information and the operator path configuration information.

[0139] In some embodiments, the operator project creation module 530 modifies the operator logic in the operator project, specifically including: searching for core files in the directory of the operator project, determining the operator code framework generated in the core files; and modifying the operator calculation function in the algorithm code framework according to the calculation logic of the operator.

[0140] In some embodiments, the custom operator creation module 540 compiles and runs the modified operator project, specifically including: searching for automated build and compilation scripts in the directory of the operator project, and configuring the environment variables required for operator compilation; executing compilation commands to compile the operator project; creating an operator directory, generating a custom operator installation package in the operator directory, and entering the operator directory to install the custom operator installation package.

[0141] In some embodiments, the apparatus further includes: a custom operator testing module, configured to test the custom operator based on a test case to test the performance of the custom operator.

[0142] In some embodiments, the custom operator testing module tests the custom operator based on the test case, specifically including: creating a system test case definition file, and filling and modifying the fields in the system test case definition file according to the operator information; generating a test case according to the system test case definition file through an operator testing tool, and executing the test case to verify the function of the custom operator.

[0143] In some embodiments, the device further includes: a custom operator adaptation module, configured to perform heterogeneous environment adaptation on the custom operator so that the custom operator can be applicable to the heterogeneous environment.

[0144] In some embodiments, the custom operator adaptation module adapts the custom operator to a heterogeneous environment, specifically including: generating a source code format file corresponding to the heterogeneous environment based on the written code of the custom operator; downloading the heterogeneous environment source code, configuring the mapping function of the custom operator, and adding the source code format file to the heterogeneous environment source code; compiling the heterogeneous environment source code with the added source code format file, and installing the custom operator.

[0145] In some embodiments, the device also includes: a custom operator file loading module, used to load the custom operator file corresponding to the operating system from the directory of the operator library; a custom operator registration module, used to register basic information, shape reasoning function registration and dimension verification function registration in the operator database according to the information in the custom operator file.

[0146] In some embodiments, the device also includes: a target operator call request sending module, which is used to traverse each target operator node in the network model topology diagram, and each target operator node sends a target operator call request to the operator execution node; a target operator information acquisition module, which is used to obtain a shape reasoning function and a dimension verification function from an operator database through the operator execution node, and execute the shape reasoning function and the dimension verification function to obtain the operator information of the target operator.

[0147] The operator creation device provided in the embodiment of the present application can be used to execute the technical solution of the operator creation method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0148] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the operator configuration information generation module 510 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above operator configuration information generation module 510. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0149] Figure 8This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0150] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the scheme in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0151] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0152] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0153] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0154] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0155] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the operator creation method in the above embodiment.

[0156] The embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the operator creation method of the above embodiment.

[0157] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the operator creation method in the above embodiment can be implemented.

[0158] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0159] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for creating an operator, characterized in that: include: When the operator development environment is installed, the operator configuration information is generated according to the operator information entered by the user; Performing operator information identification on the operator configuration information, extracting operator information customized in the operator configuration information, and writing an operator definition file with the operator information as content; wherein the operator information is used to reflect expression information and input and output information of the customized operator; Create an operator project, mark the operator definition file path and pre-configured operator related information path in the operator project, and modify the operator logic in the operator project; The modified operator project is compiled and run, the operator definition file and operator related information are searched based on the operator definition file path and the pre-configured operator related information path, and the modified operator logic is executed based on the operator definition file and the operator related information to create a custom operator.

2. The method according to claim 1, characterized in that Performing operator information identification on the operator configuration information, extracting operator information customized in the operator configuration information, and writing an operator definition file with the operator information as content, including: Identify the operator configuration information, and extract the expression information and input and output information of the operator; A shape reasoning function and a dimension verification function in an operator definition file are generated according to the expression information and the input and output information.

3. The method according to claim 1, characterized in that An operator definition file is written with the operator information as the content, including: Generate a class object that defines a call operator through the operator definition function interface; The class object is called to generate the operator definition file according to the operator information.

4. The method according to claim 1, characterized in that: Create an operator project, and mark the operator definition file path and the pre-configured operator related information path in the operator project, including: Configure operator parameter configuration information in the operator related information, wherein the operator parameter configuration information includes input parameter configuration information, output parameter configuration information, value range, operator attribute configuration information and operator definition function configuration information; Configure the operator path configuration information in the operator related information, wherein the operator path configuration information includes the file storage path after the operator development environment is installed, the operator definition file path, the computing resource path used by the operator, and the path where the generated file is located; The operator project is created according to the operator definition file, the operator parameter configuration information, and the operator path configuration information.

5. The method according to claim 1, characterized in that Modify the operator logic in the operator project, including: Searching for a core file in the directory of the operator project, and determining an operator code framework generated in the core file; According to the calculation logic of the operator, the operator calculation function is modified in the algorithm code framework.

6. The method according to claim 1, characterized in that Compiling and running the modified operator project includes: Search for the automated build and compile scripts in the directory of the operator project, and configure the environment variables required for operator compilation; Execute the compile command to compile the operator project; Create an operator directory, generate a custom operator installation package in the operator directory, enter the operator directory, and install the custom operator installation package.

7. The method according to any one of claims 1 to 6, characterized in that After compiling and running the modified operator project, the method further includes: The custom operator is tested based on the test case to test the performance of the custom operator.

8. The method according to claim 7, characterized in that Test custom operators based on test cases, including: Creating a system test case definition file, and filling and modifying fields in the system test case definition file according to the operator information; The operator testing tool generates a test case according to the system test case definition file, and executes the test case to verify the function of the custom operator.

9. The method according to claims 1-6, characterized in that: After compiling and running the modified operator project, the method further includes: The custom operator is adapted to the heterogeneous environment so that the custom operator can be applicable to the heterogeneous environment.

10. The method according to claim 9, characterized in that The custom operator is adapted to a heterogeneous environment, including: Generate source code format files corresponding to heterogeneous environments based on the code written by custom operators; Download the heterogeneous environment source code, configure the mapping function of the custom operator, and add the source code format file to the heterogeneous environment source code; Compile the heterogeneous environment source code with the source code format file added, and install the custom operator.

11. The method according to claim 1, characterized in that: After compiling and running the modified operator project, the method further includes: Load the custom operator file corresponding to the operating system from the operator library directory; Basic information registration, shape reasoning function registration and dimension verification function registration are performed in the operator database according to the information in the custom operator file.

12. The method according to claim 11, characterized in that After registering basic information, shape reasoning function and dimension verification function in the operator database according to the information in the custom operator file, the method further includes: Traverse each target operator node in the network model topology graph, and each target operator node sends a target operator call request to the operator execution node; The shape reasoning function and the dimension verification function are obtained from the operator database through the operator execution node, and the shape reasoning function and the dimension verification function are executed to obtain the operator information of the target operator.

13. An operator creation device, characterized in that: include: The operator configuration information generation module is used to generate operator configuration information according to the operator information input by the user when the operator development environment is installed; An operator definition file generation module, used to identify the operator information of the operator configuration information, extract the operator information customized in the operator configuration information, and write an operator definition file with the operator information as the content; wherein the operator information is used to reflect the expression information and input and output information of the customized operator; An operator project creation module, used to create an operator project, mark the operator definition file path and the pre-configured operator related information path in the operator project, and modify the operator logic in the operator project; The custom operator creation module is used to compile and run the modified operator project, search for the operator definition file and operator related information based on the operator definition file path and the pre-configured operator related information path, and execute the modified operator logic to create a custom operator based on the operator definition file and the operator related information.

14. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the operator creation method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the operator creation method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the operator creation method according to any one of claims 1 to 12.