Operator registration methods and related products

Abstract

This application discloses an operator registration method and related products. The method includes: compiling a registration configuration file to obtain a class corresponding to a target operator; adding the class corresponding to the target operator and an inference function to a registry; the inference function is used to infer the shape of the output of the target operator. In this application embodiment, compiling the registration configuration file is sufficient to obtain the class corresponding to the target operator, eliminating the need to define a class for the operator, thus improving the efficiency of adding operators to the computing library and simplifying the operation.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to an operator registration method and related products. Background Technology

[0002] The number of operators in a computational library is a crucial indicator of its computational power. Currently, adding an operator to a computational library is inefficient. Therefore, it is necessary to research more efficient methods for adding operators to computational libraries. Summary of the Invention

[0003] This application discloses an operator registration method and related products, which can solve the technical problem of low efficiency in adding operators to a computing library.

[0004] In a first aspect, embodiments of this application provide a method for registering an operator, the method comprising: compiling a registration configuration file to obtain a class corresponding to a target operator; adding the class corresponding to the target operator and an inference function to a registry; the inference function being used to infer the shape of the output of the target operator.

[0005] The current approach to adding operators to the computation library involves: 1. Defining a class for the operator, used for dispatching to different computation backends; 2. Adding the class defined in step 1 to the registry; 3. Adding corresponding tests for the operator. In other words, users need to define the operator class. In this application, the class corresponding to the target operator can be obtained by compiling the registration configuration file, i.e., the operator class is automatically generated, eliminating the need to define a class for the operator, thus simplifying the operation.

[0006] In this embodiment, compiling the registration configuration file can obtain the class corresponding to the target operator, eliminating the need to define a class for the operator. This improves the efficiency of adding operators to the computing library and simplifies the operation.

[0007] In one possible implementation, compiling the registration configuration file to obtain the class corresponding to the target operator includes: parsing the registration configuration file to obtain registration configuration information; the registration configuration information includes the name of the target operator, the input parameters of the target operator, the output parameters of the target operator, and the kernel function corresponding to the target operator; generating the class corresponding to the target operator based on the registration configuration information; the class corresponding to the target operator is used to call the kernel function corresponding to the target operator to process and obtain the output of the target operator.

[0008] In this implementation, the registration configuration information obtained by parsing the registration configuration file is used to generate the class corresponding to the target operator; the class corresponding to the target operator can be generated automatically and quickly.

[0009] In one possible implementation, the registration configuration information further includes first configuration information, which indicates the order of at least one kernel function call parameters corresponding to the target operator.

[0010] In this implementation, the first configuration information indicates the order of at least one kernel function call parameters corresponding to the target operator, which can quickly and accurately generate the corresponding kernel function.

[0011] In one possible implementation, the registration configuration information further includes the name of the inference function; after parsing the registration configuration file to obtain the registration configuration information, the method further includes: generating registration code based on the name of the inference function and the name of the target operator; the registration code is used to associate the class corresponding to the target operator with the inference function.

[0012] In this implementation, the class corresponding to the target operator and the inference function can be quickly and accurately associated.

[0013] In one possible implementation, the registration configuration information further includes second configuration information; after parsing the registration configuration file to obtain the registration configuration information, the method further includes: generating the inference function based on the second configuration information; and associating the inference function with the class corresponding to the target operator.

[0014] In this implementation, the required inference function can be generated based on the second configuration information;

[0015] In one possible implementation, the inference function includes a second code, and generating the inference function based on the second configuration information includes: generating the second code based on the first code in the second configuration information; the first code and the second code have at least partially identical code, and the second code is code used to infer the shape of the output of the target operator.

[0016] In this implementation, an inference function is generated based on the second configuration information, enabling users to generate the inference function they need according to their own requirements, thus satisfying different user needs.

[0017] In one possible implementation, the inference function includes a fourth code, and generating the inference function based on the second configuration information includes: generating the fourth code based on the third code in the second configuration information; the fourth code is used to check the parameter to be checked indicated by the third code at the time of invocation.

[0018] In this implementation, a fourth code is generated based on the third code in the second configuration information, so that the generated inference function can check the parameter to be detected when it is called.

[0019] In one possible implementation, before compiling the registration configuration file to obtain the class corresponding to the target operator, the method further includes: generating the registration configuration file based on the user's first operation; the first operation is to add one or more parameters to the original registration configuration file, the one or more parameters being obtained through abstraction based on the part of the target operator that distinguishes it from other operators.

[0020] In this implementation, one or more parameters are added to the original registration configuration file based on the user's first operation, which generates the registration configuration file quickly without having to write the entire registration configuration file. This solves the problem of excessive repetitive code in operator addition.

[0021] In one possible implementation, the method further includes: parsing a test configuration file using a first framework to obtain and save test data; the test data contains data required for testing a target function, which is a first function in the target operator; using the first framework to test the target function with the test data, obtaining and saving a first test result; using a second framework to read and test the target function with the test data, obtaining a second test result; the second framework is different from the first framework; based on the first test result and the second test result, obtaining a target test result; the target test result indicates whether the target function has passed or failed the test. The target function can be any function in the target operator.

[0022] This implementation method enables automated testing and achieves high testing efficiency.

[0023] In one possible implementation, the test configuration file includes generation rules; the step of parsing the test configuration file using a first framework to obtain test data and saving it includes: using the first framework to generate the test data according to the generation rules and saving it.

[0024] In this implementation, test data is generated according to the generation rules, enabling the rapid generation of different test cases.

[0025] In one possible implementation, the test configuration file further includes test program execution logic configuration information, which is used to generate the execution logic of the test program for testing the target function; the step of parsing the test configuration file using the first framework to obtain test data and save it includes: using the first framework to generate the test data according to the generation rules and the test program execution logic configuration information and saving it; the test data contains the execution logic of the test program for testing the target function.

[0026] In this implementation, automated testing is performed according to the configured generation rules and the execution logic of the test program, which can improve testing efficiency.

[0027] In one possible implementation, the function defined as a string in the test configuration file indicates the generation rule.

[0028] This implementation allows users to insert Python code when defining tensor generation rules and test program execution logic. Users can define functions as strings in the test configuration file to specify generation rules, ensuring boundary conditions in the tests.

[0029] In one possible implementation, before parsing the test configuration file using the first framework to obtain and save the test data, the method further includes: generating the test configuration file according to a second user operation; the second operation is used to add the format and generation rules of the test data to the original test configuration file, the generation rules being used to generate the test data.

[0030] In this implementation, adding test data format and generation rules to the original test configuration file can quickly generate a test configuration file that can generate the required test data according to the needs.

[0031] In one possible implementation, the first framework is the PyTorch framework and the second framework is the Parrots framework; or, the first framework is the Parrots framework and the second framework is the PyTorch framework.

[0032] Secondly, embodiments of this application provide an operator registration apparatus, comprising: a first processing unit for compiling a registration configuration file to obtain a class corresponding to a target operator; and a second processing unit for adding the class corresponding to the target operator and an inference function to a registry; wherein the inference function is used to infer the shape of the output of the target operator.

[0033] In one possible implementation, the first processing unit is specifically used to parse the registration configuration file to obtain registration configuration information; the registration configuration information includes the name of the target operator, the input parameters of the target operator, the output parameters of the target operator, and the kernel function corresponding to the target operator; based on the registration configuration information, a class corresponding to the target operator is generated; the class corresponding to the target operator is used to call the kernel function corresponding to the target operator to process and obtain the output of the target operator.

[0034] In one possible implementation, the registration configuration information further includes first configuration information, which indicates the order of at least one kernel function call parameters corresponding to the target operator.

[0035] In one possible implementation, the registration configuration information further includes the name of the inference function; the first processing unit is further configured to generate registration code based on the name of the inference function and the name of the target operator; the registration code is used to associate the class corresponding to the target operator with the inference function.

[0036] In one possible implementation, the registration configuration information further includes second configuration information; the first processing unit is further configured to generate the inference function based on the second configuration information; and associate the inference function with the class corresponding to the target operator.

[0037] In one possible implementation, the inference function includes second code, and the first processing unit is specifically used to generate the second code based on the first code in the second configuration information; the first code and the second code have at least partially identical code, and the second code is code used to infer the shape of the output of the target operator.

[0038] In one possible implementation, the inference function includes a fourth code, and the first processing unit is specifically configured to generate the fourth code based on the third code in the second configuration information; the fourth code is used to check the parameter to be checked indicated by the third code at the time of invocation.

[0039] In one possible implementation, the first processing unit is further configured to generate the registration configuration file based on the user's first operation; the first operation is to add one or more parameters to the original registration configuration file, the one or more parameters being obtained through abstraction based on the part of the target operator that distinguishes it from other operators.

[0040] In one possible implementation, the first processing unit is further configured to parse a test configuration file using a first framework to obtain and save test data; the test data includes data required for testing the target function, which is a first function in the target operator; the first framework is used to test the target function using the test data to obtain and save a first test result; a second framework is used to read and test the target function using the test data to obtain a second test result; the second framework is different from the first framework; a target test result is obtained based on the first test result and the second test result; the target test result indicates whether the target function has passed or failed the test. The target function can be any function in the target operator.

[0041] In one possible implementation, the test configuration file includes generation rules; the first processing unit is specifically used to use the first framework to generate the test data according to the generation rules and save it.

[0042] In one possible implementation, the test configuration file further includes test program execution logic configuration information, which is used to generate the execution logic of the test program for testing the target function; the first processing unit is specifically used to use the first framework to generate and save the test data according to the generation rules and the test program execution logic configuration information; the test data includes the execution logic of the test program for testing the target function.

[0043] In one possible implementation, the function defined as a string in the test configuration file indicates the generation rule.

[0044] In one possible implementation, the first processing unit is further configured to generate the test configuration file based on a second operation by the user; the second operation is to add the format and generation rules of the test data to the original test configuration file, wherein the generation rules are used to generate the test data.

[0045] In one possible implementation, the first framework is the PyTorch framework and the second framework is the Parrots framework; or, the first framework is the Parrots framework and the second framework is the PyTorch framework.

[0046] For the technical effects of the second aspect or various alternative implementation methods, please refer to the introduction of the technical effects of the first aspect or corresponding implementation methods.

[0047] Thirdly, embodiments of this application provide an electronic device comprising: a processor and a memory, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, causing the processor to perform the method as described in the first aspect and any possible implementation thereof.

[0048] Fourthly, embodiments of this application provide a chip including a data interface and a processor, wherein the processor is used to execute the method in the first aspect or any possible implementation thereof.

[0049] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method described in the first aspect and any optional implementation thereof.

[0050] Sixthly, embodiments of this application provide a computer program product including program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect and any of the optional implementation methods. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0052] Figure 1 A flowchart of an operator registration method provided in an embodiment of this application;

[0053] Figure 2 A flowchart of another operator registration method provided in this application embodiment;

[0054] Figure 3 A flowchart of another operator registration method provided in this application embodiment;

[0055] Figure 4 A flowchart of a testing method provided in an embodiment of this application;

[0056] Figure 5 This application provides a schematic diagram of an automated testing process.

[0057] Figure 6 This is a schematic diagram of the structure of an operator registration device provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0059] The terms "first," "second," and "third," etc., used in the specification, embodiments, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0060] Currently, adding an operator to a computing library typically involves three steps: 1) defining a class for the operator, used for dispatching to different computing backends; 2) adding the class defined for the operator to the registry; and 3) adding corresponding tests for the operator. Typically, when writing test files, the PyTorch framework is first used to generate test data. Then, the test data and the test results obtained using the PyTorch framework are saved to disk. During testing, the Parrots framework is used to read and test the data, and finally, the results are compared with those from PyTorch. The PyTorch framework is a deep learning framework based on the Python language. Essentially, all its programs are written in Python, making its source code relatively concise and widely used in the field of machine learning. Python is a cross-platform computer programming language and a high-level scripting language that combines interpreted, compiled, interactive, and object-oriented features. The Parrots framework is a deep learning framework with a Python interface.

[0061] When defining operator classes, there is a lot of duplicate code across different operators. When the framework changes, all defined classes must be modified accordingly. Each operator class needs to be registered, and when the operator name is modified, the registration must be updated. Furthermore, during testing, developers need to manually write test files and generate test cases, a very tedious process. Moreover, once generated, test cases cannot be changed, resulting in incomplete test cases. The fact that generated test cases cannot be modified (i.e., no new test cases can be added) means that using existing test cases may not be sufficient for all testing tasks, leading to incomplete test cases.

[0062] Due to the problem, operator addition is very inefficient, requiring developers and code reviewers to spend a significant amount of time ensuring code correctness. Therefore, there is a need to research more efficient operator addition solutions. This application provides an operator registration method with higher efficiency. The following is a brief introduction to the applicable scenarios for the operator registration method provided in the application embodiments.

[0063] Operator registration scenario: The operator registration device uses an automated registration framework to abstract the parts of any operator that distinguish it from other operators during operator development into parameters in the registration configuration file; it uses the automated registration framework to compile the registration configuration file, automatically generating the corresponding operator class (i.e., the class corresponding to the operator) and completing the registration of the operator; it defines a test plan through a Python configuration file, then uses Python code to parse the configuration file, and uses the PyTorch framework to execute the test items once, storing the test data and test results on the hard drive; then it uses the Parrots framework to read the test data according to each test case, executes the test items, and compares the test results of the Parrots framework with the test results of the PyTorch framework.

[0064] The operator registration method provided in this application can significantly reduce the amount of code required for operator development, alleviate the burden of code review, and improve the efficiency of operator development.

[0065] The operator registration method provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0066] Please see Figure 1 , Figure 1 This is a flowchart illustrating an operator registration method provided in an embodiment of this application. Figure 1 As shown, the method may include:

[0067] 101. The operator registration device compiles and registers the configuration file to obtain the class corresponding to the target operator.

[0068] The operator registration device can be a desktop computer, laptop computer, or other terminal device with data processing capabilities. In some embodiments, developers (or users) abstract the parts of the target operator (any operator or a specified operator) that distinguish it from other operators during the operator development process into parameters in a registration configuration file, thus obtaining the registration configuration file. In some embodiments, the registration configuration file is a YAML file (i.e., a configuration file written in YAML). The operator registration device uses an automated registration framework to compile the registration configuration file, obtaining the class corresponding to the target operator. YAML is a language specifically designed for writing configuration files; it is very concise and powerful.

[0069] In some embodiments, step 101 may be implemented as follows: Parse the registration configuration file to obtain registration configuration information, which includes the name of the target operator, the input parameters of the target operator, the output parameters of the target operator, and the kernel function corresponding to the target operator; generate a class corresponding to the target operator based on the registration configuration information; the class corresponding to the target operator can call the kernel function corresponding to the target operator to process and obtain the output of the target operator. In some embodiments, the registration configuration information further includes first configuration information, which indicates the order of calling parameters of at least one kernel function corresponding to the target operator. In some embodiments, the registration configuration information further includes the name of the inference function; after parsing the registration configuration file to obtain the registration configuration information, the operator registration device may perform the following operations: generate registration code based on the name of the inference function and the name of the target operator; the registration code is used to associate the class corresponding to the target operator and the inference function. Associating the class corresponding to the target operator and the inference function can be understood as associating the inference function used to infer the shape of the output of the target operator with the class corresponding to the target operator. After associating the class corresponding to the target operator and the inference function, the operator registration device can add the class corresponding to the target operator and the inference function to the registry, thus completing the registration of the target operator.

[0070] In some embodiments, the registration configuration information further includes second configuration information; after parsing the registration configuration file to obtain the registration configuration information, the operator registration device may perform the following operations: generate an inference function based on the second configuration information; associate the inference function with the class corresponding to the target operator. In some embodiments, the inference function includes second code, and generating the inference function based on the second configuration information includes: generating second code based on the first code in the second configuration information; the first code and the second code have at least partially identical code, and the second code is code used to infer the shape of the output of the target operator. For example, the first code in the second configuration information is calculate:outSpecs[0] = DArraySpec::scalar(inSpecs[0].elemType()); the inference function includes the second code as outSpecs[0] = DArraySpec::scalar(inSpecs[0].elemType()). In some embodiments, the inference function includes fourth code, and generating the inference function based on the second configuration information includes: generating fourth code based on the third code in the second configuration information; the fourth code is used to check the parameter to be checked indicated by the third code when the function is called. For example, the third code is used to indicate the parameter to be checked in the inference function, and the fourth code detects the parameter to be checked indicated by the third code through the macro definition PARROTS_CHECKARGS. The implementation of step 101 will be described in detail later with examples; the implementation of step 101 will not be described in detail here.

[0071] 102. The operator registration device adds the class and inference function corresponding to the target operator to the registry.

[0072] The inference function is used to infer the shape of the output of the target operator. Steps 101 and 102 can be understood as the implementation method of the operator registration device adding the target operator to the operator library.

[0073] In some embodiments, after executing step 101, the operator registration device may further perform the following steps: parse the test configuration file using a first frame to obtain and save test data, the test data containing data required for testing the target function, the target function being any function in the target operator or a specified function; test the target function using the test data using the first frame to obtain and save a first test result; read and test the target function using the test data using a second frame to obtain a second test result; the second frame is different from the first frame; based on the first and second test results, obtain the target test result; the target test result indicates whether the target function has passed or failed the test. Testing the target function can be understood as testing the target operator, that is, testing the function interface corresponding to the target operator. In some embodiments, the target function is the kernel function corresponding to the target operator. The first frame is different from the second frame.

[0074] The test configuration file can be a Python configuration file. In some embodiments, the test configuration file includes generation rules; parsing the test configuration file using a first framework to obtain and save test data can be done by: using the first framework to generate and save test data according to the generation rules. In some embodiments, the test configuration file also includes test program execution logic configuration information, which is used to generate the execution logic of the test program for the test target function; parsing the test configuration file using the first framework to obtain and save test data can be done by: using the first framework to generate and save test data according to the generation rules and the test program execution logic configuration information; the test data contains the execution logic of the test program for the test target function. The operator registration device can perform tests according to the execution logic of the test program for the test target function. The first framework can be a PyTorch framework, and the second framework can be a Parrots framework. In some embodiments, the operator registration device uses an automated testing framework to test the Parrots interface, that is, to test the functions in the target operator. In some embodiments, the test scheme for the operator registration device is defined by a Python configuration file (i.e., a test configuration file). A possible test process is as follows: use Python code to parse the test configuration file, and use the PyTorch framework (corresponding to the first framework) to execute a test item once, and store the test data and test results to the hard disk or a temporary file; use the Parrots framework (corresponding to the second framework) to read the test data according to each test case (corresponding to the function interface), execute the test item, and compare the test results of the Parrots framework with the test results of the PyTorch framework.

[0075] In this embodiment, the operator registration device can obtain the class corresponding to the target operator by compiling the registration configuration file, without needing to define a class for the operator, which can improve the efficiency of adding operators to the computing library and simplify the operation.

[0076] Figure 2 A flowchart of another operator registration method provided in an embodiment of this application. Figure 2 The method and process in the middle is Figure 1 The methods and processes in the document need to be refined and improved. For example... Figure 2 As shown, the method includes:

[0077] 201. The operator registration device parses the registration configuration file to obtain the registration configuration information.

[0078] The registration configuration information includes the parameters that distinguish the abstract target operator (any operator or a specified operator) from other operators. In other words, the registration configuration information contains all the parameters and / or information required by the operator registration device to generate the target operator.

[0079] In some embodiments, before executing step 201, the operator registration device may perform the following operations: The operator registration device generates a registration configuration file based on a user's first operation; the first operation is adding one or more parameters to the original registration configuration file, whereby the one or more parameters are obtained through abstraction based on the parts of the target operator that distinguish it from other operators. The original registration configuration file may be pre-configured with some basic configuration information (e.g., configuration information essential for generating any operator). The user's operation of adding one or more parameters to the original registration configuration file can be understood as adjusting the original registration configuration file to a registration configuration file for the class corresponding to the target operator obtained through compilation. In practical applications, developers abstract the parts of each operator that distinguish it from other operators into parameters in the registration configuration file during operator development. During compilation, the operator registration device can automatically generate the corresponding operator class (i.e., the class corresponding to the operator) based on the registration configuration file and complete the registration of the operator.

[0080] The output of an operator is typically the output of its corresponding kernel function. When an operator registration device needs to generate an operator, it must provide the corresponding infer function and kernel function. Simply put, the infer function allocates early memory space for the kernel function's output, so that its output can directly utilize this memory when the kernel function finishes computation. The following example, using the `addc` operator, illustrates how to add an operator to a computation library.

[0081] (1) Below is an example of the registration configuration information required by the operator registration device to automatically generate the operator addc:

[0082] 1. addc:#(1) The name of the operator

[0083] 2.ins:in#(2) The tensor type part of the input of the operator may have multiple

[0084] 3.outs:out#(3) The tensor type part of the operator's output may have multiple

[0085] 4.attr:c#(4) The input of the operator may contain multiple parts of other types.

[0086] 5. inferfunc:inferUnaryOp#(5) The name of the inference function corresponding to the operator, used to specify the output of the inference function, can be omitted.

[0087] 6. dispatch: #(6) The kernel function corresponding to the operator may be multiple

[0088] 7.native: addc(ctx,c,in,out) #(7) The native backend indicates that this is a kernel function of the computing library itself.

[0089] As can be seen, the name `addc` written in the registration configuration file will be converted to `AddcOp` by the operator registration device using Pascal's naming convention. `ins` and `outs` in the registration configuration file are the packed lists of inputs and outputs of type tensor, respectively. In this example, `ins` actually contains the unique input `in`, and `outs` contains the unique output `out`. In some embodiments, the operator registration device unpacks the inputs and outputs in the generated code, thus obtaining the inputs and outputs of type tensor. It should be understood that `ins` represents the names of the input tensors used in the operator interface (i.e., the kernel function corresponding to the operator), with different tensors separated by commas; `outs` represents the names of the output tensors used in the operator interface; `attr` represents the hyperparameters and scalars used in the operator interface; and `infer` represents obtaining the shape of the output array (DArray) and performing some type checks. Some simple inference functions can be generated directly using simple rules, while complex inference functions require writing the inference function directly and specifying the function name. `attr` is the parameter other than a tensor passed to the operator during the call. It can be assigned a default value, and the type will be converted accordingly when converting to the backend. For example, `double(int)eps = 1e-05` means that the type of this `attr` is `double`, the name is `eps`, the default value is `1e-05`, and it will be converted to an integer type (`int`) when calling the backend function.

[0090] In some embodiments, the operator registration device can obtain a unified interface for all backend function calls from ins, outs, and attrs, for example, void layer_norm(constDArray&input,constDArray&weight,constDArray&bias,DArray&output,vector<int>>).<uint64_t> normalized_shape,double eps,bool cudnn_enable).

[0091] Most operators have an indefinite number of other types of inputs besides tensors. In this case, there is only c (actually, there is a backend input by default), which is a scalar type. In the configuration, these indefinite inputs are usually packed into attr, and then unpacked in the generated code to get them one by one, as shown in line 4 of (1) and line 11 of (2). This way, the non-tensor type inputs and outputs are obtained.

[0092] After obtaining all the parameters required to generate the target operator from the registration configuration information, the operator registration device can call the kernel function for calculation. However, an operator may have multiple kernel functions available. The operator registration device needs to determine which one to use, and it needs to know the order of the kernel function call parameters. This is the necessity of the configuration in line 7 of (1): it indicates that the operator registration device needs the kernel function addc in the native backend to generate the class corresponding to the operator (e.g., the target operator), and it indicates the order of the parameters. ctx is the first parameter of all native kernel functions. Due to its commonality, it will not be elaborated here. The native backend indicates that this is the computation library of the operator registration device itself. In some embodiments, the registration configuration information also includes first configuration information, which indicates the order of the call parameters of at least one kernel function corresponding to the target operator. That is, the first configuration information in the registration configuration information indicates the order of the call parameters of at least one kernel function corresponding to the target operator. The operator registration device generates the kernel function corresponding to the target operator according to the first configuration information.

[0093] 202. The operator registration device generates the class corresponding to the target operator based on the registration configuration information.

[0094] The class corresponding to the target operator is used to call the kernel function corresponding to the target operator to process and obtain the output of the target operator. In some embodiments, the registration configuration information further includes first configuration information, which indicates the order of the call parameters of at least one kernel function corresponding to the target operator.

[0095] 203. The operator registration device generates a registration code based on the name of the inference function and the name of the target operator obtained from the registration configuration information.

[0096] The registration code is used to associate the class and inference function corresponding to the target operator. As mentioned earlier, the operator registration device needs not only a kernel function to perform calculations to generate a complete operator, but also an inference function to perform output inference. The operator registration device also needs to associate the generated operator code (i.e., the class corresponding to the operator) with the corresponding inference function: using the information in line 5 of (1) and the class corresponding to the operator, the operator registration device can generate the registration code, which is used to associate the class and inference function corresponding to the target operator. After generating the registration code, the operator registration device can associate the class and inference function corresponding to the target operator.

[0097] (2) The following is the registration code generated by the operator registration device, which associates the inferred function with the class corresponding to the operator and names the operator as "addc".

[0098] 1.OperatorRegistry::add("addc",

[0099] 2.OpCreateWithArchByOutput <addcop>(),

[0100] 3.inferUnaryOp,

[0101] 4.ComputeOpTypeInfer());

[0102] The operator registration device can use the information in line 5 of (1) and the class corresponding to the operator to generate the registration code shown in (2); the registration code shown in (2) can associate the inference function and the class corresponding to the operator. In this way, the two parts required to generate a complete operator are completed.

[0103] 204. The operator registration device adds the class and inference function corresponding to the target operator to the registry.

[0104] The inference function is used to infer the shape of the output of the target operator. In this embodiment, the operator registration device can use an automated registration framework to generate the class corresponding to the target operator based on the registration configuration file. The operator registration device can also generate an inference function, which is used to infer the shape of the output of the target operator. Finally, the operator registration device adds the class corresponding to the target operator and the inference function to the registry. Adding the class corresponding to the target operator and the inference function to the registry can be understood as the operator registration device adding the target operator to the computing library.

[0105] In this embodiment, the operator registration device generates a class corresponding to the target operator, then associates the class corresponding to the target operator with the inference function, and finally adds the class corresponding to the target operator and the inference function to the registry; this enables automatic registration of operators with high efficiency.

[0106] Figure 3 A flowchart of another operator registration method provided in an embodiment of this application. Figure 3 The method and process in the middle is Figure 1 The methods and processes in the document need to be refined and improved. For example... Figure 3 As shown, the method includes:

[0107] 301. The operator registration device parses the registration configuration file to obtain the registration configuration information.

[0108] 302. The operator registration device generates the class corresponding to the target operator based on the registration configuration information.

[0109] The class corresponding to the target operator is the code that calls the kernel function corresponding to the target operator to process and obtain the output of the target operator.

[0110] 303. The operator registration device generates an inference function based on the second configuration information in the registration configuration information.

[0111] In one possible implementation, the inference function includes a second code; the operator registration device generates the inference function based on the second configuration information in the registration configuration information, which may include: generating the second code based on the first code in the second configuration information; the first code and the second code have at least partially identical code, and the second code is code used to infer the shape of the output of the target operator.

[0112] In one possible implementation, the inference function includes a fourth code. The operator registration device generates the inference function based on the second configuration information in the registration configuration information. This generation of the inference function may include: generating the fourth code based on the third code in the second configuration information; the fourth code is used to check the parameter to be checked indicated by the third code at the time of invocation.

[0113] When an operator registration device needs to generate an operator, it must provide the corresponding inference function and kernel function. If the operator registration device can generate the kernel function for the operator, then the only thing the operator registration device needs to do is generate the corresponding inference function. The following section uses the `trace` operator as an example to illustrate the generation of the inference function.

[0114] (3) Configuration required to generate the inference function of the trace operator (corresponding to the second configuration information)

[0115] 1-trace:

[0116] 2 ins:input

[0117] 3 outs:out

[0118] 4 infer:

[0119] 5 check:inSpecs[0].ndims()==2

[0120] 6 requirements:inSpecs[0]

[0121] 7 calculate:outSpecs[0]=DArraySpec::scalar(inSpecs[0].elemType());

[0122] 8 dispatch:

[0123] 9. aten: Tensor trace(input);

[0124] (4) The following is the inference function of the automatically generated trace operator:

[0125] 1.bool inferTraceOp(const SSElement&attrs,

[0126] 2. const spec_list_t&inSpecs,spec_list_t&outSpecs){

[0127] 3.PARROTS_CHECKARGS(inSpecs.size()==1);

[0128] 4.PARROTS_CHECKARGS(outSpecs.size()==1);

[0129] 5. if(inSpecs[0].isNone()){

[0130] 6. return false;

[0131] 7.} 8.

[0133] 9.PARROTS_CHECKARGS(inSpecs[0].ndims()==2);

[0134] 10.outSpecs[0]=DArraySpec::scalar(inSpecs[0].elemType());

[0135] 11. Return true;

[0136] 12.}

[0137] As can be seen, the configuration information in lines 4 to 7 of (3) (corresponding to the first code) generates the corresponding inference function. For the required parameters, the requirements can be used for configuration. For the parameters that need to be checked (corresponding to the third code), whether in attr or ins / outs, the operator registration device uniformly uses the macro definition PARROTS_CHECKARGS for checking. The most important purpose of the inference function is to normalize the output of this operator. This is the necessity of line 7 of (3). It instructs the operator registration device to generate line 10 of (4), indicating that the output of our operator is a scalar object with the same data type as the input.

[0138] The trace function uses the kernel function of the aten backend, as shown in line 9 of (3). The connection between the inference function and the operator has been explained in detail in the previous example and will not be repeated here.

[0139] 304. The operator registration device associates the inference function with the class corresponding to the target operator.

[0140] In some embodiments, the operator registration device may associate the inference function with the class corresponding to the target operator through registration code similar to that in (2) above.

[0141] 305. The operator registration device adds the class and inference function corresponding to the target operator to the registry.

[0142] In this embodiment of the application, an inference function is generated based on the second configuration information, enabling users to generate the required inference function according to their own needs, which can meet different user needs.

[0143] The foregoing embodiments described how to add the class and inference function corresponding to the target operator to the registry. After adding the operator to the computing library, the operator registration device also needs to test the operator. The following describes how to automate the testing of operators.

[0144] Figure 4 This is a flowchart of a testing method provided in an embodiment of this application. Figure 4 As shown, the method includes:

[0145] 401. The operator registration device uses the first frame to parse the test configuration file, obtain the test data, and save it.

[0146] The test data contains the data required to test the target function, which can be any function in the target operator, such as a kernel function. The test configuration file can be written in Python. The primary framework can be either PyTorch or Python.

[0147] In some embodiments, before executing step 401, the operator registration device may perform the following operation: The operator registration device generates a test configuration file based on the user's second operation. The second operation involves adding the test data format and generation rules to the original test configuration file. The generation rules are used to generate test data. The original test configuration file may be pre-configured with some basic configuration information (e.g., configuration information essential for generating any test data). The user's operation of adding the test data format and generation rules to the original test configuration file can be understood as adjusting the original test configuration file to a test configuration file that can be parsed to obtain the required test data. In practical applications, developers add the test data format and test data generation rules to the original test configuration file so that the operator registration device can generate the test configuration file.

[0148] 402. Use the first framework to test the objective function with the test data, obtain the first test result and save it.

[0149] In some embodiments, the operator registration device uses Python code to parse the test configuration file, executes the test item once using the PyTorch framework, and stores the generated test data and the test results (corresponding to the first test result) on the hard disk. Test Scope: The tests mainly include tests on the torch interface, tensor interface, functional interface, and module interface. A functional interface can be understood as an interface with one and only one abstract method, but can have multiple non-abstract methods. A module interface can be understood as a modular interface specifically designed for neural networks, which can contain the definitions of each layer of the network and the forward method. Because the torch interface and tensor interface are basically the same, the rules for both interfaces are defined in the same configuration file, while the other functional and module interfaces are defined in their respective configuration files. Each interface can be understood as a function; testing an interface is equivalent to testing a function.

[0150] Define rules: Rules include generating test data according to user requirements (shape, data type, generation rules), the range of hyperparameter selection, function calls, and verification of test results. To make rules more concise and flexible, users can insert Python code when defining tensor generation rules and test program execution logic. Users can define functions as strings in the test configuration file to specify generation rules, ensuring boundary conditions for testing. To avoid introducing a large number of extra parameters for handling rare cases, the operator registration device predefines some possible cases for some parameters and sets default values. Users can select a predefined case using an alias; if the user ignores this attribute, the default value will be automatically assigned.

[0151] Data generation and storage: When generating test data, users can freely specify whether it is randomly generated by rules (generation rules) or assigned specific values. When several tensors satisfy the same set of rules, these tensors can be written on the same line, separated by semicolons according to whether the tensors need differentiation, and then the tensor generation rules for this line are defined. Data is stored on disk according to test cases, storing only the test data and test results generated by PyTorch, that is, the test data input and the test results obtained when using PyTorch to test according to the test cases. When Parrots tests according to each test case, it reads the test data corresponding to that test case, performs the test, and compares the test result obtained with the stored test result of that test case (i.e., the test result generated by PyTorch). In essence, if the test result obtained by using PyTorch to test according to a certain test case is the same as the test result obtained by using Parrots to test according to that test case, it indicates that the test case was successfully tested.

[0152] 403. Use the second frame to read and use the test data to test the target function and obtain the second test result.

[0153] The second framework differs from the first framework. The second framework can be either a Parrots framework or a PyTorch framework. In some embodiments, the first framework is a Parrots framework and the second framework is a PyTorch framework. The order in which steps 402 and 403 are executed by the operator registration device is not limited. The operator registration device may execute step 402 first, then step 403; or it may execute step 403 first, then step 402; or it may execute steps 402 and 403 in parallel (i.e., simultaneously). It should be understood that there is no dependency between steps 402 and 403. Figure 4 This is just one example.

[0154] In some embodiments, the operator registration device can build an automated testing framework for operators. Test developers can use this automated testing framework to write the format and generation rules of test data in the test configuration file. During each test, PyTorch is first used to generate the corresponding test data and test results, which are saved in a temporary file. Then, Parrots is used to read the data, perform the test, and compare the test results with those of PyTorch. Figure 5 This is a schematic diagram illustrating an automated testing process provided in an embodiment of this application. Figure 5 As shown, users can edit the test configuration file, that is, use the automated testing framework to write the test data format and generation rules in the test configuration file; the operator registration device can first use PyTorch (corresponding to the first framework) to generate the corresponding test data through the automated testing framework, and save the test data and test results in a temporary file, and then use Parrots (corresponding to the second framework) to read the test data, perform the test, and compare the test results with PyTorch. Figure 5 In this code, each case corresponds to one test case, and the number of cases is not limited. For example, case1 represents the test data of one test case and the test result of testing that test case using PyTorch, while case6 represents the test data of another test case and the test result of testing that test case using Parrots.

[0155] 404. Based on the first test result and the second test result, the target test result is obtained.

[0156] The target test result indicates whether the target function passes or fails the test. In some embodiments, if the first test result and the second test result are the same, the target function is determined to have passed the test (corresponding to the target test result); if the first test result and the second test result are different, the target function is determined to have failed the test (corresponding to the target test result).

[0157] When testing an operator, the operator registration device can perform the corresponding test on the operator by adding the corresponding test parameters to the test configuration file.

[0158] In this embodiment, the format and generation rules of the test data are written in the test configuration file. This allows for the generation of required test data as needed, solving the problems of large code volume and fixed test data in operator testing, and reducing test development costs. Furthermore, when testing operators, the operator registration device generates test data and stores the test data and results in a temporary file. This temporary storage of test data and results avoids the problem of excessive space consumption when test data is stored in a Git repository. In other words, the operator registration device only needs to store the test configuration file; test data and results are only generated during testing.

[0159] Figure 6 This is a schematic diagram of the structure of an operator registration device provided in an embodiment of this application, as shown below. Figure 6 As shown, the device includes:

[0160] The first processing unit 601 is used to compile the registration configuration file to obtain the class corresponding to the target operator;

[0161] The second processing unit 602 is used to add the class and inference function corresponding to the target operator to the registry; the inference function is used to infer the shape of the output of the target operator.

[0162] In one possible implementation, the first processing unit 601 is specifically used to parse the registration configuration file to obtain registration configuration information; the registration configuration information includes the name of the target operator, the input parameters of the target operator, the output parameters of the target operator, and the kernel function corresponding to the target operator; based on the registration configuration information, a class corresponding to the target operator is generated; the class corresponding to the target operator is used to call the kernel function corresponding to the target operator to process and obtain the output of the target operator.

[0163] In one possible implementation, the registration configuration information further includes first configuration information, which indicates the order of at least one kernel function call parameters corresponding to the target operator.

[0164] In one possible implementation, the registration configuration information also includes the name of the inferred function;

[0165] The first processing unit 601 is also used to generate registration code based on the name of the inference function and the name of the target operator; the registration code is used to associate the class and inference function corresponding to the target operator.

[0166] In one possible implementation, the registration configuration information also includes second configuration information;

[0167] The first processing unit 601 is also used to generate an inference function based on the second configuration information; and to associate the inference function with the class corresponding to the target operator.

[0168] In one possible implementation, the inference function includes a second code;

[0169] The first processing unit 601 is specifically used to generate a second code based on the first code in the second configuration information; the first code and the second code have at least some of the same code, and the second code is a code used to infer the shape of the output of the target operator.

[0170] In one possible implementation, the inference function includes a fourth code;

[0171] The first processing unit 601 is specifically used to generate a fourth code based on the third code in the second configuration information; the fourth code is used to check the parameter to be checked indicated by the third code when it is called.

[0172] In one possible implementation, the first processing unit 601 is further configured to generate a registration configuration file based on the user's first operation; the first operation is to add one or more parameters to the original registration configuration file, and the one or more parameters are obtained through abstract processing based on the part of the target operator that distinguishes it from other operators.

[0173] In one possible implementation, the first processing unit 601 is further configured to parse the test configuration file using a first framework, obtain test data, and save it; the test data contains data required for testing the target function, which is the first function in the target operator; the first framework is used to test the target function using the test data, obtaining a first test result and saving it; a second framework is used to read and use the test data to test the target function, obtaining a second test result; the second framework is different from the first framework; based on the first and second test results, a target test result is obtained; the target test result indicates whether the target function has passed or failed the test. The target function can be any function in the target operator.

[0174] In one possible implementation, the test configuration file includes generation rules;

[0175] The first processing unit 601 is specifically used to generate and save test data according to the generation rules using the first framework.

[0176] In one possible implementation, the test configuration file also includes test program execution logic configuration information, which is used to generate the execution logic of the test program for the test target function;

[0177] The first processing unit 601 is specifically used to generate and save test data using the first framework according to the generation rules and test program execution logic configuration information; the test data includes the execution logic of the test program for the test target function.

[0178] In one possible implementation, the function defined as a string in the test configuration file indicates the generation rules.

[0179] In one possible implementation, the first processing unit 601 is further configured to generate a test configuration file based on a second operation by the user; the second operation is to add the format of test data and generation rules to the original test configuration file, and the generation rules are used to generate test data.

[0180] In one possible implementation, the first framework is the PyTorch framework and the second framework is the Parrots framework; or, the first framework is the Parrots framework and the second framework is the PyTorch framework.

[0181] In one possible implementation, the operator registration device further includes an output unit 603 for outputting the target test results. For example, the operator registration device is a desktop computer, and the display is the hardware device of the output unit 603, through which the target test results are displayed.

[0182] It should be understood that the division of the various units in the above operator registration device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. For example, each of the above units can be a separately established processing element, or it can be integrated into the same chip. Alternatively, it can be stored as program code in the controller's storage element, and called and executed by a processing element of the processor. Furthermore, the units can be integrated together or implemented independently. The processing element here can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the method or each of the above units can be completed through hardware integrated logic circuits in the processor element or software instructions. This processing element can be a general-purpose processor, such as a central processing unit (CPU), or one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), etc.

[0183] Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 7 As shown, the terminal device 70 includes a processor 701, a memory 702, and a communication interface 703; the processor 701, the memory 702, and the communication interface 703 are interconnected via a bus. Figure 7 The terminal device in this context can be the operator registration device in the aforementioned embodiments.

[0184] The memory 702 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CDROM), and is used for related instructions and data. The communication interface 703 is used for receiving and sending data.

[0185] The processor 701 can be one or more CPUs. If the processor 701 is a single CPU, it can be a single-core CPU or a multi-core CPU. The steps performed by the operator registration device in the foregoing embodiments can be based on this... Figure 7 The structure of the terminal device shown is illustrated. Specifically, the processor 701 can implement... Figure 6 The functions of each unit.

[0186] In the embodiments of this application, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the operator registration method provided in the foregoing embodiments.

[0187] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the operator registration method provided in the foregoing embodiments.

[0188] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.< / addcop>

Claims

1. An operator registration method, characterized in that, include: Compile and register the configuration file to obtain the class corresponding to the target operator. The class corresponding to the target operator is used to call the kernel function corresponding to the target operator. Add the class and inference function corresponding to the target operator to the registry; The inference function is used to infer the shape of the output of the target operator; The method further includes: The test configuration file is parsed using the first framework to obtain and save the test data; the test data contains the data required for the test objective function, and the objective function is the first function in the objective operator. The first framework is used to test the objective function with the test data to obtain and save the first test result. The target function is tested using the test data through a second frame to obtain a second test result; the second frame is different from the first frame. Based on the first test result and the second test result, a target test result is obtained; the target test result indicates whether the target function has passed or failed the test.

2. The method according to claim 1, characterized in that, The compiled registration configuration file yields the following classes corresponding to the target operator: The registration configuration file is parsed to obtain registration configuration information; the registration configuration information includes the name of the target operator, the input parameters of the target operator, the output parameters of the target operator, and the kernel function corresponding to the target operator; Based on the registration configuration information, the class corresponding to the target operator is generated.

3. The method according to claim 2, characterized in that, The registration configuration information also includes first configuration information, which indicates the order of at least one kernel function call parameters corresponding to the target operator.

4. The method according to claim 2 or 3, characterized in that, The registration configuration information also includes the name of the inference function; After parsing the registration configuration file to obtain registration configuration information, the method further includes: Based on the name of the inference function and the name of the target operator, a registration code is generated; The registration code is used to associate the class corresponding to the target operator with the inference function.

5. The method according to claim 2 or 3, characterized in that, The registration configuration information also includes second configuration information; after parsing the registration configuration file to obtain the registration configuration information, the method further includes; The inference function is generated based on the second configuration information; Associat the inference function with the class corresponding to the target operator.

6. The method according to claim 5, characterized in that, The inference function includes second code, and generating the inference function based on the second configuration information includes: The second code is generated based on the first code in the second configuration information; the first code and the second code have at least some of the same code, and the second code is code used to infer the shape of the output of the target operator.

7. The method according to claim 5, characterized in that, The inference function includes a fourth code, and generating the inference function based on the second configuration information includes: The fourth code is generated based on the third code in the second configuration information; the fourth code is used to check the parameter to be checked indicated by the third code when the call is made.

8. The method according to any one of claims 1 to 3, characterized in that, Before compiling and registering the configuration file to obtain the class corresponding to the target operator, the method further includes: The registration configuration file is generated based on the user's first operation; the first operation is to add one or more parameters to the original registration configuration file, and the one or more parameters are obtained through abstraction based on the part of the target operator that distinguishes it from other operators.

9. The method according to claim 1, characterized in that, The test configuration file includes generation rules; the process of parsing the test configuration file using the first framework to obtain and save test data includes: The test data is generated and saved using the first framework according to the generation rules.

10. The method according to claim 9, characterized in that, The test configuration file also includes test program execution logic configuration information, which is used to generate the execution logic of the test program for testing the target function; The step of using the first framework to parse the test configuration file, obtain test data, and save it includes: The first framework is used to generate and save the test data according to the generation rules and the test program execution logic configuration information. The test data includes the execution logic of the test program that tests the objective function.

11. The method according to claim 9 or 10, characterized in that, The functions defined as strings in the test configuration file indicate the generation rules.

12. The method according to claim 1, characterized in that, Before parsing the test configuration file using the first framework to obtain and save the test data, the method further includes: Based on the user's second operation, the test configuration file is generated; the second operation is to add the format and generation rules of the test data to the original test configuration file, and the generation rules are used to generate the test data.

13. The method according to claim 1, characterized in that, The first framework is the PyTorch framework, and the second framework is the Parrots framework; Alternatively, the first framework can be the Parrots framework, and the second framework can be the PyTorch framework.

14. An operator registration device, characterized in that, include: The first processing unit is used to compile and register the configuration file to obtain the class corresponding to the target operator. The class corresponding to the target operator is used to call the kernel function corresponding to the target operator. The second processing unit is used to add the class and inference function corresponding to the target operator to the registry; the inference function is used to infer the shape of the output of the target operator. The first processing unit is further configured to parse the test configuration file using the first framework, obtain test data, and save it; the test data includes data required for the test target function, and the target function is the first function in the target operator; The first framework is used to test the target function with the test data to obtain a first test result and save it; the second framework is used to read and test the target function with the test data to obtain a second test result. The second frame is different from the first frame; Based on the first test result and the second test result, a target test result is obtained; the target test result indicates whether the target function has passed or failed the test.

15. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, such that the processor performs the method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 13.

Images