Method, device and equipment for adapting external device operator, and storage medium

Abstract

The present disclosure provides an adaptation method and device of an external device operator, equipment and storage medium, and relates to the field of deep learning. According to the target configuration file and the operator type of the to-be-adapted operator of the external device, the present disclosure can obtain the target implementation function of the to-be-adapted operator, and call a code generation tool to automatically register the target implementation function of the to-be-adapted operator on the target scheduling key in the target deep learning framework corresponding to the back end of the external device. The present disclosure can greatly reduce the manual workload of adapting the deep learning framework by the external device operator, improve the adaptation efficiency of the external device operator in the deep learning framework, and reduce the adaptation cost of the external device operator in the deep learning framework.

Description

Technical Field

[0001] This disclosure relates to the field of deep learning, and more particularly to a method, apparatus, device, and storage medium for adapting external device operators. Background Technology

[0002] In the field of deep learning, deep learning frameworks (such as the PyTorch framework) are software frameworks used to build, train, and run deep learning models. Deep learning frameworks need to explicitly provide operator interfaces to users, internally manage the registration and dispatch of various device backend implementations, and achieve hardware acceleration for the target platform.

[0003] In related technologies, for external devices not built into the deep learning framework, developers need to define custom implementation functions for external device operators and manually register these implementation functions with the corresponding scheduling key of the deep learning framework, such as PrivateUse1 (private backend reserved key 1). This enables the external device operators to be adapted in the deep learning framework. Then, at runtime, the deep learning framework dispatches the tensor to the registered implementation function for computation based on factors such as the external device to which the tensor belongs and whether differentiation is required.

[0004] However, in related technologies, each external device operator requires manual implementation functions and manual registration of the external device operator implementation functions into the deep learning framework. The operator adaptation process is cumbersome, inefficient, and costly. Summary of the Invention

[0005] To address the problems existing in the aforementioned related technologies, this disclosure provides an adaptation method, apparatus, device, and storage medium for external device operators, which can reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0006] The first aspect of this disclosure provides a method for adapting an external device operator, comprising:

[0007] Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0008] Determine the operator type of the operator to be adapted;

[0009] Based on the operator type and the attribute fields corresponding to the function declaration, determine the target implementation function of the operator to be adapted;

[0010] The code generation tool is invoked to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0011] A second aspect of this disclosure provides an adapter for an external device operator, comprising:

[0012] The acquisition module is used to acquire the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0013] The first determining module is used to determine the operator type of the operator to be adapted.

[0014] The second determination module is used to determine the target implementation function of the operator to be adapted based on the operator type and the attribute fields corresponding to the function declaration.

[0015] The registration module is used to call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework. The target scheduling key is the scheduling key corresponding to the backend of the external device.

[0016] A third aspect of this disclosure provides a computer device including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the adaptation method of the external device operator described in the first aspect.

[0017] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the adaptation method for the external device operator of the first aspect described above.

[0018] The technical solution provided in this disclosure has the following advantages compared with related technologies:

[0019] In this embodiment, the target configuration file of the operator to be adapted from the external device is obtained, wherein the target configuration file includes attribute fields corresponding to the function declarations of the operator to be adapted; the operator type of the operator to be adapted is determined; based on the operator type and the attribute fields corresponding to the function declarations, the target implementation function of the operator to be adapted is determined; and a code generation tool is invoked to register the target implementation function to the target scheduling key in the target deep learning framework, wherein the target scheduling key is the scheduling key corresponding to the backend of the external device. Therefore, only the target configuration file and operator type of the operator to be adapted from the external device are needed to obtain the target implementation function of the operator to be adapted, and the code generation tool can be invoked to automatically register the target implementation function of the operator to be adapted to the target scheduling key in the target deep learning framework corresponding to the backend of the external device. This can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0022] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an adaptation method for an external device operator provided in an embodiment of this disclosure;

[0024] Figure 2 This is a flowchart of another method for adapting an external device operator provided in an embodiment of this disclosure;

[0025] Figure 3 This is a flowchart of another method for adapting an external device operator provided in an embodiment of this disclosure;

[0026] Figure 4 This is a flowchart of another method for adapting an external device operator according to an embodiment of this disclosure;

[0027] Figure 5 This is a flowchart of another method for adapting an external device operator according to an embodiment of this disclosure;

[0028] Figure 6 This is a flowchart of another method for adapting an external device operator according to an embodiment of this disclosure;

[0029] Figure 7 This is a flowchart of another method for adapting an external device operator according to an embodiment of this disclosure;

[0030] Figure 8 This is a flowchart of a method for adapting a structured operator according to an embodiment of this disclosure;

[0031] Figure 9 This is a flowchart of an adaptation method for an unstructured operator provided in an embodiment of this disclosure;

[0032] Figure 10 This is a schematic diagram of the structure of an adapter device for an external device operator provided in an embodiment of this disclosure;

[0033] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0034] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0035] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0036] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0037] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0038] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0039] To better understand the inventive concept of the embodiments of this disclosure, the technical solutions of the embodiments of this disclosure will be described below in conjunction with exemplary embodiments.

[0040] The external device operator adaptation method provided in this disclosure can be executed by a computer device. This device can be understood as any device with processing and computing capabilities, including but not limited to electronic devices such as smartphones, laptops, tablets, in-vehicle terminals, wearable devices, digital TVs, desktop computers, smart home devices, etc.

[0041] Figure 1 This is a flowchart illustrating an adaptation method for an external device operator provided in an embodiment of this disclosure. This method can be executed by a computer device. Figure 1 As shown, the adaptation method for external device operators provided in this embodiment includes the following steps:

[0042] Step 110: Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0043] In this embodiment of the disclosure, the computer device can obtain the target configuration file of the operator to be adapted from the external device.

[0044] The target configuration file can include the function declarations of the operators to be adapted and the corresponding attribute fields of the function declarations.

[0045] A function declaration can be understood as a declaration of information used to identify the operator to be adapted or the calling form of the operator to be adapted.

[0046] Function declarations include operator names, and may also include overload flags, parameter information, and other information.

[0047] Attribute fields can be understood as configuration fields associated with function declarations, used to indicate the adaptation method or code generation method of the operator to be adapted.

[0048] The attribute fields can include data such as the source name of the target scheduling key bound to the backend of the external device, structured inheritance fields, backend binding name, implementation function name, and inherited base class fields.

[0049] For example, the target configuration file can be a declarative file (custom_functions.yaml). A declarative file can be a readable data serialization language (YAML Ain't Markup Language) file.

[0050] In some embodiments, a user can create a configuration file for an external device to be adapted within a computer device. In response to the configuration file creation operation for the external device to be adapted, the computer device can create a target configuration file corresponding to the external device to be adapted.

[0051] In other embodiments, the computer device may receive a target configuration file of the operator to be adapted, uploaded by the user.

[0052] Therefore, the target configuration file corresponding to the operator to be adapted can be obtained accurately and quickly.

[0053] Step 120: Determine the operator type of the operator to be adapted.

[0054] In this embodiment of the disclosure, the computer device determines the operator type of the operator to be adapted.

[0055] The operator type can be either structured or unstructured.

[0056] A structured operator is an operator with structured implementation information. A structured operator contains multiple overloaded functions. For each overloaded function, a template wrapper class is generated and inherits the common base class of preprocessing logic and computation logic in turn.

[0057] The preprocessing logic is used to determine the metadata of the tensor. Also known as metadata (Meta) derivation logic, the preprocessing logic is used to deduce the shape and type (Dtype) of the tensor.

[0058] The metadata of a tensor can include information such as its shape, type, and stride.

[0059] The computation (Implementation, Impl) logic is used to perform operator computations based on the metadata of tensors.

[0060] Unstructured operators can be understood as operators whose entire process is implemented within a single function. Unstructured operators do not have overloaded functions; developers need to manually write the complete preprocessing and computation code within the implementation function.

[0061] In some embodiments, the computer device may read the native operator configuration file; in the native operator configuration file, find the native adaptation type of the operator to be adapted; and determine the native adaptation type of the operator to be adapted as the operator type of the operator to be adapted.

[0062] Specifically, the computer device can read the native operator configuration file (native_functions.yaml) of the PyTorch tensor computation library (ATen), and then find the native adaptation type of the operator to be adapted in the native operator configuration file; and determine the native adaptation type of the operator to be adapted as the operator type of the operator to be adapted.

[0063] In other embodiments, the target configuration file of the operator to be adapted may also include the operator type of the operator to be adapted. The computer device can read the operator type of the operator to be adapted from the target configuration file of the operator to be adapted.

[0064] Therefore, the operator type of the operator to be adapted can be accurately and quickly determined.

[0065] Step 130: Based on the operator type and the attribute fields corresponding to the function declaration of the operator to be adapted, determine the target implementation function of the operator to be adapted.

[0066] In this embodiment of the disclosure, after obtaining the attribute fields corresponding to the operator type and function declaration of the operator to be adapted, the computer device can determine the target implementation function of the operator to be adapted based on the attribute fields corresponding to the operator type and function declaration of the operator to be adapted.

[0067] The target implementation function can be understood as the implementation logic of the operator to be adapted.

[0068] Step 140: Call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0069] In this embodiment of the disclosure, after determining the target implementation function of the operator to be adapted, the computer device can call the code generation tool in the deep learning framework to register the target implementation function to the target scheduling key in the target deep learning framework, wherein the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0070] A deep learning framework is a software framework used to build, train, and run deep learning models. It may include operator interfaces, tensor computation libraries, operator scheduling mechanisms, backend registration mechanisms, code generation mechanisms, etc.

[0071] For example, the target deep learning framework could be the PyTorch framework.

[0072] The target scheduling key is used to call the target implementation function to execute the operator to be adapted when the input tensor of the operator to be adapted is associated with the backend of an external device.

[0073] The target scheduling key can be PrivateUse1 (private backend reserved key 1), which can be used to identify the backend of external devices.

[0074] For example, the backend of an external device may include the GPU in the external device.

[0075] In this embodiment, the target implementation function of the operator to be adapted can be obtained simply by using the target configuration file and operator type of the operator to be adapted on the external device. Then, the code generation tool is called to automatically register the target implementation function of the operator to be adapted to the target scheduling key in the target deep learning framework corresponding to the backend of the external device. This can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0076] In some embodiments of this disclosure, the computer device can generate a target binding function (wrapper) corresponding to the operator to be adapted based on the function declaration, attribute fields, and target implementation function of the operator to be adapted; and register the target binding function to the target scheduling key in the target deep learning framework, wherein the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0077] The target binding function can be understood as an intermediate function that connects the operator calls of the target deep learning framework with the implementation logic of the operator to be adapted. It serves as a bridge function between the scheduling entry point of the target deep learning framework and the specific implementation logic of the operator to be adapted.

[0078] Therefore, the target binding function corresponding to the operator to be adapted can be automatically generated, and the target binding function can be accurately and quickly registered to the target scheduling key in the target deep learning framework corresponding to the backend of the external device. There is no need to manually write binding functions and registration code, which improves the adaptation efficiency of external device operators in deep learning frameworks and reduces the adaptation cost of external device operators in deep learning frameworks.

[0079] In some embodiments, when the operator type of the operator to be adapted is a structured operator, the target binding function corresponding to the operator to be adapted can be used to perform device verification consistent with the target deep learning framework, correctly set the device environment; instantiate the operator template class and import parameters; execute the meta method of the template class to complete preprocessing; execute the impl method of the template class to complete device calculation; process the results and write back.

[0080] In some embodiments, when the operator type of the operator to be adapted is an unstructured operator, the target binding function corresponding to the operator to be adapted can be used to perform device verification consistent with the target deep learning framework, correctly set the device environment; directly call the target implementation function of the operator to be adapted; and process and write back the results.

[0081] Figure 2 This is a flowchart illustrating an adaptation method for an external device operator provided in an embodiment of this disclosure. This method can be executed by a computer device. Figure 2 As shown, the adaptation method for external device operators provided in this embodiment includes the following steps:

[0082] Step 210: Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0083] Step 220: Determine the operator type of the operator to be adapted.

[0084] Step 230: When the operator type of the operator to be adapted is a structured operator, read the attribute field corresponding to the main function declaration from the attribute field corresponding to the function declaration of the operator to be adapted. The main function declaration is the declaration of the main function for the operator to be adapted, and the main function is an overloaded function with an output value.

[0085] The function declarations of structured operators include main function declarations and metafunction declarations.

[0086] The attribute fields corresponding to the function declarations of structured operators include the attribute fields corresponding to the main function declarations and the attribute fields corresponding to the meta-function declarations.

[0087] A main function declaration is a declaration for the main function. A metafunction declaration is a declaration for the metafunction.

[0088] The main function can be understood as an overloaded function with an output value (out), that is, an overloaded function with output. Conversely, a metafunction can be understood as an overloaded function without an output value (functional), that is, an overloaded function without output.

[0089] Function overloading refers to a group of functions with the same name but different parameter lists. Function overloading allows multiple functions with the same name to be defined within the same scope, as long as their parameter lists differ.

[0090] In this embodiment of the disclosure, when the operator type of the operator to be adapted is a structured operator, the function declaration of the operator to be adapted includes a main function declaration and a metafunction declaration. The attribute fields corresponding to the function declaration of the operator to be adapted include attribute fields corresponding to the main function declaration and attribute fields corresponding to the metafunction declaration. The computer device can read the attribute fields corresponding to the main function declaration from the attribute fields corresponding to the function declaration of the operator to be adapted.

[0091] Step 240: Based on the attribute fields corresponding to the main function declaration of the operator to be adapted, determine the target implementation function of the operator to be adapted.

[0092] In this embodiment of the disclosure, after obtaining the attribute fields corresponding to the main function declaration of the operator to be adapted, the computer device can determine the target implementation function of the operator to be adapted based on the attribute fields corresponding to the main function declaration of the operator to be adapted.

[0093] Step 250: Call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0094] Therefore, when the operator type to be adapted is a structured operator, the attribute fields corresponding to the main function declaration are read from the attribute fields corresponding to the function declaration of the operator to be adapted; based on the attribute fields corresponding to the main function declaration, the target implementation function of the operator to be adapted is determined. Thus, the target implementation function of the operator to be adapted can be obtained without developers writing preprocessing and computation functions, which can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0095] Figure 3 This is a flowchart illustrating an adaptation method for an external device operator provided in an embodiment of this disclosure. This method can be executed by a computer device. Figure 3 As shown, the adaptation method for external device operators provided in this embodiment includes the following steps:

[0096] Step 310: Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0097] Step 320: Determine the operator type of the operator to be adapted.

[0098] Step 330: When the operator type of the operator to be adapted is a structured operator, read the attribute field corresponding to the main function declaration from the attribute field corresponding to the function declaration of the operator to be adapted. The main function declaration is the declaration of the main function for the operator to be adapted, and the main function is an overloaded function with an output value.

[0099] Step 340: If the source name and structured inheritance fields corresponding to the main function declaration of the operator to be adapted do not exist in the attribute fields, then obtain the native preprocessing function and native computation function corresponding to the operator to be adapted.

[0100] In this embodiment of the disclosure, if the source name and structured inheritance field bound to the target scheduling key are not present in the attribute field corresponding to the main function declaration of the operator to be adapted, the computer device can obtain the native preprocessing function and native computation function corresponding to the operator to be adapted.

[0101] For example, computer devices can obtain the native preprocessing functions and native computation functions corresponding to the operators to be adapted from deep learning frameworks.

[0102] The source name bound to the target scheduling key can be set as needed; there are no restrictions here.

[0103] The `structured_inherits` field indicates the base class used by the code generation tool when generating the template class corresponding to a structured operator. The `structured_inherits` field tells the code generator to automatically generate a structure (Struct / Class) that inherits from a standard base class for the operator to be adapted.

[0104] The native preprocessing function of the operator to be adapted can be understood as the existing preprocessing function of the operator to be adapted.

[0105] The native computation function of the operator to be adapted can be understood as the existing computation function of the operator to be adapted.

[0106] The preprocessing function is the preprocessing logic. The computation function is the computation logic.

[0107] Step 350: Based on the native preprocessing function and native computation function corresponding to the operator to be adapted, generate the target implementation function of the operator to be adapted.

[0108] In this embodiment of the disclosure, after obtaining the native preprocessing function and native computation function corresponding to the operator to be adapted, the computer device can generate the target implementation function of the operator to be adapted based on the native preprocessing function and native computation function corresponding to the operator to be adapted. That is, the existing native preprocessing function and native computation function of the operator to be adapted are directly reused to generate the target implementation function of the operator to be adapted.

[0109] For example, the native preprocessing function and native computation function corresponding to the operator to be adapted can be determined as the target implementation function of the operator to be adapted.

[0110] The above-mentioned acquisition of the native preprocessing function and native computation function corresponding to the operator to be adapted, and the generation of the target implementation function of the operator to be adapted based on the native preprocessing function and native computation function, can be called the first shared implementation strategy of the operator to be adapted.

[0111] Step 360: Call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0112] Therefore, when the operator type of the operator to be adapted is a structured operator, if the attribute fields corresponding to the main function declaration of the operator to be adapted do not contain the source name and structured inheritance fields bound to the target scheduling key, then the target implementation function of the operator to be adapted can be generated based on the native preprocessing function and native computation function corresponding to the operator to be adapted. The native preprocessing function and native computation function corresponding to the operator to be adapted can be directly reused. Developers do not need to write the preprocessing function and computation function of the operator to be adapted to obtain the target implementation function of the operator to be adapted. This can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0113] Figure 4 This is a flowchart illustrating an adaptation method for an external device operator provided in an embodiment of this disclosure. This method can be executed by a computer device. Figure 4 As shown, the adaptation method for external device operators provided in this embodiment includes the following steps:

[0114] Step 410: Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0115] Step 420: Determine the operator type of the operator to be adapted.

[0116] Step 430: When the operator type of the operator to be adapted is a structured operator, read the attribute field corresponding to the main function declaration from the attribute field corresponding to the function declaration of the operator to be adapted. The main function declaration is the declaration of the main function for the operator to be adapted, and the main function is an overloaded function with an output value.

[0117] Step 440: If the attribute field corresponding to the main function declaration of the operator to be adapted contains the source name bound to the target scheduling key and does not contain a structured inheritance field, then obtain the native preprocessing function corresponding to the operator to be adapted and read the custom calculation function of the operator to be adapted from the target configuration file.

[0118] In this embodiment of the disclosure, if the attribute field corresponding to the main function declaration of the operator to be adapted contains the source name bound to the target scheduling key and does not contain a structured inheritance field, the computer device can obtain the native preprocessing function corresponding to the operator to be adapted and read the custom calculation function of the operator to be adapted from the target configuration file of the operator to be adapted.

[0119] For example, a computer device can obtain the native preprocessing function corresponding to the operator to be adapted from the deep learning framework, and read the custom computation function of the operator to be adapted from the target configuration file of the operator to be adapted.

[0120] Step 450: Based on the native preprocessing function and the custom calculation function of the operator to be adapted, generate the target implementation function of the operator to be adapted.

[0121] In this embodiment of the disclosure, after obtaining the native preprocessing function and the custom computation function of the operator to be adapted, the computer device can generate the target implementation function of the operator to be adapted based on the native preprocessing function and the custom computation function. That is, the target implementation function of the operator to be adapted is generated based on the existing native preprocessing function of the operator to be adapted and the computation function of the operator to be adapted defined by the developer.

[0122] For example, native preprocessing functions and custom computation functions can be identified as the target implementation functions of the operators to be adapted.

[0123] The process of obtaining the native preprocessing function corresponding to the operator to be adapted and reading the custom calculation function of the operator to be adapted from the target configuration file, and generating the target implementation function of the operator to be adapted based on the native preprocessing function and the custom calculation function, can be called the preprocessing fallback implementation strategy of the operator to be adapted.

[0124] Custom computation functions can be understood as computation functions defined by the developer for operators to be adapted.

[0125] Step 460: Call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0126] Therefore, when the operator type of the operator to be adapted is a structured operator, if the attribute field corresponding to the main function declaration of the operator to be adapted contains the source name bound to the target scheduling key and does not contain a structured inheritance field, then the target implementation function of the operator to be adapted can be generated based on the native preprocessing function and the custom calculation function corresponding to the operator to be adapted. The native preprocessing function corresponding to the operator to be adapted can be directly reused, and developers do not need to write the preprocessing function of the operator to be adapted. Only the calculation function of the operator to be adapted needs to be customized to obtain the target implementation function of the operator to be adapted. This can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0127] Figure 5 This is a flowchart illustrating an adaptation method for an external device operator provided in an embodiment of this disclosure. This method can be executed by a computer device. Figure 5 As shown, the adaptation method for external device operators provided in this embodiment includes the following steps:

[0128] Step 510: Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0129] Step 520: Determine the operator type of the operator to be adapted.

[0130] Step 530: When the operator type of the operator to be adapted is a structured operator, read the attribute field corresponding to the main function declaration from the attribute field corresponding to the function declaration of the operator to be adapted. The main function declaration is the declaration of the main function for the operator to be adapted, and the main function is an overloaded function with an output value.

[0131] Step 540: If the attribute fields corresponding to the main function declaration of the operator to be adapted contain the source name and structured inheritance fields bound to the target scheduling key, then read the custom preprocessing function and custom calculation function of the operator to be adapted from the target configuration file.

[0132] In this embodiment of the disclosure, if the attribute field corresponding to the main function declaration of the operator to be adapted contains the source name and structured inheritance field bound to the target scheduling key, the computer device can read the custom preprocessing function and custom computation function of the operator to be adapted from the target configuration file. In other words, as long as the attribute field corresponding to the main function declaration of the operator to be adapted contains the structured inheritance field, the computer device will read the custom preprocessing function and custom computation function of the operator to be adapted from the target configuration file.

[0133] Custom preprocessing functions can be understood as preprocessing functions for operators to be adapted, defined by the developer.

[0134] Custom computation functions can be understood as computation functions defined by the developer for operators to be adapted.

[0135] The `structured_inherits` field indicates the base class used by the code generation tool when generating the template class corresponding to a structured operator. The `structured_inherits` field tells the code generator to automatically generate a structure (Struct / Class) that inherits from a standard base class for the operator to be adapted.

[0136] Step 550: Based on the custom preprocessing function and custom computation function of the operator to be adapted, generate the target implementation function of the operator to be adapted.

[0137] In this embodiment of the disclosure, after obtaining the custom preprocessing function and custom computation function of the operator to be adapted, the computer device can generate the target implementation function of the operator to be adapted based on the custom preprocessing function and custom computation function. That is, the target implementation function of the operator to be adapted is generated based on the custom preprocessing function and custom computation function of the operator to be adapted.

[0138] For example, custom preprocessing functions and custom computation functions can be identified as the target implementation functions of the operators to be adapted.

[0139] The process of reading the custom preprocessing function and custom computation function of the operator to be adapted from the target configuration file, and generating the target implementation function of the operator to be adapted based on the custom preprocessing function and custom computation function, can be called the first custom implementation strategy of the operator to be adapted.

[0140] In some embodiments, when the operator type of the operator to be adapted is a structured operator, if the attribute field corresponding to the main function declaration of the operator to be adapted contains a structured inheritance field and there is no source name bound to the target scheduling key, the target implementation function of the operator to be adapted cannot be generated. The computer device can issue a prompt message, which includes at least one of the following: the source name bound to the target scheduling key cannot be obtained, or the source name bound to the target scheduling key needs to be supplemented, to remind the user to supplement the source name bound to the target scheduling key.

[0141] Step 560: Call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0142] Therefore, when the operator type of the operator to be adapted is a structured operator, if the attribute fields corresponding to the main function declaration of the operator to be adapted contain the source name and structured inheritance fields bound to the target scheduling key, then the target implementation function of the operator to be adapted can be generated based on the custom preprocessing function and custom computation function of the operator to be adapted. That is, the target implementation function of the operator to be adapted can be obtained by customizing the preprocessing function and computation function of the operator to be adapted. This can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0143] It's important to note that structured operators don't rely on a single function for adaptation; instead, they use a class inheritance chain to combine "general preprocessing functions," "device calculation functions," and "overload difference handling functions." The code generation tool determines, based on the configuration file, which classes in this inheritance chain reuse native implementations and which use custom implementations for external devices. Structured operators achieve device-independent reuse of general logic through class inheritance. During the code generation phase, namespaces are used to distinguish between general and custom implementations; the namespace for custom implementations is "custom." The inheritance relationship from bottom to top is as follows:

[0144] At the lowest level are the core base classes (such as the tensor iterator TensorIterator): These core base classes provide commonly used low-level utility functions during the execution of structured operators, such as tensor traversal, output construction, type handling, and device-related auxiliary capabilities. They represent fundamental capabilities that many operators can share. Their namespace is native.

[0145] The second layer is the Meta class (metadata preprocessing class): responsible for preprocessing logic, such as parameter validation, shape deduction, type deduction, and determining output tensor information. Its class name typically includes the metafunction name, and its native namespace is meta (meta information namespace). This layer addresses the question of "what shape and type the output should have, and whether the parameters are valid, before the operator is executed."

[0146] The third layer is the Impl class (computation class or implementation class): responsible for the actual computation logic, that is, calling or carrying out the device computation implementation. Its class name usually includes the backend binding name of the main function, and the native namespace is native. This layer solves the problem of "how to calculate after preprocessing".

[0147] The fourth layer is the operator template class: its class name contains the backend binding name of the main function and the overload type, responsible for handling parameter differences under different overload conditions. Because this template class is used for external device adaptation, its namespace is custom.

[0148] Figure 6 This is a flowchart illustrating an adaptation method for an external device operator provided in an embodiment of this disclosure. This method can be executed by a computer device. Figure 6 As shown, the adaptation method for external device operators provided in this embodiment includes the following steps:

[0149] Step 610: Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0150] Step 620: Determine the operator type of the operator to be adapted.

[0151] Step 630: If the operator type of the operator to be adapted is an unstructured operator, and the source name bound to the target scheduling key does not exist in the attribute field corresponding to the function declaration of the operator to be adapted, then obtain the native implementation function corresponding to the operator to be adapted.

[0152] In this embodiment of the disclosure, when the operator type of the operator to be adapted is an unstructured operator type, if the source name bound to the target scheduling key is not present in the attribute field corresponding to the function declaration of the operator to be adapted, the computer device can obtain the native implementation function corresponding to the operator to be adapted.

[0153] For example, computer devices can obtain the native implementation functions corresponding to the operators to be adapted from deep learning frameworks.

[0154] Step 640: Determine the native implementation function corresponding to the operator to be adapted as the target implementation function of the operator to be adapted.

[0155] In this embodiment of the disclosure, after obtaining the native implementation function corresponding to the operator to be adapted, the computer device can determine the native implementation function corresponding to the operator to be adapted as the target implementation function of the operator to be adapted. That is, the existing native implementation function of the operator to be adapted is determined as the target implementation function of the operator to be adapted.

[0156] The above-mentioned method of obtaining the native implementation function corresponding to the operator to be adapted and determining the native implementation function as the target implementation function of the operator to be adapted can also be called the second shared implementation strategy of the operator to be adapted.

[0157] Step 650: Call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0158] Therefore, when the operator type of the operator to be adapted is an unstructured operator, if the source name bound to the target scheduling key does not exist in the attribute field corresponding to the function declaration of the operator to be adapted, the computer device can obtain the native implementation function corresponding to the operator to be adapted and determine the native implementation function as the target implementation function of the operator to be adapted. That is, the target implementation function of the operator to be adapted can be obtained by reusing the native implementation function corresponding to the operator to be adapted. The native implementation function corresponding to the operator to be adapted can be directly reused, and the implementation logic of the operator to be adapted does not need to be written by the developers. This can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0159] Figure 7 This is a flowchart illustrating an adaptation method for an external device operator provided in an embodiment of this disclosure. This method can be executed by a computer device. Figure 7 As shown, the adaptation method for external device operators provided in this embodiment includes the following steps:

[0160] Step 710: Obtain the target configuration file of the operator to be adapted from the external device. The target configuration file includes the attribute fields corresponding to the function declarations of the operator to be adapted.

[0161] Step 720: Determine the operator type of the operator to be adapted.

[0162] Step 730: If the operator type of the operator to be adapted is an unstructured operator, and the source name bound to the target scheduling key exists in the attribute field corresponding to the function declaration of the operator to be adapted, then the custom implementation function of the operator to be adapted is read from the target configuration file.

[0163] In this embodiment of the disclosure, when the operator type of the operator to be adapted is an unstructured operator, if the attribute field corresponding to the function declaration of the operator to be adapted contains the source name bound to the target scheduling key, the computer device can read the custom implementation function of the operator to be adapted from the target configuration file of the operator to be adapted.

[0164] For example, when the operator type of the operator to be adapted is an unstructured operator, the source name bound to the target scheduling key can be the function name of the target implementation function of the operator to be adapted.

[0165] Custom implementation functions can be understood as implementation functions defined by the developer for the operators to be adapted.

[0166] Step 740: Determine the custom implementation function of the operator to be adapted as the target implementation function of the operator to be adapted.

[0167] In this embodiment of the disclosure, after obtaining the custom implementation function of the operator to be adapted, the computer device can determine the custom implementation function of the operator to be adapted as the target implementation function of the operator to be adapted. That is, the implementation function of the operator to be adapted defined by the developer is determined as the target implementation function of the operator to be adapted.

[0168] The above-mentioned method of reading the custom implementation function of the operator to be adapted from the target configuration file of the operator to be adapted and determining the custom implementation function as the target implementation function of the operator to be adapted can be called the second custom implementation strategy of the operator to be adapted.

[0169] Step 750: Call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0170] Therefore, when the operator type of the operator to be adapted is an unstructured operator type, if the source name bound to the target scheduling key exists in the attribute field corresponding to the function declaration of the operator to be adapted, the custom implementation function of the operator to be adapted is read from the target configuration file of the operator to be adapted, and the custom implementation function is determined as the target implementation function of the operator to be adapted. This can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve the adaptation efficiency of external device operators in deep learning frameworks, and reduce the adaptation cost of external device operators in deep learning frameworks.

[0171] In some embodiments of this disclosure, when the operator type of the operator to be adapted is a structured operator, the computer device can execute... Figure 8 A flowchart of a structured operator adaptation method is provided.

[0172] like Figure 8 As shown, the computer device runs a target deep learning framework (such as the PyTorch framework) and its code generation tools. Figure 8 The adaptation process is shown when the target adaptation type of the operator to be adapted is a structured operator type, from top to bottom as follows: declarative configuration file area (custom_functions.yaml), code generation tool (Codegen), and custom function area (Custom Functions).

[0173] Figure 8 The top section is a declarative configuration area created by the developer, used to declare the function declarations of the operators to be adapted to external devices and the corresponding attribute fields of those function declarations. Figure 8The text uses three entries, A, B, and C, to illustrate three adaptation scenarios. The operator names A, B, and C, as well as the backend binding names fa, fb, and fc, are merely illustrative examples and do not constitute a limitation on specific operators or names. Entries in the declarative configuration file area marked with PyTorch native_functions.yaml belong to the native operator configuration file of the target deep learning framework (e.g., native_functions.yaml of the PyTorch tensor computation library ATen). These entries record the native adaptation type of each operator and an index of its native implementation logic. Figure 8 They are shown side-by-side for comparison. The native operator configuration file and the declarative configuration file are two different files. Figure 8 The meanings of the Chinese and English fields are as follows:

[0174] -func is the starting identifier for a function declaration entry, followed by the operator name, used to declare an operator to be adapted or one of its calling forms. For example, -func:A indicates that an operator named A is declared.

[0175] The `.out` directive is appended to the operator name to identify an overloaded function that has an output value (`out`), i.e., the main function. Conversely, an overloaded function without `.out` is a function without an output value (`functional`), i.e., a metafunction. In the diagram, `(example)` in `-func: A.out(example)` is an illustrative label, indicating that this overloaded entry is listed as an example of overload for ease of explanation.

[0176] The `dispatch` property is a dispatch declaration block within the attribute field. Below it, key-value pairs specify the implementation names that the operator is bound to under different backends. These values ​​represent the source names bound to the target dispatch key. The key `PrivateUse1` is the target dispatch key (private backend reserved key 1), identifying the backend of the external device, such as the graphics processing unit (GPU) in the external device; the key `CPU` identifies the central processing unit backend. For example, `dispatch:PrivateUse1:fb` means that operator B is bound to the source name `fb` under the `PrivateUse1` backend. The source names are used by code generation tools to determine the naming of the `Impl` class, operator template class, and bound functions.

[0177] `structured_inherits` is a structured inheritance field in the attribute fields. Its value indicates the core base class explicitly inherited by the code generation tool when generating the custom Meta class corresponding to the structured operator. For example, `structured_inherits:TensorIterator` indicates that the TensorIterator is used as the core base class. Figure 8Only operator C declares this field.

[0178] The custom function area is located in Figure 8 The bottom section is used to store functions manually implemented by developers, such as custom calculation functions and custom preprocessing functions. Binding functions generated by the code generation tool reference functions in the custom function area when needed.

[0179] The implementation of structured operators is divided into two parts: a preprocessing function and a computation function. The preprocessing function determines the metadata (Meta) derivation logic of the tensor; the computation function, also known as the implementation (Impl) logic, performs operator computation based on the tensor's metadata. Structured operators typically declare both overloaded functions with output values ​​(out) and overloaded functions without output values ​​(functional). The overloaded functions with output values ​​serve as the main function, and the overloaded functions without output values ​​serve as metafunctions. The code generation tool reads the main function declaration of the operator from the declarative configuration file and, based on the corresponding attribute fields of the main function declaration, executes the code generation process. Figure 8 One of the three parallel adaptation paths shown.

[0180] The code generation tool implements structured operators through a class inheritance chain. This chain, from bottom to top, consists of the core base class, the Meta class, the Impl class, and the operator template class. During code generation, namespaces distinguish between generic and custom implementations: `native` is the native namespace, `meta` is the meta-information namespace, and `custom` is the external device-defined namespace. The core base class, for example, is a tensor iterator `native::TensorIterator`, providing low-level utility functions for tensor traversal, output construction, and type handling. The Meta class name includes the meta-function name, carrying preprocessing functions. The Impl class name includes the backend binding name of the main function, carrying computation functions. The operator template class name includes the backend binding name of the main function and overloaded types, carrying functions for handling parameter differences under different overloaded conditions.

[0181] First shared implementation strategy:

[0182] See Figure 8 On the left branch, the code generation tool parses the main function declaration of a certain operator A to be adapted in the declarative configuration file. If the attribute field corresponding to the main function declaration does not contain either the source name bound to the target scheduling key or the structured_inherits field, then the adaptation strategy for this operator is determined to be the first shared implementation strategy.

[0183] Under the first shared implementation strategy, the code generation tool performs templating, generating an operator template class `custom::structured_fa_out {}` that inherits from the native `Impl` class `native::structured_fa {}`. This `Impl` class further inherits from the native `Meta` class `meta::structured_A {}`, which in turn inherits from the core base class `native::TensorIterator`. Thus, the operator template class reuses the native preprocessing functions and native computation functions through the inheritance chain, eliminating the need for developers to write any functions in the custom function area.

[0184] The code generation tool then performs binding, generating the binding function `wrapper_PrivateUse1_A_out_out`. This binding function executes the following steps in sequence: device verification and setting up the device environment (devicecheck / guard) to match the target deep learning framework; instantiating the operator template class `custom::structured_fa_out`; calling `meta::structured_A::meta()` to complete preprocessing; calling `native::structured_fa::impl()` to complete device computation; and finally writing back and returning the result.

[0185] Finally, the code generation tool performs the registration, calling the TORCH_LIBRARY_IMPL macro to register the binding function wrapper_PrivateUse1_A_out_out to the target scheduling key PrivateUse1 backend. Subsequently, when the target deep learning framework runs, if an input tensor is associated with the backend of an external device, the above-mentioned implementation logic will be executed via this binding function, acting as operator A.

[0186] Under the first shared implementation strategy, the preprocessing functions and computation functions of the operators to be adapted directly reuse the native logic of the target deep learning framework. Developers do not need to write any device-related code to complete the adaptation, which can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve adaptation efficiency, and reduce adaptation costs.

[0187] Preprocessing rollback implementation strategy:

[0188] See Figure 8In the intermediate branch, the code generation tool parses the main function declaration of the operator B to be adapted. If the attribute field corresponding to the main function declaration contains the source name bound to the target dispatch key (e.g., dispatch: PrivateUse1: fb), but does not contain a structured inheritance field, then the adaptation strategy for this operator is determined to be the preprocessing fallback implementation strategy. The preprocessing fallback implementation strategy can be understood as: reusing the native preprocessing function of the target deep learning framework, while the developer defines a custom computation function.

[0189] Under the preprocessing fallback implementation strategy, the code generation tool performs templating, generating an operator template class `custom::structured_fb_out {}` that inherits from a custom `Impl` class `custom::structured_fb { void impl();}`. This custom `Impl` class further inherits from the native `Meta` class `meta::structured_B {}`, which in turn inherits from the core base class `native::TensorIterator`. Thus, the inheritance chain reuses the native `meta::structured_B` at the preprocessing layer, while the computation layer is handled by the custom `custom::structured_fb`. Developers then implement the computation function `custom::structured_fb::impl()` of this custom `Impl` class in the custom function area; this is the computation function of operator B on the external device. Developers do not need to implement the preprocessing function.

[0190] The code generation tool then performs the binding, generating the binding function `wrapper_PrivateUse1_B_out_out`. This binding function executes the following steps in sequence: device verification and setting the device environment; instantiating `custom::structured_fb_out`; calling `meta::structured_B::meta()` to complete native preprocessing; calling `custom::structured_fb::impl()` to complete the developer-defined device calculations; and finally writing back and returning the result. The call to `custom::structured_fb::impl()` is the binding function's reference to the custom function area.

[0191] Finally, the code generation tool performs the registration, calling the TORCH_LIBRARY_IMPL macro to register wrapper_PrivateUse1_B_out_out to the PrivateUse1 backend.

[0192] Under the preprocessing fallback implementation strategy, the preprocessing functions of the operators to be adapted reuse the native logic. Developers only need to implement the device-related calculation functions to complete the adaptation. While preserving the correctness of the native preprocessing, it also takes into account the differences in external device calculations, reducing the workload of developers to repeatedly write preprocessing code and improving adaptation efficiency.

[0193] First custom implementation strategy:

[0194] See Figure 8 On the right-hand branch, the code generation tool parses the main function declaration of operator C. If the main function declaration contains both the source name bound to the target dispatch key (e.g., dispatch: PrivateUse1: fc) and a structured inheritance field (e.g., structured_inherits: TensorIterator), then the adaptation strategy for this operator is determined to be the first custom implementation strategy. The first custom implementation strategy can be understood as: both the preprocessing function and the computation function are defined by the developer. The structured inheritance field is used to indicate the core base class used by the code generation tool when generating the template class corresponding to the structured operator.

[0195] Under the first custom implementation strategy, the code generation tool performs templating, generating an operator template class `custom::structured_fc_out {}` that inherits from the custom `Impl` class `custom::structured_fc { void impl();}`. This custom `Impl` class further inherits from the custom `Meta` class `custom::structured_C { void meta();}`, which explicitly inherits from the core base class `native::TensorIterator` indicated by the structured inheritance field. Thus, the inheritance chain is carried by custom classes under the `custom` namespace at both the preprocessing and computation layers. Developers then implement the computation function `custom::structured_fc::impl()` and the preprocessing function `custom::structured_C::meta()` in the custom function area.

[0196] The code generation tool then performs binding, generating the binding function `wrapper_PrivateUse1_C_out_out`. This binding function executes the following steps in sequence: device verification and setting the device context; instantiating `custom::structured_fc_out`; calling `custom::structured_C::meta()` to complete custom preprocessing; calling `custom::structured_fc::impl()` to complete custom device calculations; and finally writing back and returning the result. The calls to these two custom functions constitute the binding function's reference to the custom function area.

[0197] Finally, the code generation tool performs the registration, calling the TORCH_LIBRARY_IMPL macro to register wrapper_PrivateUse1_C_out_out to the PrivateUse1 backend.

[0198] Under the first custom implementation strategy, developers can provide implementations that match the characteristics of external devices for both the preprocessing and computation stages, covering operator adaptation scenarios where native logic cannot be directly reused. Meanwhile, the instantiation of template classes and the generation and registration of binding functions are still automatically completed by the code generation tool. Developers only need to focus on writing device-related code, avoiding the tedious work of manually instantiating template classes and manually writing binding and registration code, improving adaptation efficiency and reducing maintenance costs.

[0199] In some embodiments of this disclosure, when the operator type of the operator to be adapted is an unstructured operator, the computer device can execute... Figure 9 A flowchart of an adaptation method for unstructured operators is provided.

[0200] like Figure 9 As shown, from top to bottom, they are: declarative configuration file (custom_functions.yaml), code generation tool (Codegen), and custom function area (Custom Functions). The functions of each area are as follows: Figure 8 same. Figure 9 The declarative configuration file area at the top uses two entries, operators D and E, to illustrate two adaptation scenarios respectively. Operator D illustrates the scenario of reusing the native implementation, while operator E illustrates the scenario of custom implementation. The operator names D and E, as well as the backend binding names fd and fe, are examples and do not constitute limitations. The entries marked with PyTorch native_functions.yaml in the declarative configuration file area belong to the native operator configuration file.

[0201] Figure 9`-func` is the starting identifier of the function declaration entry; `dispatch` is the dispatch declaration block, under which `PrivateUse1` is the target dispatch key and `CPU` is the central processing unit backend. The meanings of each field are as follows: Figure 8 Same. Operators of unstructured operator types in Figure 9 Each operator declares only a single function entry, without the .out overload flag, and the structured_inherits field does not appear.

[0202] Unstructured operator types can be understood as operators whose entire operation is completed within a single implementation function. They have no overloaded functions, do not distinguish between preprocessing functions and computation functions, and therefore lack the intermediate strategy of preprocessing fallback. Correspondingly, the code generation tool reads the operator's function declaration from the declarative configuration file and, based on the corresponding attribute fields of the function declaration, proceeds... Figure 9 One of the two parallel adaptation paths shown directly generates a binding function based on the operator name.

[0203] Second sharing implementation strategy:

[0204] See Figure 9 On the left branch, the code generation tool parses the function declaration of the operator D to be adapted. If the source name bound to the target scheduling key does not exist in the attribute field corresponding to the function declaration, the adaptation strategy for this operator is determined to be the second shared implementation strategy. The second shared implementation strategy can be understood as directly reusing the native implementation function of the target deep learning framework.

[0205] Under the second shared implementation strategy, the code generation tool obtains the native implementation function (e.g., `native::fd()`) corresponding to the operator and identifies this native implementation function as the target implementation function. Binding is then performed, generating the binding function `wrapper_PrivateUse1_D`. This binding function executes the following sequentially: device verification and setting up the device environment consistent with the target deep learning framework (device check / guard); direct call to the native implementation function `native::fd()`; and writing back and returning the result. Since the target implementation function is the native implementation function, this binding function does not reference the custom function area.

[0206] Finally, the code generation tool performs the registration, calling the TORCH_LIBRARY_IMPL macro to register the binding function wrapper_PrivateUse1_D to the target scheduling key PrivateUse1 backend.

[0207] Under the second shared implementation strategy, the implementation logic of the operator to be adapted directly reuses the native implementation function of the target deep learning framework. Developers do not need to write any device-related code to complete the adaptation, which can greatly reduce the manual workload of adapting external device operators to deep learning frameworks, improve adaptation efficiency, and reduce adaptation costs.

[0208] Second custom implementation strategy:

[0209] See Figure 9 On the right-hand branch, the code generation tool parses the function declaration of the operator E to be adapted. If the attribute field corresponding to the function declaration contains a source name bound to the target scheduling key (e.g., dispatch: PrivateUse1: fe), then the adaptation strategy for this operator is determined to be the second custom implementation strategy. The second custom implementation strategy can be understood as the developer defining the complete implementation logic of the operator to be adapted. In some embodiments, the source name bound to the target scheduling key in the attribute field of the unstructured operator is the name of the function implemented by the developer, and the code generation tool references this implementation function when binding code generation.

[0210] Under the second custom implementation strategy, the code generation tool reads the custom implementation function of the operator (e.g., `custom::fe()` indicated by the source name `fe`) from the declarative configuration file and identifies this custom implementation function as the target implementation function. The developer then implements the function `custom::fe()` in the custom function area to complete the overall logic of preprocessing and computation of operator E. Subsequently, the code generation tool performs binding, generating the binding function `wrapper_PrivateUse1_E`. This binding function executes the following in sequence: device verification and setting the device environment; direct call to the custom implementation function `custom::fe()`; and write-back and return the result. The call to `custom::fe()` is the binding function's reference to the custom function area.

[0211] Finally, the code generation tool performs the registration, calling the TORCH_LIBRARY_IMPL macro to register the binding function wrapper_PrivateUse1_E to the PrivateUse1 backend.

[0212] Under the second custom implementation strategy, developers only need to implement a complete device calculation function. Template-independent binding and registration are still automatically completed by the code generation tool, avoiding the need to manually write binding functions and registration calls. This improves adaptation efficiency and reduces maintenance costs while covering unstructured operator adaptation scenarios that cannot be reused by native logic.

[0213] Figure 10This is a schematic diagram of the structure of an adapter device for an external device operator provided in an embodiment of this disclosure. This device can be understood as the aforementioned computer device or a functional module within the aforementioned computer device. For example... Figure 10 As shown, the adapter 1000 for the external device operator includes:

[0214] The acquisition module 1010 is used to acquire the target configuration file of the operator to be adapted from the external device, wherein the target configuration file includes the attribute fields corresponding to the function declaration of the operator to be adapted;

[0215] The first determining module 1020 is used to determine the operator type of the operator to be adapted.

[0216] The second determining module 1030 is used to determine the target implementation function of the operator to be adapted based on the operator type and the attribute fields corresponding to the function declaration.

[0217] The registration module 1040 is used to call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, where the target scheduling key is the scheduling key corresponding to the backend of the external device.

[0218] Optionally, the above-mentioned acquisition module includes:

[0219] Create a submodule to respond to the configuration file creation operation of the operator to be adapted to the external device, and create the target configuration file of the operator to be adapted.

[0220] Optionally, the first determining module mentioned above includes:

[0221] The file reading submodule is used to read native operator configuration files;

[0222] The search submodule is used to find the native adaptation type of the operator to be adapted in the native operator configuration file;

[0223] The determination submodule is used to determine the native adaptation type of the operator to be adapted as the operator type of the operator to be adapted.

[0224] Optionally, the target configuration file mentioned above may also include the operator type of the operator to be adapted;

[0225] The aforementioned first determining module includes:

[0226] The operator type reading submodule is used to read the operator type of the operator to be adapted from the target configuration file.

[0227] Optionally, the second determining module mentioned above includes:

[0228] The data reading submodule is used to read the attribute fields corresponding to the main function declaration from the attribute fields corresponding to the function declaration when the operator type is a structured operator. The main function declaration is the declaration of the main function for the operator to be adapted, and the main function is an overloaded function with output values.

[0229] The function determination submodule is used to determine the target implementation function of the operator to be adapted based on the attribute fields declared in the main function.

[0230] Optionally, the above function determines the submodules including:

[0231] The first acquisition unit is used to acquire the native preprocessing function and native computation function corresponding to the operator to be adapted if the source name and structured inheritance field bound to the target scheduling key do not exist in the attribute fields corresponding to the main function declaration.

[0232] The first generation unit is used to generate the target implementation function of the operator to be adapted based on the native preprocessing function and the native computation function.

[0233] Optionally, the above function determines the submodules including:

[0234] The second acquisition unit is used to acquire the native preprocessing function corresponding to the operator to be adapted if the source name bound to the target scheduling key exists in the attribute field corresponding to the main function declaration and there is no structured inheritance field, and to read the custom calculation function of the operator to be adapted from the target configuration file.

[0235] The second generation unit is used to generate the target implementation function of the operator to be adapted based on the native preprocessing function and the custom calculation function.

[0236] Optionally, the above function determines the submodules including:

[0237] The third acquisition unit is used to read the custom preprocessing function and custom calculation function of the operator to be adapted from the target configuration file if the source name and structured inheritance field bound to the target scheduling key exist in the attribute field corresponding to the main function declaration.

[0238] The third generation unit is used to generate the target implementation function of the operator to be adapted based on the custom preprocessing function and the custom calculation function.

[0239] Optionally, the second determining module mentioned above includes:

[0240] The submodule is used to retrieve the native implementation function corresponding to the operator to be adapted if the source name bound to the target scheduling key does not exist in the attribute field corresponding to the function declaration when the operator type is unstructured.

[0241] The first function determination submodule is used to determine the native implementation function as the target implementation function of the operator to be adapted.

[0242] Optionally, the second determining module mentioned above includes:

[0243] The function reading submodule is used to read the custom implementation function of the operator to be adapted from the target configuration file if the source name bound to the target scheduling key exists in the attribute field corresponding to the function declaration when the operator type is unstructured.

[0244] The second function determines the submodule, which is used to determine the custom implementation function as the target implementation function of the operator to be adapted.

[0245] Optionally, the above registration module includes:

[0246] The binding function generation submodule is used to generate the target binding function corresponding to the operator to be adapted, based on the function declaration, attribute fields, and target implementation function.

[0247] The registration submodule is used to register the target binding function to the target scheduling key.

[0248] The adapter device for the external device operator provided in this embodiment can implement the method of any of the above embodiments, and its execution mode and beneficial effects are similar, so they will not be described again here.

[0249] This disclosure also provides a computer device, which includes a processor and a memory, wherein the memory stores a computer program. When the computer program is executed by the processor, it can implement the methods of any of the above embodiments. The execution method and beneficial effects are similar, and will not be described again here.

[0250] The computer device in this disclosure can be understood as any device with processing and computing capabilities, including but not limited to electronic devices such as smartphones, laptops, tablets, in-vehicle terminals, wearable devices, digital TVs, desktop computers, smart home devices, etc.

[0251] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure, such as... Figure 11 As shown, the computer device 1100 may include a processor 1110 and a memory 1120. The memory 1120 stores a computer program 1121. When the computer program 1121 is executed by the processor 1110, it can implement the method provided in any of the above embodiments. The execution mode and beneficial effects are similar and will not be described again here.

[0252] Of course, for the sake of simplicity, Figure 11Only some of the components of the computer device 1100 relevant to the present invention are shown in this illustration; components such as buses, input / output interfaces, input devices, and output devices are omitted. In addition, the computer device 1100 may include any other suitable components depending on the specific application.

[0253] This disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the methods of any of the above embodiments. The execution method and beneficial effects are similar, and will not be described again here.

[0254] The aforementioned computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0255] The computer program described above can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer device, partially on the user's device, as a standalone software package, partially on the user's computer device and partially on a remote computer device, or entirely on a remote computer device or server.

[0256] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0257] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0258] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for adapting an external device operator, characterized in that, include: Obtain the target configuration file of the operator to be adapted from the external device, wherein the target configuration file includes attribute fields corresponding to the function declarations of the operator to be adapted; Determine the operator type of the operator to be adapted; When the operator type is a structured operator, the attribute field corresponding to the main function declaration is read from the attribute field corresponding to the function declaration, and the target implementation function of the operator to be adapted is determined based on the attribute field corresponding to the main function declaration; wherein, the main function declaration is the declaration of the main function for the operator to be adapted, and the main function is an overloaded function with an output value; When the operator type is an unstructured operator, if the source name bound to the target scheduling key does not exist in the attribute field corresponding to the function declaration, then the native implementation function corresponding to the operator to be adapted is obtained, and the native implementation function is determined as the target implementation function of the operator to be adapted. The code generation tool is invoked to register the target implementation function to the target scheduling key in the target deep learning framework, wherein the target scheduling key is the scheduling key corresponding to the backend of the external device.

2. The method according to claim 1, characterized in that, The process of obtaining the target configuration file of the operator to be adapted to the external device includes: In response to the configuration file creation operation for the operator to be adapted to the external device, a target configuration file for the operator to be adapted is created.

3. The method according to claim 1, characterized in that, Determining the operator type of the operator to be adapted includes: Read the native operator configuration file; In the native operator configuration file, find the native adaptation type of the operator to be adapted; The native adaptation type of the operator to be adapted is determined as the operator type of the operator to be adapted.

4. The method according to claim 1, characterized in that, The target configuration file also includes the operator type of the operator to be adapted; Determining the operator type of the operator to be adapted includes: Read the operator type of the operator to be adapted from the target configuration file.

5. The method according to claim 1, characterized in that, The step of determining the target implementation function of the operator to be adapted based on the attribute fields corresponding to the main function declaration includes: If the source name and structured inheritance field bound to the target scheduling key are not present in the attribute fields corresponding to the main function declaration, then the native preprocessing function and native computation function corresponding to the operator to be adapted are obtained; Based on the native preprocessing function and the native computation function, the target implementation function of the operator to be adapted is generated.

6. The method according to claim 1, characterized in that, The step of determining the target implementation function of the operator to be adapted based on the attribute fields corresponding to the main function declaration includes: If the source name bound to the target scheduling key exists in the attribute field corresponding to the main function declaration and there is no structured inheritance field, then the native preprocessing function corresponding to the operator to be adapted is obtained, and the custom calculation function of the operator to be adapted is read from the target configuration file; Based on the native preprocessing function and the custom calculation function, the target implementation function of the operator to be adapted is generated.

7. The method according to claim 1, characterized in that, The step of determining the target implementation function of the operator to be adapted based on the attribute fields corresponding to the main function declaration includes: If the attribute field corresponding to the main function declaration contains the source name and structured inheritance field bound to the target scheduling key, then the custom preprocessing function and custom calculation function of the operator to be adapted are read from the target configuration file; Based on the custom preprocessing function and the custom computation function, the target implementation function of the operator to be adapted is generated.

8. The method according to claim 1, characterized in that, The method further includes: When the operator type is an unstructured operator, if the source name bound to the target scheduling key exists in the attribute field corresponding to the function declaration, then the custom implementation function of the operator to be adapted is read from the target configuration file; The custom implementation function is determined as the target implementation function of the operator to be adapted.

9. The method according to any one of claims 1-8, characterized in that, The step of registering the target implementation function to the target scheduling key in the target deep learning framework includes: Based on the function declaration, the attribute fields, and the target implementation function, generate the target binding function corresponding to the operator to be adapted; Register the target binding function to the target scheduling key.

10. An adapter for an external device operator, characterized in that, include: The acquisition module is used to acquire the target configuration file of the operator to be adapted from the external device, wherein the target configuration file includes attribute fields corresponding to the function declarations of the operator to be adapted; The first determining module is used to determine the operator type of the operator to be adapted; The second determining module is configured to, when the operator type is a structured operator, read the attribute field corresponding to the main function declaration from the attribute field corresponding to the function declaration, and determine the target implementation function of the operator to be adapted based on the attribute field corresponding to the main function declaration; wherein, the main function declaration is the declaration of the main function for the operator to be adapted, and the main function is an overloaded function with an output value; when the operator type is an unstructured operator, if the source name bound to the target scheduling key does not exist in the attribute field corresponding to the function declaration, then obtain the native implementation function corresponding to the operator to be adapted, and determine the native implementation function as the target implementation function of the operator to be adapted; The registration module is used to call the code generation tool to register the target implementation function to the target scheduling key in the target deep learning framework, wherein the target scheduling key is the scheduling key corresponding to the backend of the external device.

11. A computer device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the adaptation method for an external device operator as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the adaptation method for external device operators as described in any one of claims 1-9.

Images