Method, computing device, medium and program product for determining a matching operator

Abstract

The present invention relates to a method, a computing device, a medium and a program product for determining a matching operator. The method comprises: obtaining attribute combination information about a computing task, wherein the attribute combination information is used to indicate a plurality of attribute information related to the computing task; establishing a mapping relationship between the attribute combination information and a function pointer in a key-value pair manner; determining a corresponding function pointer based on the attribute combination information via a determined specialization template function; and calling an operator matching the computing task based on the determined corresponding function pointer. The present invention can accurately and quickly seek matching operators for different application scenarios.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of artificial intelligence, and more specifically, to a method, a computing device, a computer-readable storage medium, and a computer program product for determining a matching operator. Background Art

[0002] Due to the increasing complexity and diversity of AI application scenarios (for example, AI application scenarios involve image processing, speech recognition, advertising recommendations, etc.), large-scale operations are required. These operation functions are reflected in the software as so-called "operators". It should be understood that different application scenarios usually require different parameter configurations, the corresponding software operator implementation methods are different, and the hardware resources used are also different. If the corresponding operator cannot be called correctly, it will cause an error or make the program unable to execute. Therefore, it is necessary to select an adaptive operator to utilize different hardware and other resources to achieve the best performance effect.

[0003] Traditional methods for determining matching operators mainly use a manually maintained "if else" conditional branch structure to select operators that match the application scenario based on parameter combinations related to operator adaptation. As application scenarios become more complex, the number of related parameter combinations is increasing, which results in a longer search time for matching operators in the software library; at the same time, the error probability of the "if else" conditional branch structure will also increase.

[0004] In summary, the traditional method for determining the matching operator has the disadvantage that it is difficult to quickly and accurately determine the matching operator for different application scenarios. Summary of the invention

[0005] The present invention provides a method, a computing device, a computer-readable storage medium and a computer program product for determining a matching operator, which can accurately and quickly find a matching operator for different application scenarios.

[0006] According to a first aspect of the present invention, a method for determining a matching operator is provided. The method comprises: obtaining attribute combination information about a computing task, the attribute combination information being used to indicate a plurality of attribute information related to the computing task; establishing a mapping relationship between the attribute combination information and a function pointer in a key-value pair manner; determining a corresponding function pointer based on the attribute combination information via a determined specialization template function; and calling an operator matching the computing task based on the determined corresponding function pointer.

[0007] In some embodiments, the plurality of attribute information includes multiple items of: storage architecture, broadcast mode, shape type, and data type.

[0008] In some embodiments, the operator matched with the computing task is a kernel function matched with the computing task, and the specialized template function is determined through specialization of a function template.

[0009] In some embodiments, the method for determining matching operators also includes: determining whether there is a lack of operators that support the computing task; in response to determining that there is a lack of operators that support the computing task, calling a default function; in response to the default function being called, generating error information indicating that the operator does not support the computing task; and in response to determining that there is no lack of operators that support the computing task, specializing the function template based on attribute combination information of the computing task to generate a specialized template function, and the specialized template function is used to call the operator that supports the computing task.

[0010] In some embodiments, obtaining attribute combination information about a computing task includes: for multiple attribute information, determining multiple attribute weights corresponding to the multiple attribute information respectively, the attribute weight being greater than or equal to a predetermined value, and the attribute weights corresponding to different types of attribute information are different from each other; and based on the multiple attribute information about the computing task and the determined multiple attribute weights, calculating the attribute combination information about the current computing task.

[0011] In some embodiments, establishing a mapping relationship between the attribute combination information and the function pointer by means of a key-value pair includes: establishing a mapping relationship between the calculated attribute combination information and the function pointer by means of a hash table or a red-black tree.

[0012] In some embodiments, determining multiple attribute weights corresponding to the multiple attribute information includes: determining multiple attribute weights corresponding to the multiple attribute information so that the weighted sum of each attribute information and the corresponding attribute weight corresponding to different attribute information combinations is different.

[0013] In some embodiments, the specialized template function is determined by: determining the specialized template function via a compiler based on template parameters of the function template and a predetermined table, wherein the predetermined table indicates the correspondence between the template parameters of the function template and the default function and the specialized template function.

[0014] In some embodiments, the method for determining a matching operator also includes: in response to adding a supporting operator for a computing task, adding a new specialized template function based on attribute combination information of the computing task; and, via a compiler, adding an entry corresponding to the added new specialized template function in a predetermined table.

[0015] In some embodiments, the number of function pointers is equal to the number of specialized template functions plus one.

[0016] According to a second aspect of the present invention, a computing device is also provided. The computing device comprises: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the computing device can perform the method of the first aspect of the present invention.

[0017] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a machine, the method of the first aspect of the present invention is executed.

[0018] According to a fourth aspect of the present invention, there is further provided a computer program product, comprising a computer program, wherein when the computer program is executed by a machine, the method of the first aspect of the present invention is performed.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements.

[0021] Figure 1 The figure schematically shows a schematic diagram of a computing device implementing a method for determining a matching operator according to an embodiment of the present invention.

[0022] Figure 2 A flow chart of a method for determining a matching operator according to an embodiment of the present invention is shown.

[0023] Figure 3 A flow chart of a method for establishing a mapping relationship between attribute combination information and function pointers according to an embodiment of the present invention is shown.

[0024] Figure 4 A flowchart of a method for generating a specialized template function according to an embodiment of the present invention is shown.

[0025] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0026] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0027] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects.

[0028] As described above, in the traditional method for determining matching operators, for different application scenarios, especially in artificial intelligence-related application scenarios, an operator matching the application scenario is usually selected through a manually maintained conditional branch structure of "if else". The following Table 1 schematically shows the code of the traditional method for determining matching operators.

[0029] Table 1

[0030]

[0031] It should be understood that in artificial intelligence-related application scenarios, the number of parameter types related to operator adaptation and the number of parameters included in each parameter type are very large. For example, taking data types as an example, it includes at least four types: fp32, bf16, fp16, int8, etc. For storage architecture, it includes two modes: UMA and NUMA. For broadcast mode, it includes four situations: A matrix broadcasts but B matrix does not broadcast, B matrix broadcasts but A matrix, A matrix and B matrix are broadcast, and A matrix and B matrix are not broadcast. Therefore, considering only the case of one matrix A, there are already 4*2*4 = 16 different parameter combinations. In the actual calculation process, at least three matrices are involved, so there will be 16*3 = 48 different parameter combinations. Therefore, each additional type of parameter will cause the number of parameter combinations to grow rapidly. In the traditional method for determining the matching operator, for different parameter combinations, as shown in Table 1, the "if else" conditional branch structure is used to select the appropriate operator that matches the parameter combination. Because of the large number of parameter combinations, not only will the maintenance of the operator-related code of the software library become very difficult, but also the search for the matching operator for the software library will be extremely time-consuming due to the large number of search paths. In addition, during the execution of different operators, the error probability of the "if else" conditional branch structure adopted will also increase.

[0032] In summary, the traditional method for determining the matching operator has the disadvantage that it is difficult to quickly and accurately determine the matching operator for different application scenarios.

[0033] In order to at least partially solve one or more of the above problems and other potential problems, an exemplary embodiment of the present invention proposes a method for determining a matching operator. In this method, a mapping relationship between the acquired attribute combination information (which indicates a variety of attribute information related to a computing task) and a function pointer is established in a key-value pair manner; and based on the attribute combination information, the corresponding function pointer is determined through the determined special template function. The present invention can match a large amount of different attribute combination information related to computing tasks in different application scenarios to a limited number of function pointers through special template functions, and call the operator matching the computing task through the determined function pointer, thereby significantly reducing the cost and time of searching for matching operators. In addition, because there is no need to use the "if else" conditional branch structure to select a suitable operator that matches the parameter combination, the problem of easy error is avoided. Therefore, the present invention can accurately and quickly seek matching operators for different application scenarios.

[0034] Figure 1A schematic diagram of a computing device 100 for implementing a method for determining a matching operator according to an embodiment of the present invention is shown schematically. Figure 1 As shown, the computing device 100 may have one or more processing units and a memory. The one or more processing units include, for example, a special processing unit such as a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a general-purpose computing on graphics processing units (GPGPU), and a general-purpose processing unit such as a CPU. It should be understood that the one or more processing units configured in the computing device 100 also run multiple software modules. The software modules include, for example: a computing task attribute combination information acquisition unit 102, an attribute combination information and function pointer mapping unit 104, a corresponding function pointer determination unit 106, and a matching operator calling unit 108.

[0035] The attribute combination information acquisition unit 102 about the computing task is used to acquire the attribute combination information about the computing task, where the attribute combination information is used to indicate a variety of attribute information related to the computing task.

[0036] The attribute combination information and function pointer mapping unit 104 is used to establish a mapping relationship between the attribute combination information and the function pointer in a key-value pair manner.

[0037] The corresponding function pointer determining unit 106 is used to determine the corresponding function pointer via the determined specialization template function based on the attribute combination information.

[0038] The matching operator calling unit 108 is used to call the operator matching the computing task based on the determined corresponding function pointer.

[0039] The following will be combined Figure 2 A method 200 for determining a matching operator according to an embodiment of the present invention is described. Figure 2 FIG. 2 is a flowchart of a method 200 for determining a matching operator according to an embodiment of the present invention. It should be understood that the method 200 may be implemented in Figure 1 The method 200 is executed at the described computing device 100. The method 200 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0040] At step 202 , the computing device 100 obtains attribute combination information about a computing task, where the attribute combination information is used to indicate a variety of attribute information related to the computing task.

[0041] Regarding various attribute information, it includes, for example, multiple items of the following items: a storage architecture (MemArch), a broadcast mode (BroadCastMode), a shape type (ShapeType), and a data type (DataType).

[0042] Regarding the attribute combination information, it is used, for example, to indicate a variety of attribute information related to the computing task. In some embodiments, the attribute combination information is configured as a string type, for example. For example, the attribute combination information is MemArch_BroadCastMode_ShapeType. The attribute key value, for example, includes a plurality of different attribute information, and different attribute information is connected, for example, with underscores. Specifically, the attribute combination information is, for example, ALL_UMA_LtAiB_Normal, or NUMA_LtAiBi_Large. In some embodiments, the attribute combination information, for example, indicates different search paths of a binary tree or a multi-fork tree.

[0043] Regarding the attribute information of storage architecture, it indicates, for example but not limited to, different storage architectures such as ALL_UMA and NUMA, where ALL_UMA means that the storage architectures of the A matrix, the B matrix, and the C matrix are all of the UMA type.

[0044] Regarding the broadcast mode, it indicates, for example but not limited to, different broadcast modes such as LtAiB, LtAiBi, etc. Among them, LtAiB, for example, represents that the A matrix has batches, while the B matrix does not have batches, and the B matrix is ​​broadcast to the matrix. LtAiBi, for example, represents that neither the A matrix nor the B matrix is ​​broadcast.

[0045] Regarding the shape type, it indicates, for example but not limited to: Normal, Large, etc., where Large, for example, means that at least one dimension of the matrix exceeds a predetermined standard (for example, the predetermined standard is, for example but not limited to, 8192). Normal means that no dimension exceeds the predetermined standard.

[0046] Regarding the shape type, it includes at least: fp32, bf16, fp16, int8 and other types.

[0047] In some embodiments, each type of attribute information is represented as a corresponding enumeration value according to its corresponding situation. For example, when the storage architecture attribute information is ALL_UMA, it is represented as "1". When the storage architecture indicates NUMA, for example, it is represented as "2". When the broadcast mode indicates LtAiB, for example, the corresponding attribute information is represented as "1". When the broadcast mode indicates LtAiBi, for example, the corresponding attribute information is represented as "2". When the shape type indicates Normal, for example, the corresponding attribute information is represented as "1". When the shape type indicates Large, for example, the corresponding attribute information is represented as "2". Thus, when the attribute combination information indicates "ALL_UMA_LtAiB_Normal", for example, it is represented as "1_1_1". When the attribute combination information indicates "NUMA_LtAiBi_Large", for example, it is represented as "2_2_2".

[0048] At step 204, the computing device 100 establishes a mapping relationship between the attribute combination information and the function pointer through a key-value pair method.

[0049] For example, the computing device 100 establishes a mapping relationship between attribute combination information and function pointers in the form of key-value pairs. Among them, Key is, for example, attribute combination information, which is used to indicate various attribute combinations required by hardware and applications. In some embodiments, the number of attribute combination information is large. Value is, for example, a function pointer, which is used to indicate the address (or "entry") of the corresponding operator. In some embodiments, Value is, for example, a function pointer of a C++ template function.

[0050] Specifically, in some embodiments, the method for establishing the mapping relationship between the attribute key value and the function pointer includes, for example: using a red-black tree (std::map) or a hash table (std::unordered_map) to establish a mapping relationship between the attribute combination information and the function pointer (for example, a mapping relationship in the form of a key-value pair). It should be understood that whether the red-black tree or the hash table is used, the mapping between the key-value pairs is unique. In addition, although there are differences in the internal implementation of storing the mapping relationship between the attribute combination information and the function pointer using a red-black tree or a hash table, the external interfaces of the two can be configured to be consistent. In the red-black tree method, if the same Key already exists, data cannot be inserted. If the corresponding Key is not found, this search will report an error. The following table 2 schematically shows an exemplary pseudo code for establishing a mapping relationship between the attribute combination information and the function pointer using a red-black tree. The pseudo code of Table 2 illustrates the mapping between the function pointer of the operator "L4NodesFunc" and the attribute combination information.

[0051] Table 2

[0052]

[0053] At step 206, the computing device 100 determines a corresponding function pointer via the determined specialized template function based on the attribute combination information.

[0054] In some embodiments, the number of function pointers is equal to the number of specialization template functions plus one. The function pointers of the present invention include, for example: a function pointer corresponding to the default function, and a function pointer corresponding to each specialization template function. Table 5 below illustrates two specialization template functions, namely, specialization template function 1 and specialization template function 2. A first function pointer corresponding to specialization template function 1, a second function pointer corresponding to specialization template function 2, and a third function pointer corresponding to the default function are configured.

[0055] For example, regarding the corresponding method of determining the function pointer, it is implemented, for example, via a search function. For example, the computing device 100 searches for a function pointer corresponding to the string_name in a search set (e.g., std::map) based on the string_name used to indicate the attribute combination information. If the computing device 100 confirms that the string_name used to indicate the attribute combination information is not found, an error message is indicated. The following Table 3 schematically shows an exemplary pseudo code for determining the corresponding function pointer.

[0056] Table 3

[0057]

[0058] Regarding the specialized template function, it is determined, for example, through the specialization of the function template. In some embodiments, regarding the method of generating a specialized template function, it includes, for example: determining whether there is a lack of operators that support the computing task; in response to determining that there is a lack of operators that support the computing task, calling a default function; in response to the default function being called, generating an error message indicating that the operator does not support the computing task; and in response to determining that there is no lack of operators that support the computing task, specializing the function template based on the attribute combination information of the computing task to generate a specialized template function, and the specialized template function is used to call the operator that supports the computing task. The following will be combined with Figure 4 The method for generating a specialized template function is described in detail and will not be repeated here.

[0059] It should be understood that if you want to support a certain combination of attributes, you need to perform function template specialization to generate a specialized template function. Regarding the method of determining the specialized template function, it includes, for example: the computing device 100 determines the specialized template function based on the template parameters of the function template and the predetermined table via the compiler. The predetermined table is used to indicate different template parameters and function instances corresponding to the template parameters. Specifically, it indicates, for example, the correspondence between the template parameters of the function template and the default function and the specialized template function. It should be understood that the predetermined table can be transparent to the developer and does not require the developer to care. When the program is running, the C++ program will match the corresponding template parameters to call the default function or the specialized template function.

[0060] The correspondence between the function to be called (which includes, for example, a default function or a specialized template function) and the template parameter must be uniquely matched. If it is not uniquely matched or cannot be matched, a compilation error will occur. Therefore, in the predetermined table, the template parameter can only correspond to the default function or the specialized template function, and there is no other corresponding path. In some embodiments, before adding a specialized template function each time, the computing device 100 determines whether the correspondence between the template parameter (or attribute combination) and the default function or the specialized template function is unique; in response to determining that the correspondence between the template parameter (or attribute combination) and the default function or the specialized template function is not unique, a compilation error indication is generated.

[0061] In some embodiments, the method 200 further includes: in response to adding a support operator for a computing task, adding a new specialization template function based on the attribute combination information of the computing task; and adding an entry corresponding to the added new specialization template function in a predetermined table via a compiler. Thus, the present invention facilitates incremental development and simplifies code maintenance.

[0062] At step 208 , the computing device 100 calls the operator that matches the computing task based on the determined corresponding function pointer.

[0063] The operator matching the computing task is, for example, a kernel function matching the computing task.

[0064] Regarding the kernel function matching the computing task, it is, for example, configured as a sub-function at the address pointed to by the function pointer.

[0065] In the above scheme, the present invention establishes a mapping relationship between the acquired attribute combination information (which indicates multiple attribute information of the computing task) and the function pointer; and based on the attribute combination information, determines the function pointer through the determined special template function. The present invention can match a large amount of different attribute combination information corresponding to computing tasks in different application scenarios to a limited number of corresponding function pointers through the special template function, and call the operator matching the computing task through the determined corresponding function pointer. As a result, the cost and time of searching for matching operators are significantly reduced. As a result, the cost and time of searching for matching operators can be significantly reduced. In addition, because there is no need to use the "if else" conditional branch structure to select a suitable operator that matches the parameter combination, the problem of easy errors is avoided.

[0066] In the traditional method for determining the matching operator, since there are many parameter combinations corresponding to different application scenarios, there are many search paths for the operator (taking the traditional binary tree matching operator as an example, the leaf nodes correspond to the operator, and each parameter combination constitutes the search path). Therefore, finding the matching operator for different application scenarios according to the traditional method requires a lot of search cost and time, and is prone to errors in the search results.

[0067] The measured data of the present invention show that the time required to accurately match the required function for different scenarios using the method for determining the matching operator of the present invention is less than 1 us. Therefore, the present invention can quickly and accurately match the function required by the calculation task.

[0068] The following will be combined Figure 3 A method 300 for establishing a mapping relationship between attribute combination information and function pointers according to an embodiment of the present invention is described. Figure 3 FIG. 3 is a flowchart of a method 300 for establishing a mapping relationship between attribute combination information and function pointers according to an embodiment of the present invention. It should be understood that the method 300 may be used, for example, in Figure 1 The method 300 is executed at the described computing device 100. The method 300 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0069] At step 302, the computing device 100 determines, for various types of attribute information, a plurality of attribute weights respectively corresponding to the various types of attribute information, wherein the attribute weights are greater than or equal to a predetermined value, and the attribute weights corresponding to different types of attribute information are different from each other.

[0070] It should be understood that different attribute information (such as but not limited to: storage architecture, broadcast mode, shape type and data type) is usually represented as enumerated integer values ​​in the system, such as 1, 2, 3, etc., and these integer values ​​are very close. If these integer values ​​used to represent attribute information are simply connected or added, the hash values ​​after different attribute combinations may be the same, which can easily lead to hash collisions. It should be understood that hash collisions can cause query results to be incorrect, seriously affecting the availability of the hash table.

[0071] In order to prevent hash conflicts, the present invention needs to determine multiple attribute weights corresponding to the multiple attribute information for multiple attribute information. Regarding the method for determining multiple attribute weights corresponding to the multiple attribute information, for example, it includes: the computing device 100 determines multiple attribute weights corresponding to the multiple attribute information, so that the weighted summation results of each attribute information corresponding to different attribute information combinations and the corresponding attribute weights are different. For example, the attribute weight corresponding to the storage architecture is "10", the attribute weight corresponding to the broadcast mode is "100", the attribute weight corresponding to the shape type is "1000", and the attribute weight corresponding to the data type is, for example, "10000".

[0072] At step 304, the computing device 100 calculates attribute combination information about the current computing task based on the various attribute information about the computing task and the determined attribute weights. By adopting the above means, the attribute information is integer-multiplied to prevent hash collisions.

[0073] The method for calculating the attribute combination information includes, for example, performing weighted summation based on the multiple attribute information about the computing task and the determined multiple attribute weights to obtain the attribute combination information about the computing task. For example, if the current attribute combination information about the computing task is "MemArch Bro_adCast _shape_type", the value of the attribute combination information about the computing task obtained is, for example, MemArch * 10, BroadCast * 100, shape_type *1000.

[0074] At step 306, the computing device 100 uses a hash table to establish a mapping relationship between the calculated attribute combination information and the function pointer.

[0075] By adopting the above-mentioned technical means, the present invention makes the attribute weight greater than or equal to the predetermined value, and the attribute weights corresponding to different attribute information are different from each other, so that the value of the attribute combination information obtained about the computing task can be a relatively large discrete value, thereby avoiding query result errors caused by hash conflicts.

[0076] The following will be combined Figure 4 A method 400 for generating a specialized template function according to an embodiment of the present invention is described. Figure 4 FIG. 4 is a flowchart of a method 400 for generating a specialization template function according to an embodiment of the present invention. It should be understood that the method 400 can be used, for example, in Figure 1 The method 400 is executed at the described computing device 100. The method 400 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0077] At step 402 , the computing device 100 determines whether there is a lack of operators supporting the computing task.

[0078] At step 404, if the computing device 100 determines that there is a lack of operators that support the computing task, a default function is called.

[0079] Regarding the default function, it is used, for example, to indicate the entry of a function when there is a lack of supporting operators. In some embodiments, if the user calls an unsupported template combination, the present invention will call the address of the default function (i.e., the branch entry of the default function) in the C++ program.

[0080] It should be understood that the number of attribute combination information corresponding to the Key is very large, so generally speaking, the number of Values ​​(i.e., function pointers) corresponding to the attribute combination information is also very large. Due to the difference in AI models, in actual applications, it is not necessary to generate corresponding codes for the Values ​​(i.e., function pointers) of all attribute combinations at the beginning. Instead, incremental development or code generation can be performed according to application requirements. For example, there are 100 kinds of attribute combination information. The called operator (e.g., kernel function) can be opened for some of the attribute combination information (e.g., the example in Table 5 supports 2 kinds of attribute combination information), and the called operator can be temporarily closed for the remaining attribute combination information, thereby facilitating incremental development or incremental code generation. It should be understood that the function interface of the operator is actually very similar except that the corresponding Key value is different. Therefore, the present invention uses the C++ template method to achieve the unification of the function interface. And the default function is configured so that the remaining attribute combinations all take the same entrance "error entrance".

[0081] At step 406, if the computing device 100 determines that the default function is called, an error message is generated to indicate that the operator does not support the computing task.

[0082] Table 4 below schematically shows an exemplary pseudo code for constructing a default function. For example, for the attribute combination information "NUMA_LtABi_Normal", by querying a table indicating the mapping relationship between the attribute combination information and the function pointer, it is determined that the function pointer (e.g., void Node_func_ptr (function parameter 1, function parameter 2, ...)) points to the default function. As indicated by the pseudo code in Table 4: If it is determined that the default function is called, an error message is generated to indicate that the operator does not support the computing task.

[0083] Table 4

[0084]

[0085] At step 408, if the computing device 100 determines that there is no lack of operators supporting the computing task, the function template is specialized based on the attribute combination information of the computing task to generate a specialized template function, and the specialized template function is used to call the operator supporting the computing task.

[0086] It should be understood that if you want an operator to support a certain combination of properties (or "no lack of operators that support computing tasks"), you need to perform function template specialization.

[0087] Regarding the function template, for example, it is a C++ function template. In some embodiments, a predetermined table is implicitly constructed for the C++ template function via the compiler. The predetermined table indicates the template parameters of the function template, and the corresponding relationship between the default function and the specialization template function.

[0088] Regarding the template parameter, it corresponds to the attribute combination information, for example. In some embodiments, the template parameter is, for example, "ComputeMemoryPair", "BroadcastType", and "ShapeType" as shown in Table 5.

[0089] Regarding the way of specializing the function template, for example, it includes: when the computing device 100 compiles the program, the compiler determines the specialization template function based on the template parameters of the function template and the predetermined table, wherein the predetermined table indicates the corresponding relationship between the template parameters of the function template and the default function and the specialization template function. In some embodiments, the generated specialization template function includes a sub-function for calling the corresponding kernel function. The following Table 5 schematically shows an exemplary pseudo code for specializing the template function.

[0090] Table 5

[0091]

[0092] For example, the specialization template function 1 shown in Table 5 includes a sub-function for calling the kernel function Kernel 1. The specialization template function 2 shown in Table 5 includes a sub-function for calling the kernel function Kernel 2.

[0093] For another example, the exemplary code in Table 5 indicates that, for the attribute combination information "ALL_UMA_LtAiBi_Normal", the computing device 100 determines that there is no lack of operators supporting the computing task, and then uses the corresponding specialization template function 1, for example, the corresponding function pointer points to " , , ", and finally calls the kernel function Kernel1 on the GPU corresponding to the attribute combination information "ALL_UMA_LtAiBi_Normal".

[0094] The exemplary code in Table 5 also indicates, for example, that for the attribute combination information "NUMA_LtAiBi_Normal", the corresponding specialization template function 2 is used, for example, the corresponding function pointer points to " , , ", and finally calls the kernel function Kernel2 on the GPU that corresponds to the attribute combination information "NUMA_LtAiBi_Normal".

[0095] By adopting the above means, the present invention uses the default function to report an error when there is a lack of operators, and uses a specialized template function to determine the function entry when there is no lack of supporting operators. Therefore, it is suitable for scenarios of incremental code development or code generation.

[0096] The various processes and processing described herein, such as methods 200, 300, and 400, may be performed at a computing device. The computing device, for example, includes: at least one processor (at least one graphics processor and at least one central processing unit); and a memory connected to at least one processor in communication; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor. In some embodiments, methods 200, 300, and 400 may be implemented as a computer software program or program product, which is tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on a computing device via a read-only memory (ROM) and / or a communication unit. When the computer program is loaded into a random access memory (RAM) and executed by a GPU and a CPU, one or more actions of methods 200, 300, and 400 described above may be performed.

[0097] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for executing various aspects of the present invention. The computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0098] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each box of the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0099] These computer-readable program instructions can be provided to a central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the central processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0100] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the function involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0101] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0102] The above specific implementations do not constitute a limitation on the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made according to design requirements and other factors.

Claims

1. A method for determining a matching operator, characterized in that: The method comprises: Acquire attribute combination information about a computing task, the attribute combination information being used to indicate a plurality of attribute information related to the computing task, the plurality of attribute information including a plurality of the following items: a storage architecture, a broadcast mode, a shape type, and a data type, wherein acquiring the attribute combination information about the computing task comprises: for the plurality of attribute information, determining a plurality of attribute weights respectively corresponding to the plurality of attribute information, and the attribute weights corresponding to different types of attribute information are different from each other; and calculating the attribute combination information about the current computing task based on the plurality of attribute information about the computing task and the determined plurality of attribute weights; A mapping relationship between attribute combination information and function pointers is established through key-value pairs; Based on the attribute combination information, a corresponding function pointer is determined through the determined specialization template function, where the function pointer is used to indicate an address of a corresponding operator, wherein the specialization template function is determined through specialization of a function template; and Based on the determined corresponding function pointer, the operator matching the computing task is called.

2. The method according to claim 1, characterized in that The operator matched with the computing task is a kernel function matched with the computing task.

3. The method according to claim 1, characterized in that Also includes: Determine whether there is a lack of operators to support the computing task; In response to determining that there is a lack of operators supporting the computation task, calling a default function; In response to the default function being called, generating error information indicating that the operator does not support the computing task; as well as In response to determining that there is no lack of operators supporting the computing task, a function template is specialized based on attribute combination information of the computing task to generate a specialized template function, wherein the specialized template function is used to call the operator supporting the computing task.

4. The method according to claim 1, characterized in that: The attribute weight is greater than or equal to a predetermined value.

5. The method according to claim 4, characterized in that The mapping relationship between attribute combination information and function pointer is established by key-value pair method, including: A hash table is used to establish a mapping relationship between the calculated attribute combination information and the function pointer.

6. The method according to claim 4, characterized in that Determining the multiple attribute weights respectively corresponding to the multiple attribute information includes: A plurality of attribute weights respectively corresponding to the plurality of attribute information are determined so that weighted summation results of the attribute information and the corresponding attribute weights corresponding to different attribute information combinations are different.

7. The method according to claim 1, characterized in that The specialization template function is determined by: A specialized template function is determined via a compiler based on the template parameters of the function template and a predetermined table, wherein the predetermined table indicates a correspondence between the template parameters of the function template and the default function and the specialized template function.

8. The method according to claim 3, characterized in that Also includes: In response to adding a supporting operator for a computing task, adding a new specialization template function based on attribute combination information of the computing task; as well as Through the compiler, an entry corresponding to the added new specialized template function is added to the predetermined table.

9. The method according to claim 1, characterized in that: The number of function pointers is equal to the number of specializations of the template function plus one.

10. A computing device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a machine, the method according to any one of claims 1 to 9 is performed.

12. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a machine, performs the method according to any one of claims 1 to 9.

Images