Operator execution method and device

Abstract

The invention provides an operator execution method and device, and the method comprises the steps: determining a target kernel from a plurality of kernels with incomplete functions based on the operation parameters of an operator; based on the operation parameters, determining first performance of the target kernel when the operator is executed, and based on the description information of the operator, determining second performance of the function-complete kernel when the operator is executed; based on the first performance and the second performance, selecting an execution kernel from the target kernel and the function-complete kernel; the operator is executed on the execution kernel. According to the operator execution method and device provided by the invention, the target kernel meeting the operator execution requirement is selected from the plurality of kernels with incomplete functions, then the first performance when the target kernel executes the operator is compared with the second performance when the kernel with complete functions executes the operator, and the kernel with better performance is selected as the execution kernel; the optimal performance kernel execution operator is selected from the incomplete-function kernel and the complete-function kernel, and the operator execution efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an operator execution method and device. Background Art

[0002] In the field of artificial intelligence and deep learning, operators are basic units that perform specific mathematical operations, such as matrix multiplication, addition, activation functions, etc. On artificial intelligence chips, kernels are usually compiled into low-level machine code or microcode to implement the specific operation logic of operators. In order to fully utilize the computing power of artificial intelligence chips and obtain better performance, programmers usually write multiple kernels with different optimization schemes for different computing scenarios, including fully functional kernels and incomplete functional kernels.

[0003] Currently, users can select an execution kernel from multiple kernels by inputting description information of operators. However, the description information input by users is usually insufficient, so users can only select kernels with complete functions as execution kernels, which limits the application of kernels with incomplete functions in specific scenarios. Summary of the invention

[0004] The present invention provides an operator execution method and device to solve the defects existing in the related technology.

[0005] The present invention provides an operator execution method, comprising the following steps: Based on the operation parameters of the operator, a target kernel is determined from multiple functionally incomplete kernels; Determining a first performance of the target kernel when executing the operator based on the operating parameters, and determining a second performance of the functionally complete kernel when executing the operator based on the description information of the operator; Selecting an execution kernel from the target kernel and the functionally complete kernel based on the first performance and the second performance; The operator is executed on the execution kernel.

[0006] According to an operator execution method provided by the present invention, the functionally complete kernel is determined from a plurality of candidate functionally complete kernels based on the description information of the operator in the operator selection stage; and the target kernel is determined from the plurality of functionally incomplete kernels based on the operating parameters in the operator execution stage.

[0007] According to an operator execution method provided by the present invention, the target kernel is determined based on at least one of the following steps: In a case where the operator operation indicated by the running parameter does not have a target operation, determining the target kernel from the multiple functionally incomplete kernels; When the currently reserved video memory indicated by the operating parameter is greater than a threshold, the target kernel is determined from the multiple functionally incomplete kernels.

[0008] According to an operator execution method provided by the present invention, when the operator operation indicated by the running parameter satisfies at least one of the amplification parameter being 0 and the participating address being empty, it is determined that the target operation does not exist.

[0009] According to an operator execution method provided by the present invention, the operating parameter is determined based on at least one of a task requirement corresponding to the operator and a currently reserved video memory.

[0010] According to an operator execution method provided by the present invention, the first performance is determined based on the theoretical execution time of the target kernel when executing the operator, and the second performance is determined based on the theoretical execution time of the functionally complete kernel when executing the operator.

[0011] The present invention also provides an operator execution device, comprising the following units: A first determining unit, configured to determine a target kernel from a plurality of functionally incomplete kernels based on an operating parameter of an operator; a second determining unit, configured to determine, based on the operating parameters, a first performance of the target kernel when executing the operator, and to determine, based on the description information of the operator, a second performance of the functionally complete kernel when executing the operator; a kernel selection unit, configured to select an execution kernel from the target kernel and the functionally complete kernel based on the first performance and the second performance; An operator execution unit is used to execute the operator on the execution kernel.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned operator execution methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned operator execution methods.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned operator execution methods.

[0015] The operator execution method and device provided by the present invention can more accurately reflect the requirements and restrictions in the operator execution process because the operation parameters provide more specific and quantitative information, so that the target kernel that is most suitable for executing the operator can be determined from multiple functionally incomplete kernels according to the operation parameters, thereby achieving full utilization of the functionally incomplete kernels. In addition, the present invention first selects a target kernel that meets the operator execution requirements from multiple functionally incomplete kernels, and then compares the first performance of the target kernel when executing the operator with the second performance of the functionally complete kernel when executing the operator, and selects the kernel with better performance as the execution kernel, thereby achieving the selection of the kernel with the best performance from the functionally incomplete kernel and the functionally complete kernel to execute the operator, thereby improving the operator execution efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of the operator execution method provided by the present invention.

[0018] Figure 2 It is a flowchart of another operator execution method provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the operator execution device provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In order to fully utilize the computing power of artificial intelligence chips and obtain better performance, programmers usually write multiple kernels with different optimization schemes for different computing scenarios, including functionally complete kernels and functionally incomplete kernels. The functionally complete kernels and functionally incomplete kernels are defined as follows: Full-featured kernel: This type of kernel can support all relevant calculations of the operator, including basic functions, various post-processing operations (epilogue), etc. It has high flexibility and versatility and can cope with various complex computing requirements. The operator here can be an MMA (Matrix Multiply-Accumulate) operator.

[0023] Incomplete kernels: These kernels only support limited operator functions due to optimization scheme limitations (such as using registers instead of cache for calculation accumulation). Although they may have higher performance in some specific scenarios, they lack versatility.

[0024] In order to select a suitable kernel, related technologies usually provide an application programming interface (API) through which related functions are called, requiring the user to pass in description information about the operator. The API uses a simulation algorithm to select one or more kernels with the best theoretical performance from multiple fully functional kernels and return them to the user. After the user completes the tuning, the user selects one of the kernels to execute the operator to obtain the final calculation result.

[0025] However, the description information provided by the user is usually insufficient and cannot fully reflect all the details of the actual computing scenario. Therefore, when the user selects the kernel independently, only the fully functional kernel participates in the selection and calculation, which limits the application of the incomplete kernel in specific scenarios. Since the incomplete kernel has lower resource consumption in specific scenarios, if the user selects the fully functional kernel independently, it will lead to the problem of wasting resources in scenarios where the incomplete kernel can be used.

[0026] Figure 1 is a flow chart of the operator execution method provided by the present invention, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .

[0027] Step 110: Determine a target kernel from a plurality of functionally incomplete kernels based on the operating parameters of the operator.

[0028] Here, the operator is the operator to be executed. The target kernel refers to the kernel that meets the operator execution requirements and has the best performance, selected from multiple incomplete kernels according to the operator's operating parameters. The operator's operating parameters (runtime parameters) refer to the specific values ​​or settings that the operator relies on during execution to control its behavior or performance. The operating parameters usually determine how the operator processes the input data, as well as the resources used in the processing process, the complexity of the algorithm, and other characteristics. The description information of the operator is usually a textual description of the operator's functions, uses, inputs and outputs, etc. It provides a general understanding of the overall behavior of the operator. Compared with the operating parameters, the description information of the operator focuses more on the function and use of the operator, without involving specific execution details. The functionally complete kernel can calculate the theoretical performance, that is, the second performance, through the description information of the operator.

[0029] For example, for an MMA operator, its operating parameters may include the magnification, the address of the matrix, each value of each matrix, and the predetermined video memory address and size. From these parameters, it can be inferred whether accumulation is required, whether it is in-situ accumulation (matrix multiplication is directly accumulated into the input matrix), whether there is a predetermined video memory size limit, etc. The description information may include the size of the matrix, the data type involved in the calculation, whether there is a matrix transposition, etc.

[0030] In addition, the running parameters are directly related to the execution performance and resource requirements of the operator. Different kernels may have different characteristics such as computing power, memory bandwidth, parallel processing efficiency, etc., which determine their performance under different running parameters. By comparing the performance predictions or actual test results of different kernels under given running parameters, a kernel that best suits the operator execution requirements can be selected as the target kernel. However, the description information lacks sufficient details to guide the selection of kernels. The description information usually only provides a general description of the operator's functions and uses, without involving specific execution details or performance requirements. Therefore, it is impossible to accurately determine which kernel is most suitable for executing a given operator based on the description information alone. In contrast, the running parameters provide more specific and quantitative information, which can more accurately reflect the requirements and limitations of the operator execution process, so that the target kernel that is most suitable for executing the operator can be determined from multiple kernels with incomplete functions based on the running parameters.

[0031] Step 120: Determine a first performance of the target kernel when executing the operator based on the operating parameters, and determine a second performance of the functionally complete kernel when executing the operator based on the description information of the operator.

[0032] Specifically, the first performance refers to the performance of the target kernel when executing the operator. The first performance can also be understood as the performance of the incomplete kernel when executing the operator. The first performance can include processing speed, resource consumption, parallel efficiency, etc. The second performance refers to the performance of the complete kernel when executing the operator. Similarly, the second performance can also include processing speed, resource consumption, parallel efficiency, etc.

[0033] Considering that the first performance is used to characterize the performance of the target kernel when executing the operator, the functionally complete kernel usually provides complete operator support and optimization, which may have functions or characteristics that the target kernel does not have. In order to select the kernel with the best performance, after determining the target kernel, the embodiment of the present invention also needs to determine the first performance of the target kernel when executing the operator and the second performance of the functionally complete operator when executing the operator, so that the first performance can be compared with the second performance to determine the kernel with the best performance. Optionally, a performance evaluation tool or simulator can be used to predict the performance of the target kernel and the functionally complete kernel when executing the operator.

[0034] Step 130: Select an execution kernel from the target kernel and the functionally complete kernel based on the first performance and the second performance.

[0035] Step 140: Execute the operator on the execution kernel.

[0036] Specifically, the execution kernel can be understood as a kernel that meets the operator execution requirements and has the best performance, selected from the target kernel and the fully functional kernel according to the operator's operating parameters. Based on this, a kernel with better performance can be selected from the target kernel and the fully functional kernel as the execution kernel, and the operator is executed on the execution kernel to obtain the calculation result.

[0037] It should be noted that the execution subject of the embodiment of the present invention can be any electronic device that can realize the execution of the operator, for example, it can be any one of a smart phone, a smart watch, a desktop computer, a laptop computer, etc. In the embodiment of the present invention, the operator can be regarded as a component in a neural network model, and the neural network model can be applied to the fields of text generation, image recognition, speech processing, natural language processing, etc. The neural network model can be, for example, a convolutional neural network model (CNN), a recurrent neural network model (RNN), etc.

[0038] Taking the application of a neural network model to text generation as an example, the operator is executed based on the method of the above embodiment, a specific transformation is performed on the input text to obtain a calculation result, and the model performs reasoning based on the calculation result to generate a corresponding output text.

[0039] In addition, compared to the solution of first calculating the performance of all kernels when executing the operator and then selecting the optimal kernel based on the calculated performance, the embodiment of the present invention first selects a target kernel that meets the operator execution requirements from multiple incomplete kernels, and then compares the first performance of the target kernel when executing the operator with the second performance of the complete kernel when executing the operator, and selects the kernel with better performance as the execution kernel. The embodiment of the present invention does not need to calculate the performance of all incomplete kernels when executing the operator, but only needs to calculate the performance of the target kernel when executing the operator, thereby reducing the amount of calculation, saving computing resources and improving the efficiency of operator execution.

[0040] The operator execution method provided by the embodiment of the present invention can more accurately reflect the requirements and restrictions in the operator execution process because the operation parameters provide more specific and quantitative information, so that the target kernel that is most suitable for executing the operator can be determined from multiple functionally incomplete kernels according to the operation parameters, thereby achieving full utilization of the functionally incomplete kernels. In addition, the embodiment of the present invention first selects a target kernel that meets the operator execution requirements from multiple functionally incomplete kernels, and then compares the first performance of the target kernel when executing the operator with the second performance of the functionally complete kernel when executing the operator, and selects the kernel with better performance as the execution kernel, thereby achieving the selection of the kernel with the best performance from the functionally incomplete kernel and the functionally complete kernel to execute the operator, thereby improving the operator execution efficiency.

[0041] Based on the above embodiment, the functionally complete kernel is determined from a plurality of candidate functionally complete kernels based on the description information of the operator in the operator selection stage; The target kernel is determined from multiple incomplete kernels based on the operation parameters during the operator execution phase.

[0042] Specifically, the operator selection phase is used to select an operator according to the user's task requirements, and the operator execution phase is used to execute the selected operator. In the operator selection phase, an operator matching the user's task requirements can be selected by calling an operator selection function, and in the operator execution phase, an operator can be executed by calling an operator execution function and outputting the operation result.

[0043] In the operator selection stage, since there are no specific operating parameters, it is impossible to determine whether the incomplete kernel meets the operator execution requirements based on the operating parameters. Therefore, it is safer to select a complete kernel, which can ensure that the operator can be executed correctly in various situations and avoid potential problems caused by incomplete functions. In addition, in the operator selection stage, the description information of the operator can be obtained, and the functions required by the target kernel can be known based on the description information. Then, according to the description information of the operator, a matching complete kernel can be determined from multiple candidate complete kernels. Optionally, if there are multiple matching complete kernels, the kernel with the best performance can be selected as the final matching complete kernel.

[0044] During the operator execution phase, since specific operating parameters have been generated and the incomplete kernel has lower resource consumption, the target kernel can be determined from multiple incomplete kernels based on the operating parameters, which not only meets the operator execution requirements but also ensures lower resource consumption.

[0045] It can be seen that the embodiment of the present invention adopts different kernel selection strategies according to different stages of the operator selection stage and the operator execution stage, which can not only ensure low resource consumption, but also ensure the correctness and robustness of operator execution.

[0046] Based on any of the above embodiments, the target kernel is determined based on at least one of the following steps: When the operator operation indicated by the running parameter does not have a target operation, determining a target kernel from a plurality of functionally incomplete kernels; When the currently reserved video memory indicated by the operating parameter is greater than a threshold, a target kernel is determined from a plurality of kernels with incomplete functions.

[0047] Specifically, the operator operation is used to describe the specific operation information in the operator. For example, for the matrix addition (matrixadd) operator, the operator operation is used to describe how to add two matrices of the same size, that is, the elements in corresponding positions are added one by one.

[0048] In addition, the target operation can be understood as an operation that consumes a lot of resources, such as an operation involving a large amount of data processing, data transfer, calculation, video memory access, etc. Correspondingly, the reduction of target operations means that the amount of data to be processed, the amount of data transfer, the amount of calculation or the amount of tasks to be performed during the execution of the operator is reduced, and thus fewer resources are consumed.

[0049] If the operator operation indicated by the running parameters does not have a target operation, it means that the task corresponding to the operator does not have a target operation requirement, and fewer resources are required after reducing the target operation. Therefore, the target kernel can be selected from the incomplete kernels with low resource consumption, which can not only meet the operator execution requirements but also ensure low resource consumption.

[0050] For example, if the operator operation indicated by the running parameters does not include a matrix add operation, some data movement and calculation can be reduced, the required resources can be reduced, and the target kernel can be determined from multiple kernels with incomplete functions.

[0051] In addition, the currently reserved video memory indicated by the operation parameters can be understood as the video memory space available when the operator is executed, which can be the video memory on the GPU. The currently reserved video memory indicated by the operation parameters can be used to store the intermediate data generated during the execution of the operator. The larger the currently reserved video memory, the larger the amount of intermediate data that can be stored, so the amount of data required to be transported is less, and space is exchanged for time, which consumes space resources, so the time consumed is shorter.

[0052] If the currently reserved video memory indicated by the running parameters is greater than the threshold, it indicates that there is enough video memory to store the intermediate data generated during the operator execution, so that fewer resources are needed, and the target kernel can be selected from the incomplete kernels with low resource consumption, which can not only meet the operator execution requirements, but also ensure low resource consumption. Among them, workspace = reserved video memory.

[0053] Wherein, when the operator operation indicated by the running parameter does not have the target operation, the functionally complete kernel can also be determined from multiple candidate functionally complete kernels. Similarly, when the currently reserved video memory indicated by the running parameter is greater than a threshold, the functionally complete kernel can also be determined from multiple candidate functionally complete kernels.

[0054] Based on any of the above embodiments, when the operator operation indicated by the running parameter satisfies at least one of the amplification parameter being 0 and the participating address being empty, it is determined that the target operation does not exist.

[0055] Specifically, the magnification parameter can be understood as the coefficient used to scale the matrix elements in the operator operation. If the magnification parameter is 0, no matter what the original value of the matrix is, the result after scaling by the magnification parameter will be 0. A matrix with all zeros will not only not produce any valid results when operated in a matrix operation, but also consume computing resources. Therefore, when the magnification parameter of the operator operation indicated by the running parameter is 0, the target operation is also invalid, and it can be considered that there is no target operation.

[0056] The participating address is used to indicate the location where data is stored. If the participating address is empty, it means that there is no valid data participating in the operation, and thus the target operation cannot be performed. Therefore, when the participating address of the operator operation is empty, it can also be considered that there is no target operation.

[0057] For example, if the amplification parameter of matrix add is 0 or the participant address of matrix add is empty, it means that the matrix add operation is not required, that is, it can be considered that there is no matrix add operation. At this time, the target kernel can be determined from multiple kernels with incomplete functions.

[0058] Based on any of the above embodiments, the operating parameters are determined based on at least one of the task requirements corresponding to the operator and the currently reserved video memory.

[0059] Specifically, the task requirements corresponding to the operator can be understood as the goals that the user expects the operator to achieve after execution. For model training tasks, the task requirements can be computing requirements, and different computing types correspond to different operating parameters. For example, for the reverse calculation of weights, it is first necessary to determine whether there is a matrix add computing requirement, and then further determine the operating parameters based on the computing requirement. The operating parameters may include the address of the input matrix, the matrix magnification during matrix add, whether the matrix add is in situ, etc.

[0060] For example, for the accumulation operation D=A×B+C, if C and D are different matrices, then when performing the accumulation operation, C and D need to be loaded into the GPU memory at the same time. If C and D are the same matrix, only one matrix (C or D) needs to be loaded into the GPU memory, and the required memory space is half of the original.

[0061] If C and D are the same matrix, an incomplete kernel can be designed for this scenario. For example, if C and D are the same matrix, the pipeline during the accumulation operation can be increased, that is, the C matrix required for the next accumulation operation can be loaded into the second half of the video memory space in advance, shortening the waiting time for calculation, thereby shortening the time to process the problem. For another example, if the currently reserved video memory space is limited, in this limited space, if C and D are the same matrix, a kernel specially designed for this scenario can be selected, which has better performance.

[0062] In addition, the currently reserved video memory can be understood as the video memory space available when the operator is executed, and the video memory can be used to store the intermediate data generated during the execution of the operator. In this case, it is necessary to pay attention to the area of ​​the video memory space and the size of the video memory space, so as to determine whether the currently reserved video memory is sufficient to store the intermediate data generated during the execution of the operator. Therefore, the area of ​​the video memory space and the size of the video memory space can be used as operating parameters.

[0063] Based on any of the above embodiments, the first performance is determined based on the theoretical execution time of the target kernel when executing the operator, and the second performance is determined based on the theoretical execution time of the fully functional kernel when executing the operator.

[0064] Specifically, when executing an operator, the performance of the target kernel (such as execution speed, resource consumption, etc.) will directly affect the theoretical execution time of the operator. Therefore, the theoretical execution time of the target kernel when executing the operator is used to evaluate the first performance. A shorter execution time means higher execution efficiency and better performance.

[0065] Similarly, when executing an operator, the performance of the functionally complete kernel (such as execution speed, resource consumption, etc.) will directly affect the theoretical execution time of the operator. Therefore, the theoretical execution time of the functionally complete kernel when executing the operator is used to evaluate the second performance. A shorter theoretical execution time means higher execution efficiency and better performance.

[0066] For example, the theoretical execution time of the target kernel when executing the operator and the theoretical execution time of the fully functional kernel when executing the operator can be calculated by using corresponding mathematical formulas, or the timing code can be manually inserted at the start and end positions of the operator execution to record the execution start time and end time, and the theoretical execution time can be determined based on the two recorded times. In addition, the above theoretical time can also be obtained through a timer, and the theoretical time can be stored in the disk as offline data, so that the theoretical execution time can be directly obtained from the disk next time by matching the description information and operation parameters of the operator.

[0067] Based on any of the above embodiments, Figure 2 is a flowchart of another operator execution method provided by the present invention, such as Figure 2 As shown, the method includes: In the operator selection phase, the operator selection function is called, and based on the description information of the operator, a kernel with the best performance is selected from multiple candidate functionally complete kernels as the matching functionally complete kernel.

[0068] In the operator execution phase, the operator execution function is called to determine whether an incomplete kernel can be used based on the operator's operating parameters. If not, a complete kernel is used as the execution kernel. If yes, a kernel with the best performance is selected from multiple available incomplete kernels as the target kernel.

[0069] Next, a first performance of the target kernel when executing the operator is compared with a second performance of the fully functional kernel when executing the operator, and a kernel with better performance is selected as an execution kernel, which is used to execute the operator.

[0070] The operator execution device provided by the present invention is described below. The operator execution device described below and the operator execution method described above can be referred to in correspondence with each other.

[0071] Based on any of the above embodiments, Figure 3 is a schematic diagram of the structure of the operator execution device provided by the present invention, such as Figure 3 As shown, the device comprises: A first determining unit 310 is used to determine a target kernel from a plurality of functionally incomplete kernels based on an operation parameter of an operator; A second determining unit 320, configured to determine a first performance of the target kernel when executing the operator based on the operating parameters, and to determine a second performance of the functionally complete kernel when executing the operator based on the description information of the operator; A kernel selection unit 330, configured to select an execution kernel from the target kernel and the functionally complete kernel based on the first performance and the second performance; The operator execution unit 340 is used to execute operators on the execution kernel.

[0072] The operator execution device provided by the embodiment of the present invention can more accurately reflect the requirements and restrictions in the operator execution process because the operation parameters provide more specific and quantitative information, so that the target kernel that is most suitable for executing the operator can be determined from multiple functionally incomplete kernels according to the operation parameters, thereby achieving full utilization of the functionally incomplete kernels. In addition, the embodiment of the present invention first selects a target kernel that meets the operator execution requirements from multiple functionally incomplete kernels, and then compares the first performance of the target kernel when executing the operator with the second performance of the functionally complete kernel when executing the operator, and selects the kernel with better performance as the execution kernel, thereby achieving the selection of the kernel with the best performance from the functionally incomplete kernel and the functionally complete kernel to execute the operator, thereby improving the operator execution efficiency.

[0073] Based on any of the above embodiments, the target kernel is determined from multiple functionally incomplete kernels based on operating parameters during the operator execution phase; the functionally complete kernel is determined from multiple candidate functionally complete kernels based on operator description information during the operator selection phase.

[0074] Based on any of the above embodiments, the target kernel is determined based on at least one of the following steps: When the operator operation indicated by the running parameter does not have a target operation, determining a target kernel from a plurality of functionally incomplete kernels; When the currently reserved video memory indicated by the operating parameter is greater than a threshold, a target kernel is determined from a plurality of kernels with incomplete functions.

[0075] Based on any of the above embodiments, when the operator operation indicated by the running parameter satisfies at least one of the amplification parameter being 0 and the participating address being empty, it is determined that the target operation does not exist.

[0076] Based on any of the above embodiments, the operating parameters are determined based on at least one of the task requirements corresponding to the operator and the currently reserved video memory.

[0077] Based on any of the above embodiments, the first performance is determined based on the theoretical execution time of the target kernel when executing the operator, and the second performance is determined based on the theoretical execution time of the fully functional kernel when executing the operator.

[0078] Figure 4is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the operator execution method, which includes: determining a target kernel from a plurality of incomplete kernels based on the operation parameters of the operator; determining a first performance of the target kernel when executing the operator based on the operation parameters, and determining a second performance of a complete kernel when executing the operator based on the description information of the operator; selecting an execution kernel from the target kernel and the complete kernel based on the first performance and the second performance; and executing the operator on the execution kernel.

[0079] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0080] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the operator execution method provided by the above methods, the method including: based on the operating parameters of the operator, determining a target kernel from multiple incomplete kernels; based on the operating parameters, determining a first performance of the target kernel when executing the operator, and based on the description information of the operator, determining a second performance of a complete kernel when executing the operator; based on the first performance and the second performance, selecting an execution kernel from the target kernel and the complete kernel; and executing the operator on the execution kernel.

[0081] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the operator execution method provided by the above-mentioned methods, the method comprising: determining a target kernel from a plurality of incomplete kernels based on the operating parameters of the operator; determining a first performance of the target kernel when executing the operator based on the operating parameters, and determining a second performance of a complete kernel when executing the operator based on the description information of the operator; selecting an execution kernel from the target kernel and the complete kernel based on the first performance and the second performance; and executing the operator on the execution kernel.

[0082] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0083] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An operator execution method, characterized in that: include: Based on the operation parameters of the operator, a target kernel is determined from multiple functionally incomplete kernels; Determining a first performance of the target kernel when executing the operator based on the operating parameters, and determining a second performance of the functionally complete kernel when executing the operator based on the description information of the operator; Selecting an execution kernel from the target kernel and the functionally complete kernel based on the first performance and the second performance; The operator is executed on the execution kernel.

2. The operator execution method according to claim 1, characterized in that: The functionally complete kernel is determined from a plurality of candidate functionally complete kernels based on the description information of the operator in the operator selection phase; the target kernel is determined from the plurality of functionally incomplete kernels based on the operation parameters in the operator execution phase.

3. The operator execution method according to claim 1, characterized in that: The target kernel is determined based on at least one of the following steps: In a case where the operator operation indicated by the running parameter does not have a target operation, determining the target kernel from the multiple functionally incomplete kernels; When the currently reserved video memory indicated by the operating parameter is greater than a threshold, the target kernel is determined from the multiple functionally incomplete kernels.

4. The operator execution method according to claim 3, characterized in that: When the operator operation indicated by the running parameter satisfies at least one of the amplification parameter being 0 and the participating address being empty, it is determined that the target operation does not exist.

5. The operator execution method according to any one of claims 1 to 4, characterized in that: The operating parameter is determined based on at least one of a task requirement corresponding to the operator and a currently reserved video memory.

6. The operator execution method according to any one of claims 1 to 4, characterized in that: The first performance is determined based on a theoretical execution time of the target kernel when executing the operator, and the second performance is determined based on a theoretical execution time of the fully functional kernel when executing the operator.

7. An operator execution device, characterized in that: include: A first determining unit, configured to determine a target kernel from a plurality of functionally incomplete kernels based on an operating parameter of an operator; a second determining unit, configured to determine, based on the operating parameters, a first performance of the target kernel when executing the operator, and to determine, based on the description information of the operator, a second performance of the functionally complete kernel when executing the operator; a kernel selection unit, configured to select an execution kernel from the target kernel and the functionally complete kernel based on the first performance and the second performance; An operator execution unit is used to execute the operator on the execution kernel.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the operator execution method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the operator execution method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the operator execution method according to any one of claims 1 to 6 is implemented.

Images