Operator dispatch run-time comparison method, apparatus, and storage medium

Abstract

The application relates to the field of data processing, in particular to a scheduling running time comparison method and device of an operator and a storage medium. The method comprises the following steps: obtaining at least two candidate schedules corresponding to a target computing expression, the target computing expression being used for describing the computing logic of the operator; obtaining a cost comparison model, the cost comparison model being a model obtained by training a neural network by using multiple sample schedules; calling the cost comparison model to output a cost comparison result according to the at least two candidate schedules, the cost comparison result being used for indicating the size order of the execution time lengths of the at least two candidate schedules on a target hardware platform. According to the embodiment of the application, the relative sizes of the execution time lengths of different schedules can be directly compared without predicting the absolute execution time length of the schedule, so that the automatic optimization function of the compiler / automatic optimizer is realized, and the evaluation speed and accuracy of the scheduling running cost are greatly improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a method, apparatus and storage medium for comparing the scheduling runtime of an operator. Background Technology

[0002] Operators are used to indicate a data processing operation. For example, neural networks typically include convolution operators and pooling operators. A convolution operator indicates a convolution operation, and a pooling operator indicates a pooling operation. To run operators on a real hardware platform and perform the corresponding data processing operations, executable code for the operators needs to be generated. The generation process of executable code for operators consists of two steps: computational representation and scheduling. Computational representation refers to describing the computational logic of the operator using a specific language; that is, describing the task the operator needs to perform, as well as its inputs and outputs. Then, the language describing the computational logic is converted into an intermediate language, yielding the operator's intermediate representation information (also called a template). Scheduling refers to optimizing the intermediate representation information of the operator based on the hardware characteristics of the target hardware platform. Finally, the optimized intermediate representation information can be converted into executable code that the target hardware platform can recognize.

[0003] Automatic operator optimization is a crucial function of optimization tools and compilers. The challenge of automatic operator optimization lies in searching for the optimal scheduling implementation for a specific hardware platform within a massive scheduling space. Evaluating the execution time of different schedules of operators in a neural network on a hardware platform is paramount for successful optimization. To evaluate the execution time of a schedule on a specific hardware platform, related techniques use pre-trained cost models to assess the absolute execution time of the schedule, thus evaluating the cost of scheduling operations. However, this approach suffers from significant discrepancies between the predicted absolute execution time and the actual execution time, requires specialized personnel to build dedicated cost models for specific hardware platforms, often necessitates massive amounts of training data, and results in complex model structures. Furthermore, the large prediction error in this method fails to eliminate the uncertainty in comparing the costs of schedules with similar predictions.

[0004] In the relevant technologies, there is still no reasonable and effective method for evaluating the cost of scheduling operations. Summary of the Invention

[0005] In view of this, a method, apparatus, and storage medium for comparing the scheduling runtime of operators are proposed. Embodiments of this application provide a method, apparatus, and storage medium for comparing the scheduling runtime of operators, which directly compares the relative execution times of different schedules without predicting the absolute execution time of the schedules, thereby realizing the automatic tuning function of the compiler / automatic optimizer and greatly improving the speed and accuracy of scheduling runtime cost assessment.

[0006] In a first aspect, embodiments of this application provide a method for comparing the scheduling runtime of an operator, the method comprising:

[0007] Obtain at least two candidate schedules corresponding to the target computation expression, wherein the target computation expression is used to describe the computation logic of the operator, and the candidate schedules are executable code of the operator on the target hardware platform generated based on the target computation expression;

[0008] Obtain a cost comparison model, which is a model obtained by training a neural network using multiple sample scheduling;

[0009] Based on the at least two candidate schedules, the cost comparison model is invoked to output a cost comparison result, which is used to indicate the order of execution time of the at least two candidate schedules on the target hardware platform.

[0010] In this implementation, by obtaining at least two candidate schedules corresponding to the target computation expression, and based on the cost comparison model of at least two candidate schedules, the relative execution times of at least two candidate schedules on the target hardware platform are directly compared, thereby outputting a cost comparison result used to indicate the order of execution times. This enables the automatic tuning function of the compiler / automatic optimizer, greatly improving the evaluation speed and accuracy of scheduling execution costs.

[0011] In one possible implementation, the step of invoking the cost comparison model output to obtain the cost comparison result based on the at least two candidate schedules includes:

[0012] The at least two candidate schedules are preprocessed to obtain the at least two candidate schedules after preprocessing.

[0013] The preprocessed at least two candidate schedules are input into the cost comparison model, and the cost comparison result is output.

[0014] The cost comparison model is trained based on at least one set of sample data, each set of sample data including: at least two sample schedulings corresponding to the sample computation expression and pre-labeled correct cost comparison results.

[0015] In this implementation, at least two candidate schedules are preprocessed to obtain at least two preprocessed candidate schedules. The at least two preprocessed candidate schedules are then input into the cost comparison model, and the cost comparison result is output. Since the cost comparison model is trained based on at least one set of sample data, the high accuracy of the cost comparison model is guaranteed, which further guarantees the accuracy of the cost comparison result output by the cost comparison model.

[0016] In another possible implementation, the preprocessing of the at least two candidate schedules to obtain the preprocessed at least two candidate schedules includes:

[0017] For each of the at least two candidate schedules, feature extraction is performed on the candidate schedule to obtain a feature matrix;

[0018] The feature matrix corresponding to the candidate schedule is normalized to obtain the preprocessed candidate schedule.

[0019] In this implementation, for each of the at least two candidate schedules, feature extraction is performed on the candidate schedule to obtain a feature matrix; the feature matrix corresponding to the candidate schedule is normalized to obtain a preprocessed candidate schedule. By preprocessing the candidate schedule into a specialized data structure, the accuracy of the cost comparison results obtained by the subsequent model output is further guaranteed.

[0020] In another possible implementation, the feature matrix is ​​used to indicate at least one of loop information, input data shape information, computation encoding, axis type encoding, and data access type encoding. The loop information includes information related to the loop computation logic of the candidate scheduler. The input data shape information is used to describe the input data of the operator. The computation encoding includes the encoding of the computation instructions used in the current loop of the candidate scheduler. The axis type encoding includes the type encoding of the operation on the axis. The data access type encoding includes the type encoding of the data access.

[0021] In this implementation, candidate scheduling is transformed into a feature matrix, which may include at least one of five types of information: indication loop information, input data shape information, computation encoding, axis type encoding, and data access type encoding. The feature matrix with this data structure is then used as input data for the cost comparison model, which further improves the accuracy of the cost comparison results obtained by the subsequent model output.

[0022] In another possible implementation, the acquisition cost comparison model is preceded by:

[0023] Obtain a training sample set, wherein the training sample set includes at least one set of the sample data groups;

[0024] For each group of sample data, at least two sample schedules are preprocessed to obtain the at least two preprocessed sample schedules.

[0025] The preprocessed at least two samples are scheduled and input into the original parameter model to obtain the training result, wherein the original parameter model is a neural network model;

[0026] The training result is compared with the correct cost comparison result to obtain a calculated loss, which is used to indicate the error between the training result and the correct cost comparison result.

[0027] The cost comparison model is trained using the error backpropagation algorithm based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0028] In this implementation, a training sample set is obtained before acquiring the cost comparison model. The training sample set includes at least one set of sample data groups. For each set of sample data groups, at least two sample schedules are preprocessed to obtain at least two preprocessed sample schedules. The preprocessed at least two sample schedules are input into the original parameter model to obtain the training result. The original parameter model is a neural network model. The training result is compared with the correct cost comparison result to obtain the calculated loss, which is used to indicate the error between the training result and the correct cost comparison result. Based on the calculated loss corresponding to each of the at least one set of sample data groups, the cost comparison model is trained using the error backpropagation algorithm. This pre-trains the cost comparison model for evaluating the scheduling operation cost of the operator, ensuring the feasibility of subsequently calling the model to implement the operator's scheduling operation time comparison method.

[0029] In another possible implementation, after obtaining the cost comparison result by invoking the cost comparison model output based on the at least two candidate schedules, the method further includes:

[0030] Add the at least two candidate schedules and the cost comparison results to the training sample set to obtain an updated training sample set;

[0031] The cost comparison model is trained based on the updated training sample set to obtain the updated cost comparison model.

[0032] In this implementation, an updated training sample set is obtained by adding at least two candidate schedules and cost comparison results to the training sample set; the cost comparison model is trained based on the updated training sample set to obtain an updated cost comparison model, thereby updating the cost comparison model in a timely manner and continuously improving the accuracy of the cost comparison model.

[0033] Secondly, embodiments of this application provide a scheduling runtime comparison device for operators, the device comprising:

[0034] The first acquisition unit is used to acquire at least two candidate schedules corresponding to the target computation expression, wherein the target computation expression is used to describe the computation logic of the operator, and the candidate schedule is the executable code of the operator generated based on the target computation expression;

[0035] The second acquisition unit is used to acquire a cost comparison model, wherein the cost comparison model is a model obtained by training a neural network using multiple sample scheduling.

[0036] The calling unit is used to call the cost comparison model output to obtain the cost comparison result based on the at least two candidate schedules. The cost comparison result is used to indicate the order of the execution time of the at least two candidate schedules.

[0037] In one possible implementation, the calling unit is further configured to:

[0038] The at least two candidate schedules are preprocessed to obtain the at least two candidate schedules after preprocessing.

[0039] The preprocessed at least two candidate schedules are input into the cost comparison model, and the cost comparison result is output.

[0040] The cost comparison model is trained based on at least one set of sample data, each set of sample data including: at least two sample schedulings corresponding to the sample computation expression and pre-labeled correct cost comparison results.

[0041] In another possible implementation, the calling unit is also used for:

[0042] For each of the at least two candidate schedules, feature extraction is performed on the candidate schedule to obtain a feature matrix;

[0043] The feature matrix corresponding to the candidate schedule is normalized to obtain the preprocessed candidate schedule.

[0044] In another possible implementation, the feature matrix is ​​used to indicate at least one of loop information, input data shape information, computation encoding, axis type encoding, and data access type encoding. The loop information includes information related to the loop computation logic of the candidate scheduler. The input data shape information is used to describe the input data of the operator. The computation encoding includes the encoding of the computation instructions used in the current loop of the candidate scheduler. The axis type encoding includes the type encoding of the operation on the axis. The data access type encoding includes the type encoding of the data access.

[0045] In another possible implementation, the apparatus further includes a training unit; the training unit is used for:

[0046] Obtain a training sample set, wherein the training sample set includes at least one set of the sample data groups;

[0047] For each group of sample data, at least two sample schedules are preprocessed to obtain the at least two preprocessed sample schedules.

[0048] The preprocessed at least two samples are scheduled and input into the original parameter model to obtain the training result, wherein the original parameter model is a neural network model;

[0049] The training result is compared with the correct cost comparison result to obtain a calculated loss, which is used to indicate the error between the training result and the correct cost comparison result.

[0050] The cost comparison model is trained using the error backpropagation algorithm based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0051] In another possible implementation, the apparatus further includes an updating unit; the updating unit is used for:

[0052] Add the at least two candidate schedules and the cost comparison results to the training sample set to obtain an updated training sample set;

[0053] The cost comparison model is trained based on the updated training sample set to obtain the updated cost comparison model.

[0054] Thirdly, embodiments of this application provide a scheduling runtime comparison device for operators, the device comprising:

[0055] processor;

[0056] Memory used to store processor-executable instructions;

[0057] The processor is configured to implement the above method when executing the instructions.

[0058] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0059] Fifthly, embodiments of this application provide a computer program product, which, when run on a computer, executes the methods described above. Attached Figure Description

[0060] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0061] Figure 1 A schematic diagram of the scheduling space generation process in related technologies is shown.

[0062] Figure 2 The diagram illustrates the principles of the actual measurement method and the cost model method in related technologies.

[0063] Figure 3 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown.

[0064] Figure 4 A flowchart of a scheduling runtime comparison method for operators provided in an exemplary embodiment of this application is shown.

[0065] Figure 5 This illustration shows a schematic diagram of the principle of the operator scheduling runtime comparison method provided in an exemplary embodiment of this application.

[0066] Figure 6 A flowchart illustrating the training process of a cost comparison model provided in an exemplary embodiment of this application is shown.

[0067] Figure 7 This illustration shows a schematic diagram of the training process of a cost comparison model provided in an exemplary embodiment of this application.

[0068] Figure 8 A schematic diagram of the input-output curves of a normalization function provided in an exemplary embodiment of this application is shown.

[0069] Figure 9 A schematic diagram of the network structure of a multilayer perceptron architecture provided in an exemplary embodiment of this application is shown.

[0070] Figure 10 A flowchart of a scheduling runtime comparison method for operators provided in another exemplary embodiment of this application is shown.

[0071] Figure 11 A schematic diagram of the data structure of the feature matrix provided in an exemplary embodiment of this application is shown.

[0072] Figure 12 This illustration shows a schematic diagram of the application process of the cost comparison model provided in another exemplary embodiment of this application.

[0073] Figure 13 A block diagram of a scheduling runtime comparison apparatus for an operator provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0074] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0075] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0076] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0077] With the rapid development of artificial intelligence technology, deep learning has been widely applied in many fields. These applications are rapidly increasing their demand for computing resources, making the optimization of deep learning algorithms increasingly important. Deep learning technology builds deep learning models and iteratively fits large amounts of historical data (model training), enabling the model to establish a mapping relationship between input and output, thereby achieving prediction of results for new input data (model inference). Deep learning models contain a large number of operators, such as convolution operators, fully connected operators, and pooling operators. The overall structure formed by stacking and connecting different operators constitutes a deep learning model, also known as a neural network model. The topological architecture of a neural network is called the neural network architecture; the parameters of the operators contained in the neural network are called model parameters. To enable operators to execute efficiently on a specific hardware platform, the computational expression of the operators needs to be deeply optimized. This specific hardware platform can be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or a Neural Network Processing Unit (NPU).

[0078] The computational representation of an operator can be implemented in many ways, called scheduling. Different schedules can have vastly different performance characteristics on specific hardware platforms. By utilizing a large number of scheduling implementations and employing a compiler / automatic optimizer to automatically search for the optimal schedule for a specific hardware, optimization of deep learning applications can be achieved, thereby reducing computational requirements and increasing system throughput. In engineering implementation, an intermediate representation, called a template, can exist between the computational representation and the schedule. Multiple templates can be formed from the computational representation, and each template can generate multiple schedules.

[0079] Automatic operator optimization is a crucial feature of optimization tools and compilers. The performance of automatically optimized operators directly determines whether deep learning models can be efficiently deployed and applied to meet product requirements. The challenge of automatic operator optimization lies in searching for the optimal scheduling implementation for a specific hardware platform within a massive scheduling space. Evaluating the execution time of different scheduling operations on a hardware platform is paramount for successful optimization. In this paper, "cost" refers to the execution time of the scheduling on the hardware platform. Currently, there are two main methods for evaluating the execution time of scheduling on a hardware platform: the actual measurement method and the cost model method.

[0080] The actual measurement method refers to generating and compiling code for each scheduler, and then running it on hardware. The specific execution time is obtained by measuring the runtime. This method requires a complete compilation process. Its disadvantages include the long evaluation time (seconds or more), which is too time-consuming in real-world scenarios with hundreds of thousands or millions of schedules; and the limitation of search time makes it difficult to explore larger scheduler spaces.

[0081] The cost model method refers to evaluating the execution time of a schedule by establishing a cost model. This method has a significant advantage in evaluating execution time because it does not require compilation, execution, and measurement processes.

[0082] In related technologies, cost-based methods all use the predicted absolute execution time of the schedule to evaluate the scheduling cost. However, this approach suffers from significant discrepancies between the predicted and actual execution times, and requires specialized personnel to build dedicated cost models for specific hardware platforms, often necessitating massive amounts of training data and resulting in complex model structures. Furthermore, the large prediction error in this method fails to eliminate the uncertainty in comparing costs between schedules with similar predictions. These drawbacks limit the application of cost-based methods in practical optimization processes.

[0083] This application provides a method, apparatus, and storage medium for comparing the scheduling runtime of operators. Without predicting the absolute execution time of schedules, it directly compares the relative execution times of different schedules, thereby enabling automatic tuning by the compiler / automatic optimizer and significantly improving the speed and accuracy of scheduling cost evaluation. Compared with methods in related technologies, the operator scheduling runtime comparison method provided in this application has significant advantages in both speed and accuracy, improving the performance of the operator optimizer and significantly reducing evaluation time.

[0084] First, let me introduce some of the terms used in this application.

[0085] 1. Computational Representation (compute): This refers to the entirety of an operator's input data, output data, and computational logic. A computational representation is an instance that describes a specific computational process. In the operator automatic optimization framework, the computational representation can be user-defined, containing all the information needed to complete the computational logic required by the user. Computational representations are typically in the form of pseudocode or structured flowcharts, easy to write but unoptimized.

[0086] 2. Template: A computational representation can be transformed into a template through a series of equivalent transformations. The template serves as intermediate representation information between the computational representation and the scheduling process during the optimization of the computational representation structure. Typically, the template determines the order of computation execution and the data access patterns within the computational representation logic.

[0087] Templates alter the computation execution order and data access patterns of a computational expression, but do not restrict how the input data of the computational expression is specifically divided. For example, after an axis partitioning transformation, a single loop can be divided into several sub-loops, and different numbers of sub-loops constitute different templates. In each template, the upper and lower bounds of the sub-loops only need to satisfy equivalence with the computational expression, but the numerical values ​​of the upper and lower bounds of each sub-loop are uncertain.

[0088] 3. Scheduling: Based on the hardware characteristics of the target hardware platform, the intermediate representation information of the operator is scheduled and optimized. Scheduling determines the specific expression of all variable parameters in the template, which can be transformed into a description of the computational expression implemented in software. For the same input data, the output data of the scheduling is completely consistent with the output data of the computational expression, but the execution of the computation process can be different.

[0089] 4. Feature embedding: The intermediate output of the input data after passing through the neural network modules. Feature embedding is the mapping of the input data in another space by the neural network modules, including the extraction, enhancement, and encoding of the input data.

[0090] 5. Multilayer Perceptron: A basic unit of a neural network consisting of stacked layers of fully connected layers, activation layers, etc. Multilayer perceptrons can form part of an overall neural network architecture, or they can appear as a module within an overall architecture.

[0091] In a illustrative example, taking a single-loop transformation as an example (pseudocode), the process of generating the scheduling space is as follows: Figure 1As shown, a computer device (such as an automatic optimizer) acquires the computational expression input by the user, transforms the computational expression, and generates a template space. Templates in the template space can be transformed into scheduling implementations logically equivalent to the computational expression. The set of valid schedules forms the scheduling space. The computer device searches within the scheduling space and outputs the optimal scheduling implementation. In this example, the computational expression is a user-defined loop computation, and the computational logic within the loop body is represented by a statement (stmt). The upper and lower bounds of the loop range from 0 to 546756. After axis partitioning transformation, the single loop of the computational expression can be equivalently transformed into templates for double-nested loops, triple-nested loops, and N-level nested loops. In the template, the upper and lower bounds of each nested loop are not determined; axis partitioning allows for different planning of data access patterns. The loop boundary values ​​equivalent to the loop upper and lower bounds of the computational expression are filled into the template, and the loop statement is reasonably transformed or constrained (e.g., ...). Figure 1 In the context of `stmt_tpln_immd_constrain` (which can be an intermediate constraint array of the nth template), the resulting code, which is equivalent to the computational expression, is the scheduler.

[0092] In real-world scenarios, the complex computational logic resulting in computational expressions can often lead to millions of scheduling implementations, with different schedules having execution times that can differ by hundreds or thousands of times on the target hardware platform. Automatic optimization systems optimize operators by searching a massive search space of schedules through a series of operations to find the optimal hardware execution schedule.

[0093] like Figure 2 As shown, the actual measurement method generates valid code based on the definition of scheduling, compiles it with a compiler, executes it on hardware, and measures it to obtain performance evaluation results. The result is usually the execution time of the schedule, or it can be the number of hardware clock cycles required for execution. By performing actual measurements on a limited number of schedules, the schedule with the shortest execution time (smallest hardware clock cycle count) is finally selected. The execution time obtained in this way is the most accurate and realistic. The disadvantage is that the code generation and compilation process usually takes several seconds to several minutes to complete, and the time spent on the execution and measurement process depends on the computational load and complexity of the operators, making it very slow in large-scale optimization processes.

[0094] Machine learning-based search optimization methods can significantly accelerate the above process, reducing code generation, compilation, and execution from several seconds to millisecond-level neural network inference. However, due to the limitations of model prediction accuracy, the optimization effect may decline. Current cost models, as mentioned earlier, predict the absolute execution time or number of cycles of a schedule after feature extraction. The accuracy of the cost model in this method is low; related technologies have an average error of 16% in predicting schedule execution time, while the actual execution time differences between many schedules are less than 16%. Besides the error, acquiring the cost model is costly, requiring 1.8 million training data points, resulting in high network architecture complexity and long training convergence time. Therefore, this method cannot effectively achieve the goal of fast and accurate operator search optimization.

[0095] This application provides a novel cost model: a cost comparison model. The cost comparison model avoids directly predicting the execution time of schedules, transforming the regression problem into a classification problem that is easier for neural networks to learn. The cost comparison model takes at least two candidate schedules as input and outputs a cost comparison result, which indicates the ranking of the execution times of the at least two candidate schedules on the target hardware platform. The method provided in this application has the advantages of high accuracy, fast inference speed, and relatively low training cost compared to existing methods. In the operator optimization process, the cost comparison model provided in this application can quickly compare the execution times of different schedules, thereby achieving large-scale search optimization of operators.

[0096] It should be noted that the operator scheduling runtime comparison method provided in this application embodiment can be applied to the optimization process of an operator automatic optimization system. The core content of this application embodiment is the cost comparison model, including the model architecture, model training process, and model application process of the cost comparison model. The operator scheduling runtime comparison method provided in this application embodiment can be applied to specific computer devices (such as CPU, GPU, or NPU) to perform large-scale comparison and search of multiple candidate scheduling implementations of the target computational expression, thereby obtaining the optimal schedule and achieving the goal of optimizing the target computational expression on a specific computer device.

[0097] The execution subject of the operator scheduling runtime comparison method provided in this application embodiment is a computer device, which can be a general-purpose computer device or a special-purpose computer device. Please refer to... Figure 3 The diagram illustrates a structural schematic of a computer device provided in an exemplary embodiment of this application.

[0098] The computer device can be a terminal or a server. Terminals include tablets, laptops, and desktop computers, etc. A server can be a single server, a server cluster consisting of several servers, or a cloud computing service center.

[0099] like Figure 3 As shown, the computer device includes a processor 10, a memory 20, and a communication interface 30. Those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. Wherein:

[0100] The processor 10 is the control center of the computer device. It connects various parts of the computer device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 20, and by calling data stored in the memory 20, it performs various functions of the computer device and processes data, thereby controlling the computer device as a whole. The processor 10 can be implemented by a CPU or a GPU.

[0101] The memory 20 can be used to store software programs and modules. The processor 10 executes various functional applications and data processing by running the software programs and modules stored in the memory 20. The memory 20 may mainly include a program storage area and a data storage area. The program storage area may store the operating system 21, the first acquisition unit 22, the second acquisition unit 23, the detection unit 24, and at least one application program 25 required for a function (such as neural network training, etc.). The data storage area may store data created according to the use of the computer device. The memory 20 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Accordingly, memory 20 may also include a memory controller to provide processor 10 with access to memory 20.

[0102] The processor 20 performs the following functions by running the first acquisition unit 22: acquiring at least two candidate schedules corresponding to the target computation expression, wherein the target computation expression is used to describe the computation logic of the operator, and the candidate schedules are executable code of the operator generated based on the target computation expression; the processor 20 performs the following functions by running the second acquisition unit 23: acquiring a cost comparison model, wherein the cost comparison model is a model obtained by training a neural network using multiple sample schedules; the processor 20 performs the following functions by running the calling unit 24: according to the at least two candidate schedules, calling the cost comparison model to output a cost comparison result, wherein the cost comparison result is used to indicate the order of execution time of the at least two candidate schedules.

[0103] Optionally, the computer device acquires the computational expression code input by the user, i.e., the target computational expression, and analyzes the target computational expression through an operator optimization system. Based on optimization rules or polyhedral models, a template space is generated. A large number of valid candidate schedules are generated by instantiating the templates, and these multiple candidate schedules constitute the scheduling space. An instance in the scheduling space represents a valid schedule. The cost comparison model provided in this embodiment serves as an evaluation module, comparing at least two input candidate schedules and outputting a cost comparison result, thereby achieving the goal of finding the optimal schedule in the scheduling space.

[0104] The following illustrative examples illustrate the method for comparing the scheduling runtime of operators.

[0105] Please refer to Figure 4 This document illustrates a flowchart of a scheduling runtime comparison method for an operator provided in an exemplary embodiment of this application. This embodiment applies the scheduling runtime comparison method of this operator to... Figure 3 The computer equipment shown is used as an example. The scheduling runtime comparison method for this operator includes:

[0106] Step 401: Obtain at least two candidate schedules corresponding to the target computation expression. The target computation expression is used to describe the computation logic of the operator, and the candidate schedules are the executable code of the operator generated based on the target computation expression on the target hardware platform.

[0107] Optionally, the computer device obtains at least two candidate schedules from the scheduling space corresponding to the target computation expression. Illustratively, the computer device obtains the input target computation expression, analyzes the target computation expression, generates a template space according to a preset method, and generates multiple candidate schedules by instantiating the templates. The generated multiple candidate schedules constitute the scheduling space. The computer device obtains at least two candidate schedules from this scheduling space.

[0108] Optionally, the preset method may be dynamic programming, optimization rule method, or polyhedral model method. The preset method may also differ for different computing systems. This application does not limit the algorithm for generating the scheduling space. The scheduling space must include at least two candidate schedules for comparison before this application's embodiments can be applied.

[0109] Here, the target computation expression is a specific computation expression, such as the target computation expression being the input computation expression.

[0110] The candidate scheduler is the executable code on the target hardware platform of the operator generated based on the target computational expression. For example, the target hardware platform is a CPU, GPU, or NPU. This application does not limit this.

[0111] Optionally, upon receiving a preset acquisition instruction, the computer device acquires at least two candidate schedules corresponding to the target computational representation. Alternatively, the computer device acquires at least two candidate schedules corresponding to the target computational representation at preset time intervals. Or, the computer device acquires at least two candidate schedules corresponding to the target computational representation in real time.

[0112] The preset time interval can be either a default setting or a custom setting; this embodiment does not limit this to any particular setting.

[0113] Step 402: Obtain the cost comparison model, which is a model obtained by training the neural network using multiple sample scheduling.

[0114] The computer device acquires a pre-trained cost comparison model. In one possible implementation, when the computer device is a terminal, the terminal acquires its own pre-trained cost comparison model or acquires it from a server. In another possible implementation, when the computer device is a server, the server acquires its own pre-trained cost comparison model.

[0115] The cost comparison model is a model obtained by training the neural network using at least two sample schedulings and correct cost comparison results. In other words, the cost comparison model is determined based on at least two sample schedulings and correct cost comparison results. The correct cost comparison results are pre-labeled correct cost comparison results corresponding to at least two sample schedulings.

[0116] The cost comparison model's neural network can employ an end-to-end stacked multilayer perceptron architecture. Other reasonable variations can also achieve the cost comparison model's fitting function; different architectures will affect the model's final accuracy. Any network architecture resulting from variations, derivations, or layer replacements of this architecture should be considered equivalent to the neural network described in the embodiments of this application.

[0117] For example, the neural network is a deep neural network (DNN). For example, the neural network is a convolutional neural network (CNN). For example, the neural network is a recurrent neural network (RNN). This application does not limit the specific examples.

[0118] The cost comparison model is a neural network model that identifies the relative execution time of at least two candidate schedules on the target hardware platform.

[0119] The cost comparison model is used to transform at least two input candidate schedules into a cost comparison result. This cost comparison result is used to indicate the ranking of the execution time of the at least two candidate schedules on the target hardware platform.

[0120] The cost comparison model is used to represent the correlation between at least two candidate schedules and the cost comparison results.

[0121] The cost comparison model is a pre-defined mathematical model that includes model coefficients between at least two candidate schedules and the cost comparison results. These model coefficients can be fixed values, dynamically modified over time, or dynamically modified depending on the usage scenario.

[0122] Step 403: Based on at least two candidate schedules, call the cost comparison model to obtain the cost comparison result. The cost comparison result is used to indicate the order of execution time of at least two candidate schedules on the target hardware platform.

[0123] Optionally, the computer device preprocesses at least two candidate schedules to obtain at least two preprocessed candidate schedules; the preprocessed at least two candidate schedules are then input into a cost comparison model, and the cost comparison results are output.

[0124] The cost comparison result is used to indicate the order of execution time of at least two candidate schedules on the target hardware platform. That is, the cost comparison result does not indicate the absolute execution time of at least two candidate schedules on the target hardware platform, but rather the relative execution time of at least two candidate schedules on the target hardware platform.

[0125] Optionally, the cost comparison result is encoded information of the comparison of the execution times of at least two predicted candidate schedules. The computer device decodes the encoded information output by the cost comparison model to obtain the ranking of the execution times of at least two candidate schedules, i.e., the comparison result.

[0126] For illustrative purposes, the cost comparison results include encoded information, and the values ​​of the encoded information correspond one-to-one with the execution time comparison results of at least two candidate schedules. For example, taking at least two candidate schedules as the first and second candidate schedules, the encoded information is a first value indicating that the execution time of the first candidate schedule is less than that of the second candidate schedule; a second value indicating that the execution time of the first candidate schedule is equal to that of the second candidate schedule; and a third value indicating that the execution time of the first candidate schedule is greater than that of the second candidate schedule. The first, second, and third values ​​are all different.

[0127] Optionally, the computer device, based on the cost comparison results of at least two candidate schedules, selects the candidate schedule with the shortest execution time among the at least two candidate schedules as the target schedule, retains the target schedule, and discards the other candidate schedules.

[0128] Optionally, if the cost comparison result indicates that at least two candidate schedules have the same execution time, the computer device will select any one of the at least two candidate schedules as the target schedule, retain the target schedule, and discard the other candidate schedules. This application embodiment does not limit the method of retaining and discarding schedules.

[0129] In an illustrative example, such as Figure 5 As shown, the computer device acquires the input target computational representation, analyzes the target computational representation, generates a template space according to a preset method, and generates multiple candidate schedules by instantiating the templates. These multiple candidate schedules constitute the scheduling space. Two candidate schedules, such as schedule A and schedule B, are obtained from this scheduling space. Schedule A and schedule B are preprocessed to obtain preprocessed schedule A and schedule B. The preprocessed schedule A and schedule B are input into a cost comparison model to obtain encoded information. The encoded information is decoded to obtain the cost comparison result of schedule A and schedule B. For example, when the encoded information is 001, it indicates that the execution time of schedule A is less than the execution time of schedule B; schedule A is retained, and schedule B is discarded. When the encoded information is 002, it indicates that the execution time of schedule A is equal to the execution time of schedule B; either schedule A or schedule B is retained. When the encoded information is 100, it indicates that the execution time of schedule A is greater than the execution time of schedule B; schedule B is retained, and schedule A is discarded.

[0130] In summary, the embodiments of this application obtain at least two candidate schedules corresponding to the target computation expression, and directly compare the relative execution times of at least two candidate schedules on the target hardware platform according to the cost comparison model of at least two candidate schedules, thereby outputting a cost comparison result used to indicate the order of execution times. This can realize the automatic tuning function of the compiler / automatic optimizer, and greatly improve the evaluation speed and accuracy of scheduling execution costs.

[0131] It should be noted that before the computer equipment obtains the cost comparison model, it needs to be trained on a training sample set to obtain the cost comparison model. The training process of the cost comparison model is described below.

[0132] In one possible implementation, such as Figure 6 As shown, the training process for the cost comparison model includes the following steps:

[0133] Step 601: Obtain the training sample set, which includes at least one set of sample data.

[0134] The cost comparison model is trained on at least one set of sample data, each set of sample data including: at least two sample schedulings corresponding to the sample computation expression and pre-labeled correct cost comparison results.

[0135] Step 602: For each group of sample data, preprocess at least two sample schedules to obtain at least two preprocessed sample schedules.

[0136] For each set of sample data, the computer device extracts features from each of the at least two sample schedules to obtain a feature matrix, and then normalizes the feature matrix corresponding to the sample schedule to obtain the preprocessed sample schedule.

[0137] Illustratively, feature extraction is the process of extracting features from sample scheduling and converting those features into structured data.

[0138] It should be noted that the relevant descriptions of the feature matrix can be found in the details of the following embodiments, and will not be introduced here.

[0139] Step 603: Input at least two preprocessed samples into the original parameter model to obtain the training result. The original parameter model is a neural network model.

[0140] Optionally, the original parameter model is built based on a neural network model, for example, the original parameter model is built based on a DNN model.

[0141] To illustrate, for each set of sample data, the computer device creates an input-output pair corresponding to that set of sample data. The input parameters of the input-output pair are at least two sample schedules in that set of sample data, and the target parameter is the correct cost comparison result in that set of sample data. The computer device inputs the input parameters into the original parameter model to obtain the training result.

[0142] Optionally, the input-output pair can be represented by feature vectors.

[0143] Step 604: Compare the training results with the correct cost comparison results to obtain the calculated loss, which is used to indicate the error between the training results and the correct cost comparison results.

[0144] Optionally, the training result is the encoded information output by the original parameter model, and the correct cost comparison result is the pre-labeled encoded information. For example, the encoded information is information encoded using one-hot codes.

[0145] Optionally, the loss can be calculated using cross-entropy.

[0146] Step 605: Based on the calculated loss corresponding to at least one set of sample data groups, the cost comparison model is trained using the error backpropagation algorithm.

[0147] Optionally, the computer device determines the gradient direction of the cost comparison model based on the calculated loss using the backpropagation algorithm, and updates the model parameters in the cost comparison model layer by layer from the output layer of the cost comparison model forward.

[0148] In an illustrative example, consider at least two candidate schedules, schedule A and schedule B, such as... Figure 7 As shown in Table 1, the computer device extracts two schedules, A and B, from the scheduling space as input data for training the cost comparison model. By comparing the relative execution times of the two schedules on the target hardware platform, one-hot encoding is used to generate the encoded information (A, B) input (i.e., the correct cost comparison result), which serves as the target parameter for the backpropagation algorithm. The encoded information is shown in Table 1. A first value indicates that the execution time of schedule A is less than that of schedule B; a second value indicates that the execution time of schedule A is equal to that of schedule B; and a third value indicates that the execution time of schedule A is greater than that of schedule B.

[0149] Table 1

[0150]

[0151] The computer equipment extracts features from schedules A and B to obtain their respective feature matrices. For example, the feature matrices of schedules A and B are two 250x57 dimensional matrices. A portion of the column data in the feature matrices is normalized to limit their dynamic range. The formula for the normalization function is as follows:

[0152]

[0153] Where v represents the input data and v* represents the output data. The input-output curves of the normalization function are shown illustratively as follows: Figure 8 As shown in the figure. The horizontal axis represents the input data described above, and the vertical axis represents the output data described above.

[0154] The computer device inputs a normalized schedule A into a feature embedding module A (DNN_A) composed of a multilayer perceptron, outputting a 1x512-dimensional schedule embedding A. It then inputs a normalized schedule B into a feature embedding module B (DNN_B) composed of a multilayer perceptron, outputting a 1x512-dimensional schedule embedding B. These two schedule embeddings are subtracted bitwise, i.e., schedule embedding A minus schedule embedding B, to obtain the schedule difference embedding. This difference embedding is input into the deep network discriminator module (DNN_CLS), outputting the training result, which is the encoded information of three numbers. Based on the output data of the deep network discriminator module and the comparison results of the true labels (correct costs) of schedules A and B, the mean squared error loss function (or least squares error function) is used as the loss function to calculate the model's computational loss for the current input. The calculated loss is backpropagated using gradient descent to update the model parameters of the neural network modules, including DNN_A, DNN_B, and DNN_CLS. Repeat the above steps to train the model on the training sample set for multiple periods (e.g., 30 periods) until the model converges. The network structures of DNN_A, DNN_B, and DNN_CLS can be end-to-end stacked multilayer perceptron architectures, as shown in the example. Figure 9 As shown in the diagram. The numbers represent the number of neurons in each layer, and the ReLU function is used as the activation function between each fully connected layer.

[0155] Based on the cost comparison model trained above, please refer to... Figure 10 This document illustrates a flowchart of a scheduling runtime comparison method for an operator provided in another exemplary embodiment of this application. This embodiment applies the scheduling runtime comparison method of this operator to... Figure 3 The computer equipment shown is used as an example. The scheduling runtime comparison method for this operator includes:

[0156] Step 1001: Obtain at least two candidate schedules from the scheduling space corresponding to the target computation expression.

[0157] The target computation expression describes the computational logic of the operator, and the candidate scheduler is the executable code of the operator generated based on the target computation expression on the target hardware platform.

[0158] Optionally, the computer device acquires the input target computational representation, analyzes the target computational representation, generates a template according to a preset method, and determines a scheduling space, which includes at least two candidate schedules generated by instantiating the template. The computer device then acquires at least two candidate schedules from the scheduling space corresponding to the target computational representation.

[0159] Indicatively, the scheduling space includes n candidate schedules. One possible implementation involves comparing each schedule pairwise, retaining the optimal one, and performing n-1 comparisons to obtain the optimal target schedule. Another possible implementation uses a binary search method, for example, when n is 8. The 8 schedules are divided into 4 groups pairwise. Within each group, a cost comparison model selects the 4 fastest candidate schedules for secondary grouping. This secondary grouping is further divided into 2 groups, requiring 2 comparisons. After these comparisons, the 2 best candidate schedules are retained for a final comparison, thus obtaining the optimal target schedule from the 8 candidate schedules. This application does not limit the grouping and comparison method for the schedules in its embodiments.

[0160] Step 1002: For each of the at least two candidate schedules, perform feature extraction on the candidate schedule to obtain a feature matrix.

[0161] Optionally, for each of at least two candidate schedules, the computer device extracts multiple types of information from m cycles of that candidate schedule and combines them into a vector, which is the feature matrix corresponding to the candidate schedule, where m is a positive integer. For example, the combined vector has a size of 1x57. It supports information from up to 250 cycles, ultimately assembling a two-dimensional feature matrix of size 250x57. The number of cycles supported can vary according to actual needs, and this embodiment does not limit this.

[0162] Optionally, the feature matrix is ​​used to indicate at least one of cycle information, input data shape information, computational encoding, axis type encoding, and data access type encoding.

[0163] The loop information includes information related to the loop computation logic of the candidate schedule. Optionally, the loop information is loop information at the scheduling level, such as a loop information size of 1x6. The loop information includes at least one of the following: loop depth, nesting level, block number, identifier indicating whether it is the last loop, quotient of loop depth, and remainder of loop depth. The loop depth and quotient of loop depth need to be normalized.

[0164] The input data shape information describes the input data of the operator. For example, the size of the input data shape information is 1x10. This operator can be a single-input operator, a double-input operator, or a multi-input operator. The input data shape information includes: the shape information corresponding to each of the k input data, where k is a positive integer, and the shape information includes at least one of the following: batch size, number of channels, height, width, and minimum number of channels.

[0165] The computation code includes the encoding of the computation instructions used in the current loop of the candidate scheduler. For example, the size of the computation code is 1x6. The computation code includes at least one of the following: memory access type, program instruction, data type, storage unit, and an identifier indicating whether double buffering is used.

[0166] Axis type encoding includes the type encoding of the operation performed on the axis. For example, the size of the axis type encoding is 1x15. The axis type encoding is used to indicate at least one operation among extended and normalized axes.

[0167] Data access type encoding includes the type encoding for accessing data. For example, the size of a data access type encoding is 1x19. Data access type encoding is used to indicate at least one of the following access methods: writing data, reading data, allocation, and compilation instructions.

[0168] In an illustrative example, feature extraction is performed on candidate scheduling to obtain a feature matrix. The data structure of the feature matrix is ​​as follows: Figure 11 As shown, multiple types of information are extracted from each loop of the candidate scheduling and combined into a vector. The combined vector is 1x57 in size and supports information from up to 250 loops. Finally, it is assembled into a 250x57 two-dimensional feature matrix, which is used to indicate loop information, input data shape information, computation encoding, axis type encoding, and data access type encoding. The size of the loop information is 1x6, the size of the input data shape information is 1x10, 0, the size of the computation encoding is 1x6, the size of the axis type encoding is 1x15, and the size of the data access type encoding is 1x20.

[0169] It should be noted that, in addition to the feature extraction, mapping methods, and data structures provided in the embodiments of this application, other scheduling representations can also be used as inputs to the cost comparison model. The embodiments of this application do not limit the input data structure.

[0170] Step 1003: For each of the at least two candidate schedules, normalize the feature matrix corresponding to the candidate schedule to obtain the preprocessed candidate schedule.

[0171] Step 1004: Input the preprocessed at least two candidate schedules into the trained cost comparison model and output the cost comparison result. The cost comparison result is used to indicate the order of execution time of the at least two candidate schedules on the target hardware platform.

[0172] Optionally, the computer device acquires a trained cost comparison model, which is a model trained on a neural network using multiple sample scheduling methods. The computer device inputs at least two preprocessed candidate schedules into the trained cost comparison model and outputs a cost comparison result. This result indicates the ranking of the execution time of the at least two candidate schedules on the target hardware platform.

[0173] The process of comparing the cost of computer device invocation can be found in the relevant details in the above embodiments, and will not be repeated here.

[0174] Optionally, the computer device adds at least two candidate scheduling and cost comparison results to the training sample set to obtain an updated training sample set; the cost comparison model is trained based on the updated training sample set to obtain an updated cost comparison model.

[0175] In an illustrative example, consider at least two candidate schedules, schedule A and schedule B, such as... Figure 12 As shown. The computer device extracts two schedules, A and B, from the scheduling space, and performs feature extraction on schedules A and B to obtain their respective feature matrices. For example, the feature matrices of schedules A and B are two 250x57 dimensional matrices. Some columns of the feature matrices are normalized to limit their dynamic range. The normalization process can be compared to the description of normalization in the model training process above, and will not be repeated here. The computer device inputs the normalized schedule A into the feature embedding module A (DNN_A) composed of a multilayer perceptron, outputting a 1x512 dimensional schedule embedding A, and inputs the normalized schedule B into the feature embedding module B (DNN_B) composed of a multilayer perceptron, outputting a 1x512 dimensional schedule embedding B. These two schedule embeddings are subtracted bitwise, i.e., schedule embedding A is subtracted from schedule embedding B to obtain the schedule difference embedding. The schedule difference embedding is input into the deep network discriminant module (DNN_CLS), and the cost comparison result, i.e., the encoded result of three numbers, is output. The network structures of DNN_A, DNN_B, and DNN_CLS can be compared to the relevant descriptions in the model training process described above, and will not be repeated here. The computer device converts the encoded information of the three output numbers into a one-hot code label format.

[0176] In summary, this application embodiment further obtains feature matrix expressions for at least two candidate schedules by extracting features from at least two candidate schedules and mapping each schedule to its unique corresponding matrix expression; normalizes the two feature matrix expressions; uses a cost comparison model based on a deep neural network as input to take the preprocessed feature matrix expressions as input and outputs encoded information of the comparison result of the execution time of the predicted at least two candidate schedules; decodes the encoded information output by the cost comparison model to obtain the comparison result of the execution time of the at least two candidate schedules. That is, by comparing the execution time of different scheduling implementations with the same computational expression on a specific hardware platform through a deep learning network model, it replaces the process of scheduling implementation running and measuring on hardware after compilation, thus solving the problem of slow speed of automatic optimization systems such as automatic optimizers / compilers in large-scale searches.

[0177] In an illustrative example, the cost comparison model aims to predict the speed of operator execution. The training sample set includes 20,792 schedules from 32 operators, with each operator containing a different number of schedules. Schedules belonging to the same operator are paired to form a training instance set. After pairing, the execution times of the two operators are compared, and the target of the paired training instance is generated according to the methods described above. For example, if schedule A and schedule B are extracted, and the actual execution time of schedule A is 15 seconds while that of schedule B is 17 seconds, then (A, B) is a training instance. Since 15 seconds is less than 17 seconds, the target code for training instance (A, B) is 001. Pairing schedules belonging to the same operator in the training sample set can include a combination of a schedule and itself; the target code for the resulting training instance is 010. Schedules belonging to the same operator are paired and combined. The combination is order-sensitive; for example, the (A, B) combination differs from the (B, A) combination. If the execution times of A and B are different, the target codes of the (A, B) and (B, A) combinations will also be different. If an operator contains N (N>2) schedules, then pairwise combinations can form N squared training instances. This combinational training can construct a relatively large training dataset even with a relatively limited amount of training data. In this example, there are 20,792 schedules, forming 49 million training instances and their target codes for training the model. The model structure is as described above and will not be repeated here. The neural network model uses batch training, with 5,000 training instances input per iteration. The learning rate is set to 10e-8, and momentum stochastic gradient descent is used to train the complete training instance set for multiple periods (e.g., 30 periods). The test set includes 46,022 test instances, each consisting of two schedules belonging to the same operator. Any schedule used to generate test instances is not included in the set of schedules that generate training instances. The test target encoding is generated as a test instance using the methods described above. If the network's prediction, after passing through the maximum value parameter (argmax) function, perfectly matches the test target encoding, it is considered a correctly predicted test instance. Accuracy is defined as: the number of correctly predicted test instances / the total number of test instances tested. Through testing on 46,022 test instances, this method correctly predicted 41,242 test instances, achieving an accuracy of 89.61%. The model's accuracy can be further improved by increasing the number of training schedules and optimizing the network structure.

[0178] In summary, this application provides a method for comparing the scheduling runtime of operators. It employs the idea of ​​cost comparison to determine the comparison result of the relative execution times of at least two schedules. The cost comparison model is applied to the tuning process of an automatic operator optimization system. It also relates to a modeling method for the cost comparison model applicable to automatic operator optimization systems, including model architecture design, model training, and model inference application processes. During model training and inference application, the scheduler can be transformed into a specialized data structure through feature extraction, and the data can be normalized and the output format expressed. This method has the advantages of high accuracy, fast inference speed, and relatively low training cost compared to existing methods. In other words, it ensures high accuracy of the cost comparison model; it improves the inference speed of the cost comparison model, requiring only 3 milliseconds to compare a set of instances; and it requires relatively little data and computing power for training the cost comparison model, completing 30 training sessions on over 49 million training instances in 70 hours on a single GPU. With the cost comparison model, the code optimizer / compiler only needs to consider how to improve the accuracy of the cost comparison model. Compared with the cost model that predicts the absolute execution time of the schedule in related technologies, in addition to the accuracy of the model prediction, the cost model in related technologies also needs to consider how to deal with the boundary problems caused by the error. For example, if the difference between the predicted execution time of two schedules is less than the error of the model prediction, the absolute value model cannot give a prediction with high confidence.

[0179] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0180] Please refer to Figure 13 This diagram illustrates a block diagram of a scheduling runtime comparison apparatus for an operator provided in an exemplary embodiment of this application. This apparatus can be implemented via software, hardware, or a combination of both. Figure 3 The provided computer equipment may be all or part thereof. The device may include: a first acquisition unit 1310, a second acquisition unit 1320, and a recall unit 1330.

[0181] The first acquisition unit 1310 is used to acquire at least two candidate schedules corresponding to the target computation expression, wherein the target computation expression is used to describe the computation logic of the operator, and the candidate schedules are executable code of the operator generated based on the target computation expression.

[0182] The second acquisition unit 1320 is used to acquire the cost comparison model, which is a model obtained by training the neural network using multiple sample scheduling.

[0183] Calling unit 1330 is used to call the cost comparison model output to obtain the cost comparison result based on at least two candidate schedules. The cost comparison result is used to indicate the order of execution time of at least two candidate schedules.

[0184] In one possible implementation, calling unit 1330 is also used for:

[0185] Preprocess at least two candidate schedules to obtain at least two preprocessed candidate schedules;

[0186] Input at least two preprocessed candidate schedules into the cost comparison model and output the cost comparison results;

[0187] The cost comparison model is trained based on at least one set of sample data. Each set of sample data includes at least two sample schedulings corresponding to the sample computation expression and the pre-labeled correct cost comparison results.

[0188] In another possible implementation, calling unit 1330 is also used for:

[0189] For each of at least two candidate schedules, feature extraction is performed on the candidate schedules to obtain a feature matrix;

[0190] The feature matrix corresponding to the candidate schedule is normalized to obtain the preprocessed candidate schedule.

[0191] In another possible implementation, the feature matrix is ​​used to indicate at least one of loop information, input data shape information, computation encoding, axis type encoding, and data access type encoding. The loop information includes information related to the loop computation logic of the candidate scheduler. The input data shape information is used to describe the input data of the operator. The computation encoding includes the encoding of the computation instructions used in the current loop of the candidate scheduler. The axis type encoding includes the type encoding of the operation on the axis. The data access type encoding includes the type encoding of the data access.

[0192] In another possible implementation, the device further includes a training unit; the training unit is used for:

[0193] Obtain a training sample set, which includes at least one set of sample data.

[0194] For each set of sample data, at least two sample schedules are preprocessed to obtain at least two preprocessed sample schedules.

[0195] At least two preprocessed samples are scheduled and input into the original parameter model to obtain the training result. The original parameter model is a neural network model.

[0196] The training results are compared with the correct cost comparison results to obtain the calculated loss, which is used to indicate the error between the training results and the correct cost comparison results.

[0197] Based on the calculated loss corresponding to at least one set of sample data, a cost comparison model is trained using the error backpropagation algorithm.

[0198] In another possible implementation, the device further includes an updating unit; the updating unit is used for:

[0199] Add at least two candidate scheduling and cost comparison results to the training sample set to obtain the updated training sample set;

[0200] The cost comparison model is trained based on the updated training sample set to obtain the updated cost comparison model.

[0201] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0202] This application provides an operator scheduling runtime comparison device, which includes: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the method executed by the computer device in the above embodiments.

[0203] This application provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in a processor, the processor executes the method performed by the computer device in the above embodiments.

[0204] This application provides a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the method executed by the computer device in the above embodiments.

[0205] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital video disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing.

[0206] The computer-readable program instructions or code described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0207] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of this application.

[0208] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0209] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0210] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0211] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0212] It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented using hardware (such as circuits or ASICs (Application Specific Integrated Circuits)) that performs the corresponding function or action, or using a combination of hardware and software, such as firmware.

[0213] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0214] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for comparing the scheduling runtime of an operator, characterized in that, The method includes: Obtain at least two candidate schedules corresponding to the target computation expression, wherein the target computation expression is used to describe the computation logic of the operator, and the candidate schedules are executable code of the operator on the target hardware platform generated based on the target computation expression; Obtain a cost comparison model, which is a model obtained by training a neural network using multiple sample scheduling; Based on the at least two candidate schedules, the cost comparison model is invoked to output a cost comparison result, which is used to indicate the order of execution time of the at least two candidate schedules on the target hardware platform.

2. The method according to claim 1, characterized in that, The step of invoking the cost comparison model to obtain cost comparison results based on the at least two candidate schedules includes: The at least two candidate schedules are preprocessed to obtain the at least two candidate schedules after preprocessing. The preprocessed at least two candidate schedules are input into the cost comparison model, and the cost comparison result is output. The cost comparison model is trained based on at least one set of sample data, each set of sample data including: at least two sample schedulings corresponding to the sample computation expression and pre-labeled correct cost comparison results.

3. The method according to claim 2, characterized in that, The step of preprocessing the at least two candidate schedules to obtain the preprocessed at least two candidate schedules includes: For each of the at least two candidate schedules, feature extraction is performed on the candidate schedule to obtain a feature matrix; The feature matrix corresponding to the candidate schedule is normalized to obtain the preprocessed candidate schedule.

4. The method according to claim 3, characterized in that, The feature matrix is ​​used to indicate at least one of loop information, input data shape information, computation encoding, axis type encoding, and data access type encoding. The loop information includes information related to the loop computation logic of the candidate schedule. The input data shape information is used to describe the input data of the operator. The computation encoding includes the encoding of the computation instructions used in the current loop of the candidate schedule. The axis type encoding includes the type encoding of the operation on the axis. The data access type encoding includes the type encoding of the data access.

5. The method according to any one of claims 2 to 4, characterized in that, The acquisition cost comparison model also includes: Obtain a training sample set, wherein the training sample set includes at least one set of the sample data groups; For each group of sample data, at least two sample schedules are preprocessed to obtain the at least two preprocessed sample schedules. The preprocessed at least two samples are scheduled and input into the original parameter model to obtain the training result, wherein the original parameter model is a neural network model; The training result is compared with the correct cost comparison result to obtain a calculated loss, which is used to indicate the error between the training result and the correct cost comparison result. The cost comparison model is trained using the error backpropagation algorithm based on the calculated loss corresponding to each of the at least one set of sample data groups.

6. The method according to claim 5, characterized in that, After obtaining the cost comparison result by calling the cost comparison model based on the at least two candidate schedules, the process further includes: Add the at least two candidate schedules and the cost comparison results to the training sample set to obtain an updated training sample set; The cost comparison model is trained based on the updated training sample set to obtain the updated cost comparison model.

7. A scheduling runtime comparison device for an operator, characterized in that, The device includes: The first acquisition unit is used to acquire at least two candidate schedules corresponding to the target computation expression, wherein the target computation expression is used to describe the computation logic of the operator, and the candidate schedule is the executable code of the operator generated based on the target computation expression; The second acquisition unit is used to acquire a cost comparison model, wherein the cost comparison model is a model obtained by training a neural network using multiple sample scheduling. The calling unit is used to call the cost comparison model output to obtain the cost comparison result based on the at least two candidate schedules. The cost comparison result is used to indicate the order of the execution time of the at least two candidate schedules.

8. A scheduling runtime comparison device for an operator, characterized in that, The device includes: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1-6 when executing the instructions.

9. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is run on a computer, the computer performs the method as described in any one of claims 1-6.

Images