Dynamic operator acceleration tuning method combining meta learning and reinforcement learning

Abstract

A dynamic operator accelerated tuning method combining meta learning and reinforcement learning comprises the following steps: firstly, constructing a generalization search space based on operator scheduling primitives, extracting tensor program structure features, and establishing a dynamic operator cost model; secondly, intelligent agent parameters are quickly adjusted by applying a meta-learning optimization algorithm, and search is accelerated by utilizing a reinforcement learning state encoder and a meta-gradient descent algorithm; and further, an Autotuning technology is combined with a deep reinforcement learning architecture to carry out operator optimization strategy search. Designing a reward function, and feeding back and guiding a search behavior according to code performance. Through a control variable method and a quantitative analysis method, a calculation and memory access performance model is constructed, operator performance key factors are analyzed, and limit performance under different hardware conditions is evaluated. And finally, selecting an optimal operator to generate a code, and converting the code into a machine code of the target hardware platform. By designing a high-quality generalized search space and performance limit analysis method, the efficiency and precision of operator tuning are remarkably improved, and optimization support is provided for deployment of a deep learning system.

Description

Technical Field

[0001] The present invention relates to the application field of deep learning, compilation technology and high-performance computing cross-technology, and in particular to a dynamic operator acceleration tuning method combining meta-learning and reinforcement learning. Background Art

[0002] As the smallest execution unit for executing deep learning models, the performance of operators greatly affects the overall execution efficiency and power consumption of the model. High-performance operators need to be optimized in combination with the architectural characteristics of the target hardware to fully utilize the computing power of the target hardware. Due to the limitations of manual tuning methods such as high development cost and low computing performance, automatic tuning methods have received increasing attention from academia and industry in recent years.

[0003] Depending on whether the shape and size of the operator are all known at compile time, the operator can be divided into static tensor operators and dynamic tensor operators. Current research focuses on the automatic tuning of static tensor operators. According to the different intermediate representations of operator compilation, operator tuning is currently mainly divided into automatic tuning algorithms based on Halide representation and automatic tuning algorithms based on polyhedron representation. At the same time, a number of automatic tuning algorithms for static operators have emerged in academia, such as AutoTVM, Ansor, and FlexTensor. These automatic optimization methods use random and evolutionary search algorithms to iteratively search the search space. The disadvantage is that the search process is not thorough, and the optimal and suboptimal results cannot be found. In addition, due to the increase in the number of searches, the entire search requires a lot of time.

[0004] In recent years, polyhedral representation has also been widely used in specific fields to optimize computationally intensive programs, such as Tensor Comprehensions in the field of deep learning, using an evolutionary search-based parameter tuner to search for the best parameter configuration. In addition, some work has begun to try to expand the expressive power of polyhedral representation. Tiramisu borrowed the idea of ​​decoupling definition and scheduling from Halide representation, and introduced a set of scheduling commands based on polyhedral representation, which completely decoupled computational representation, loop transformation, data layout, and multi-machine communication through a four-level intermediate representation. This work requires users to manually write scheduling plans using scheduling commands. Stripe proposed a nested polyhedral model to analogize the nested structure in deep learning operators and accelerator topologies to support the scheduling of large-scale parallel computing on different target hardware. Polyhedral representation is mainly used for compilation optimization of nested loop programs, and requires the program to meet static control properties, that is, it only contains nested loops with constant step sizes, and the branch condition expressions and array index expressions in the loop body must be affine functions of the loop iteration variables.

[0005] At present, there is relatively little attention paid to the automatic tuning of dynamic operators. Nimble proposed by Shen et al. uses a custom template search algorithm to extend the existing Tuning algorithm to support dynamic operators. Zhang et al. use the same set of abstractions and a unified intermediate representation layer to compile and optimize dynamic operators. Cui et al. use two new programming abstractions, Cell and Router, to allow application developers to explicitly convey dynamic semantic information to the framework, thereby solving the problem of dynamic operator optimization. At present, there is no corresponding modeling and performance estimation methodology that can more conveniently evaluate the theoretical performance upper limit of operators.

[0006] The existing search space design for automatic tuning of deep learning operators is the basis of the entire optimization process. The relationship between its scheduling primitives will affect the final search efficiency. At the same time, the richness of the scheduling primitive set determines the upper limit of the computing performance of the optimal solution that the automatic tuning algorithm can theoretically find. However, for the dynamic operators in the model, it is impossible to implement operator fusion optimization during compilation, nor can it use known parameter values ​​to generate targeted optimization implementations. It can only directly call the operator library provided by the hardware manufacturer. Its optimization can only stay at the computational graph level and cannot go deep into the operator level.

[0007] At the same time, the data flow of operator optimization needs to pass through multiple storage levels such as HBM, L2 cache, L1 buf, L0 buf, etc. Its performance is affected by the content capacity of these levels and the bandwidth of the data pipeline between the levels. Therefore, any capacity bandwidth mismatch will reduce the utilization rate of the tensor computing core Tensor Core. The current operator theoretical limit analysis is still an unresolved issue.

[0008] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0009] The main purpose of the present invention is to solve the problems existing in the above-mentioned background technology and provide a dynamic operator acceleration tuning method combining meta-learning and reinforcement learning.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] In a first aspect of the present invention, a dynamic operator acceleration tuning method combining meta-learning and reinforcement learning comprises the following steps:

[0012] a. Under the deep reinforcement learning framework, construct a search space for dynamic operators, including: i. construct a generalized search space based on operator scheduling primitives; ii. characterize tensor programs and extract their structural features; iii. construct a dynamic operator cost model;

[0013] b. Using meta-learning optimization algorithms, we can quickly adjust and optimize the parameter configuration of intelligent agents through a small number of samples, and use reinforcement learning state encoders and meta-gradient descent algorithms to accelerate the search process;

[0014] c. Use auto-tuning machine learning technology and deep reinforcement learning search architecture to search for operator optimization strategies;

[0015] d. Design and implement a reward function to evaluate and guide the search behavior of the intelligent agent based on the code performance feedback on the execution device;

[0016] e. Use the control variable method and quantitative analysis method to mathematically model and analyze the key factors that affect operator performance, including building computing performance models and memory access performance models to determine the relationship between operators and hardware, and evaluate the ultimate performance of operators under different hardware conditions;

[0017] f. Select the optimal operator found for code generation, generate executable code for the target hardware platform, and convert the optimized operator scheduling strategy into machine code for the target hardware platform.

[0018] In a second aspect of the present invention, a computer program product includes a computer program, and when the computer program is executed by a processor, the dynamic operator acceleration tuning method is implemented.

[0019] The present invention has the following beneficial effects:

[0020] The present invention proposes an innovative dynamic operator acceleration tuning method combining meta-learning and reinforcement learning. The method breaks through the limitations of the prior art on dynamic operator optimization, and realizes in-depth optimization of operator scheduling primitives by constructing a generalized search space that is common to both static and dynamic operators. The present invention uses meta-learning to quickly extract common patterns in optimization tasks, provides prior knowledge and initialization strategies for intelligent agents, and significantly improves search efficiency. At the same time, through the deep reinforcement learning framework, the present invention can adapt to rapidly changing optimization tasks, realize the limit analysis of operator performance and the evaluation of theoretical performance upper limits, and guide the performance optimization of operators and chip specification design. In addition, the present invention also takes into account the hardware limitations in the real environment, such as processor architecture, memory bandwidth, and communication bottlenecks. Through the control variable method and quantitative analysis method, a computing performance model and a memory access performance model are constructed, so as to explore the optimal solution or approximate optimal solution to operator performance under the trade-off of performance, power consumption, and area, and provide strong optimization support for the deployment and operation of deep learning systems.

[0021] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The overall algorithm roadmap of the dynamic operator acceleration tuning method of an embodiment of the present invention.

[0023] Figure 2 This is a diagram of an operator optimization framework that combines meta-learning and reinforcement learning search according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0025] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0026] Traditional dynamic operator optimization schemes cannot implement operator fusion optimization during compilation, nor can they use known parameter values ​​to generate targeted optimization implementations. They can only directly call the operator library provided by the hardware manufacturer, resulting in their optimization staying at the computational graph level and unable to go deep into the operator level, and the current operator theoretical limit analysis is still an unresolved issue. The present invention proposes an acceleration and intelligent optimization search algorithm for dynamic operators, which uses deep reinforcement learning and meta-learning algorithms. First, a relatively generalized search space is constructed based on operator scheduling primitives; then, under the framework of deep reinforcement learning, meta-learning is used to quickly adjust and optimize its parameter configuration through a small number of samples, so that the optimal scheduling strategy of dynamic operators can be solved quickly. Finally, based on the control variable method and the quantitative analysis method, the operator theoretical optimal performance limit is analyzed. The present invention combines meta-learning with deep reinforcement learning to achieve a better dynamic operator optimization scheme, and at the same time provides an effective model and means for analyzing the theoretical performance of operators based on the control variable method and the quantitative analysis method.

[0027] See also Figure 1 , an embodiment of the present invention provides a dynamic operator acceleration tuning method combining meta-learning and reinforcement learning, comprising the following steps:

[0028] a. Under the deep reinforcement learning framework, construct a search space for dynamic operators, including: i. construct a generalized search space based on operator scheduling primitives; ii. characterize tensor programs and extract their structural features; iii. construct a dynamic operator cost model;

[0029] b. Using meta-learning optimization algorithms, we can quickly adjust and optimize the parameter configuration of intelligent agents through a small number of samples, and use reinforcement learning state encoders and meta-gradient descent algorithms to accelerate the search process;

[0030] c. Use auto-tuning machine learning technology and deep reinforcement learning search architecture to search for operator optimization strategies;

[0031] d. Design and implement a reward function to evaluate and guide the search behavior of the intelligent agent based on the code performance feedback on the execution device;

[0032] e. Use the control variable method and quantitative analysis method to mathematically model and analyze the key factors that affect operator performance, including building computing performance models and memory access performance models to determine the relationship between operators and hardware, and evaluate the ultimate performance of operators under different hardware conditions;

[0033] f. Select the optimal operator found for code generation, generate executable code for the target hardware platform, and convert the optimized operator scheduling strategy into machine code for the target hardware platform.

[0034] The method of the present invention is designed to construct a generalized search space that is common and high-quality for both static operators and dynamic operators, and can perform reasonable modeling based on hardware capabilities. The upper limit of the theoretical performance of the operator can be easily evaluated through operator input and hardware specification parameters. On the one hand, it guides the performance optimization of existing operators, and on the other hand, it guides the chip specification design, balances between performance, power consumption, and area, and improves operator performance under the constraints of area and power consumption. The present invention can further reduce the complexity of the code optimization space and improve the dynamic operator compilation optimization search performance. The present invention uses meta-learning to learn to quickly adapt to previously unseen scheduling template design spaces, as well as the characteristics of rapid and accurate search in space, which can improve the efficiency of deep reinforcement learning to find the best operator optimization in the optimization space. The present invention combines meta-learning with deep reinforcement learning to achieve a better dynamic operator optimization solution, and at the same time provides a model and means for analyzing the theoretical performance of operators based on the control variable method and quantitative analysis method.

[0035] In some embodiments, step a specifically includes: i. using intra-stage scheduling primitives and inter-stage scheduling primitives, combined with evolutionary algorithms and derivative inference rules to expand the search space; ii. manually extracting the structural features of the tensor program, including operator complexity, memory usage, data dependency, computing mode (serial / parallel), etc., to form a tensor program feature description; ii. using supervised learning to train a cost regression model on a feature data set, constructing a microkernel-based cost model to evaluate the relative merits of candidate scheduling microkernels, and combining the designed dispatch decision tree model to select the scheduling kernel with the best performance from a group of microkernels selected by the search algorithm.

[0036] In some embodiments, in step b, the meta-learning algorithm extracts shared knowledge and patterns between different operator optimization tasks through a small number of samples, providing prior knowledge and initialization strategies for the reinforcement learning agent; wherein, a set of initial parameters of the model are obtained through training, so that the model initialized with the initial parameters can maximize the performance of the new task through only one or several gradient updates on a small number of samples.

[0037] See also Figure 2 In some embodiments, the intelligent agent of the deep reinforcement learning framework is defined as a process state transfer function, which uses a graph neural network GNN to retain the grammatical, semantic, contextual and logical information of the source text.

[0038] Specific as Figure 2 As shown, in some embodiments, the deep reinforcement learning framework specifically includes: using a graph neural network GNN to vectorize the state of the source code to capture the grammatical and semantic contextual logic information of the source text; the intelligent agent can use a variety of reinforcement learning algorithms to define its action set, including but not limited to Q-learning, DQN, MCTS, PPO and other reinforcement learning algorithms; adjusting the parameters of the intelligent agent through the meta-gradient descent algorithm to adapt to rapidly changing optimization tasks; utilizing a server-client architecture, in which the server is responsible for the execution of optimization tasks and the calculation of rewards, and the client is responsible for executing code generation and the deployment of optimized code; designing a reward function to evaluate the reward based on the performance feedback of the optimized code running on the execution device; transferring the state of the intelligent agent from the server to the client through a transfer function to achieve state synchronization and code updating.

[0039] In some embodiments, in the construction of a deep reinforcement learning search architecture, Graph2vec encoding is used to vectorize the state of the source code; the action of the intelligent agent includes the selection of the underlying implementation of the operator, which affects the direction and effect of code optimization; the reward function evaluates the optimization effect based on the running time difference between the new model and the old model; the GRPC process call interaction method is used to transfer data between the server and the client to achieve remote execution and state synchronization of optimization tasks; search and tuning are performed on the server, and the collected performance data is used to update the strategy of the intelligent agent; in the client deployment phase, feedback on the selected optimization operation is obtained and used to further guide the search process.

[0040] In some embodiments, the deep reinforcement learning framework further includes a search strategy evaluation and optimization step based on performance feedback, wherein a value function is used to evaluate the search strategy of the intelligent agent to determine the effectiveness of the strategy; a strategy method is used to select the optimal action to maximize long-term returns; the value function and the strategy method are combined to continuously optimize the search strategy through iterative training; and the code performance feedback on the execution device is used to adjust and improve the search strategy to improve the decision-making performance of the intelligent agent.

[0041] In some embodiments, in step e, the main factors affecting the execution efficiency of the operator are analyzed by the control variable method and the quantitative analysis method, and a multivariate function model is constructed to evaluate the performance of the operator. The computing performance model is expressed as the product of the average computing time of a single element and the number of elements that the computing core needs to calculate, and the memory access performance model is constructed based on memory access performance testing and analysis.

[0042] The specific embodiments of the present invention are further described below.

[0043] The present invention proposes a dynamic operator acceleration tuning method combining meta-learning and reinforcement learning. It is proposed to construct a generalized search space based on scheduling primitives and evolutionary algorithms to achieve the purpose of efficient optimization of both static operators and dynamic operators. The control variable method and quantitative analysis method are used to analyze the key factors affecting the optimal theoretical performance limit of the operator. And based on these key influencing factors, a computing performance model and a memory access performance model are constructed. The rapid adaptability of meta-learning is used to extract common patterns in operator optimization problems, provide prior knowledge and initialization strategies for reinforcement learning agents, accelerate the optimization process and improve search efficiency.

[0044] The method proposed in the present invention is an automatic tuning method for dynamic operators in deep neural networks, analyzing the optimal theoretical performance limit of operators, including:

[0045] (1) Propose a dynamic operator optimization strategy combining meta-learning and reinforcement learning

[0046] By combining meta-learning and reinforcement learning algorithms, we propose to build a highly general and adaptable operator optimization framework that can flexibly adapt to different software and hardware environments and efficiently search for the optimal operator strategy. This automated tool provides strong optimization support for the deployment and operation of deep learning systems.

[0047] (2) Finding the optimal theoretical performance limit of the operator

[0048] Taking into account the hardware limitations in the real environment (such as processor architecture, memory bandwidth, communication bottlenecks, etc.), the present invention can analyze the relationship between operator computing speed, accuracy and resource consumption under ideal conditions, and explore how to achieve the optimal solution or approximate optimal solution of operator performance under these constraints.

[0049] The overall algorithm roadmap of the dynamic operator acceleration tuning method of the embodiment of the present invention is as follows: Figure 1 shown.

[0050] Dynamic operator feature description and search space modeling

[0051] This part mainly includes three important contents: construction of dynamic operator search space, description of tensor program features, and construction of dynamic operator cost model.

[0052] (1) Use scheduling primitives including but not limited to: Intra-stage scheduling primitives such as split, reorder, and unroll to split computing tasks and control memory access trajectories by adjusting loop structures. Inter-stage scheduling primitives such as computer_* primitives specify the evaluation timing of operator calculations, cache_* primitives specify operator cache levels, etc. Also, apply evolutionary algorithms and derived inference rules to expand the search space. Finally, combine the microkernel-based cost model to build a more generalized search space.

[0053] (2) Characterization of Tensor Programs

[0054] Analyze the basic computing properties of tensor programs and manually extract the structural characteristics of operator programs, such as operator computational complexity, memory usage, data dependency, and computing mode (serial / parallel).

[0055] (3) Construction of dynamic operator cost model

[0056] A supervised learning method is used to train the cost regression model on the feature data set, and then a microkernel-based cost model is constructed to evaluate the relative advantages and disadvantages of candidate scheduling microkernels. Combined with the designed dispatch decision tree model, the scheduling kernel with the best performance is selected from a group of microkernels selected by the search algorithm.

[0057] Operator optimization method combining meta-learning and reinforcement learning

[0058] Figure 2 The operator optimization framework combining meta-learning and reinforcement learning search in an embodiment of the present invention is shown. The operator optimization method mainly includes three important contents: deep reinforcement learning search architecture construction, meta-learning optimization algorithm, and performance evaluation of the selected search strategy.

[0059] (1) Construction of deep reinforcement learning search architecture. In the deep reinforcement learning framework, the intelligent agent is defined as a process state transfer function. Since the core of the present invention is dynamic operator optimization, the optimization involved mainly uses the code source text as the state representation. The use of graph neural network GNN can retain the grammatical, semantic, contextual and logical information of the source text, thereby improving the information storage accuracy of the entire intelligent agent. It also contains states (neural network operator optimization space) in order to represent text in vectorized form Code,The present invention uses Graph2vec to encode the state representation function. In addition, the action is defined as the operator bottom layer implementation selection, and the reward is defined as the running time difference between the new model and the old model. Finally, the code data information is converted into vectorized representation for data transmission and training, and the GRPC process calls the interactive method to transfer data for interaction, so as to realize search and tuning on the server side, and obtain feedback on the selected optimization operation in the deployment stage.

[0060] (2) Meta-learning optimization algorithm. Its main design method is to train a set of initial parameters of the model so that the model initialized with this set of parameters can maximize the performance of the new task through only one or several gradient updates on a small number of samples. The present invention uses meta-learning to extract shared knowledge and patterns between different operator optimization tasks, so that the model can quickly adjust and optimize the operator configuration through a small number of samples when facing new deep learning tasks or hardware platforms.

[0061] (3) Performance evaluation of the selected search strategy. The value function and the strategy method are used in combination. The value function is used to evaluate the quality of the current search strategy, and the strategy method is used to select the optimal action. By combining these two algorithms for continuous iterative training, the value function and the strategy method are gradually optimized to improve the decision-making performance of the intelligent agent.

[0062] Limit Analysis of Operator Optimization Theory

[0063] The theoretical limit analysis of operator optimization includes two parts: studying the key factors affecting operator performance and mathematical modeling of the relationship between operators and hardware.

[0064] (1) Operator execution efficiency is affected by factors such as operator scheduling related parameters and hardware platform related parameters. The present invention defines operator execution efficiency and related influencing factors as multivariate functions, and its related influencing factors are positioned as independent variables. The control variable method is used to quantitatively analyze the statistical reinforcement learning algorithm to search for the operator's optimal execution efficiency limit value under the condition of a single variable change, and compare the execution efficiency limit values ​​under different conditions. Analyze the main factors affecting operator execution efficiency.

[0065] (2) After discovering the key performance parameters that affect the operator's operating performance during the operator optimization process, establish a computing performance model and a memory access performance model. Establish a computing performance model: The computing model part can be expressed as the product of the average computing time of a single element (bit) and the number of elements that the computing core needs to calculate. The average computing time of a single element is obtained through the performance data collection method, and the operator execution time is estimated based on the computing load of each computing core. Establish a memory access performance model: Analyze the memory access behavior of the operator, combine the relationship between the hardware memory access bandwidth and the memory access granularity, and construct a calculation formula for the memory access time based on memory access performance testing and analysis.

[0066] In summary, the present invention proposes a dynamic operator acceleration tuning algorithm that combines meta-learning and reinforcement learning. It proposes to construct a generalized search space based on scheduling primitives and evolutionary algorithms to achieve static Both operators and dynamic operators can perform high The purpose of efficiency optimization is to use the control variable method and quantitative analysis method. Analyze the key factors that affect the optimal theoretical performance limit of operators. And build computing performance models and memory access performance models based on these key influencing factors. Use the rapid adaptability of meta-learning to extract common patterns in operator optimization problems, provide prior knowledge and initialization strategies for reinforcement learning agents, accelerate the optimization process and improve search efficiency.

[0067] Traditional dynamic operator optimization schemes cannot implement operator fusion optimization during compilation, nor can they use known parameter values ​​to generate targeted optimization implementations. They can only directly call the operator library provided by the hardware manufacturer, resulting in their optimization only staying at the computational graph level and unable to go deep into the operator level. In addition, the current operator theoretical limit analysis is still an unresolved issue. The present invention can effectively solve this problem, combining meta-learning with deep reinforcement learning to achieve a better dynamic operator optimization scheme, and at the same time providing an effective model and means for analyzing the theoretical performance of operators based on the control variable method and quantitative analysis method.

[0068] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0069] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0070] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.

[0071] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0072] In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. The device embodiments described above are only exemplary. For example, the division of the units is only a logical function division. In actual implementation, there may be other ways. The division method, such as: multiple units or components can be combined, or Can be integrated into another system, Or some features may be ignored or not performed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0073] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0074] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0075] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0076] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0077] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0078] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0079] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0080] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A dynamic operator acceleration tuning method combining meta-learning and reinforcement learning, characterized in that: The following steps are involved: a. Under the deep reinforcement learning framework, construct a search space for dynamic operators, including: i. construct a generalized search space based on operator scheduling primitives; ii. characterize tensor programs and extract their structural features; iii. construct a dynamic operator cost model; b. Using meta-learning optimization algorithms, we can quickly adjust and optimize the parameter configuration of intelligent agents through a small number of samples, and use reinforcement learning state encoders and meta-gradient descent algorithms to accelerate the search process; c. Use auto-tuning machine learning technology and deep reinforcement learning search architecture to search for operator optimization strategies; d. Design and implement a reward function to evaluate and guide the search behavior of the intelligent agent based on the code performance feedback on the execution device; e. Use the control variable method and quantitative analysis method to mathematically model and analyze the key factors that affect operator performance, including building computing performance models and memory access performance models to determine the relationship between operators and hardware, and evaluate the ultimate performance of operators under different hardware conditions; f. Select the optimal operator found for code generation, generate executable code for the target hardware platform, and convert the optimized operator scheduling strategy into machine code for the target hardware platform.

2. The dynamic operator acceleration tuning method according to claim 1, characterized in that: Step a specifically includes: i. Use intra-stage scheduling primitives and inter-stage scheduling primitives, combined with evolutionary algorithms and derived inference rules to expand the search space; ii. Manually extract the structural features of the tensor program to form a tensor program feature description; ii. Use supervised learning to train the cost regression model on the feature data set and construct a microkernel-based cost model to evaluate the relative advantages and disadvantages of candidate scheduling microkernels. Combined with the designed dispatch decision tree model, the scheduling kernel with the best performance is selected from a group of microkernels selected by the search algorithm.

3. The dynamic operator acceleration tuning method according to claim 1 or 2, characterized in that: In step b, the meta-learning algorithm extracts shared knowledge and patterns between different operator optimization tasks through a small number of samples, providing prior knowledge and initialization strategies for the reinforcement learning agent; wherein, a set of initial parameters of the model are obtained through training, so that the model initialized with the initial parameters can maximize the performance of the new task through only one or several gradient updates on a small number of samples.

4. The dynamic operator acceleration tuning method according to any one of claims 1 to 3, characterized in that: The intelligent agent of the deep reinforcement learning framework is defined as a process state transfer function, and the graph neural network GNN is used to retain the grammatical, semantic, contextual and logical information of the source text.

5. The dynamic operator acceleration tuning method according to claim 4, characterized in that: The deep reinforcement learning framework specifically includes: Use graph neural network GNN to vectorize the state of source code to capture the grammatical and semantic contextual logic information of the source text; The intelligent agent uses a reinforcement learning algorithm to define its set of actions; The parameters of the intelligent agent are adjusted through the meta-gradient descent algorithm to adapt to the rapidly changing optimization tasks; Leverage a server-client architecture, where the server is responsible for executing optimization tasks and calculating rewards, and the client is responsible for executing code generation and deploying optimization code; Design a reward function that evaluates the reward based on performance feedback from the optimized code running on the execution device; The state of the intelligent agent is transferred from the server to the client through the transfer function to achieve state synchronization and code update.

6. The dynamic operator acceleration tuning method according to claim 5, characterized in that: The state of the source code is vectorized using Graph2vec encoding; the action of the intelligent agent includes the selection of the underlying implementation of the operator, which affects the direction and effect of code optimization; the reward function evaluates the optimization effect based on the running time difference between the new model and the old model; the GRPC process call interaction method is used to transfer data between the server and the client to achieve remote execution and state synchronization of optimization tasks; Search and tuning are performed on the server side, using the collected performance data to update the strategy of the intelligent agent; during the client deployment phase, feedback on the selected optimization operations is obtained and used to further guide the search process.

7. The dynamic operator acceleration tuning method according to claim 5 or 6, characterized in that: It further includes a search strategy evaluation and optimization step based on performance feedback, wherein a value function is used to evaluate the search strategy of the intelligent agent to determine the effectiveness of the strategy; a policy method is used to select the optimal action to maximize long-term returns; the value function and the policy method are combined to continuously optimize the search strategy through iterative training; and the code performance feedback on the execution device is used to adjust and improve the search strategy to improve the decision-making performance of the intelligent agent.

8. The dynamic operator acceleration tuning method according to any one of claims 1 to 7, characterized in that: In step e, the main factors affecting the execution efficiency of the operator are analyzed by the control variable method and the quantitative analysis method, and a multivariate function model is constructed to evaluate the operator performance.

9. The dynamic operator acceleration tuning method according to any one of claims 1 to 8, characterized in that: In step e, the computing performance model is expressed as the product of the average computing time of a single element and the number of elements that the computing core needs to calculate, and the memory access performance model is constructed based on memory access performance testing and analysis.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the dynamic operator acceleration tuning method according to any one of claims 1 to 9 is implemented.

Images