Machine learning model deployment optimization technology based on search optimization

Abstract

The invention relates to the technical field of information technology, and discloses a machine learning model deployment optimization technology based on search optimization, the optimization of the model is a process of exploring an optimal solution in a complex optimization space, especially decomposition, copying and combination of a complex model, and task allocation for computing resources. Therefore, an optimal solution needs to be explored in an optimization space of four dimensions. Therefore, an optimization algorithm is constructed in a basic mode based on MCTS (Monte Carlo Tree Search) so as to optimize the throughput rate of the deployment algorithm, and under the condition that computing resources, a machine learning algorithm model set and concurrency requirements are given, model segmentation is combined with the optimization algorithm, and then the model is deployed at computing nodes in a sub-module manner, so that the throughput rate of the deployment algorithm is optimized. Therefore, computing power resources of the computing nodes are effectively utilized, communication overhead among the nodes is reduced, and a deployment scheme of utilizing the computing resources to the greatest extent is realized.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a machine learning model deployment optimization technology based on search optimization to achieve deployment optimization technology for large-scale machine learning models, which is mainly used under the premise of given computing resources, algorithm models to be deployed, and the number of concurrent processing. Background Art

[0002] Large models have become one of the main application methods for the industrialization of machine learning technology, but they also bring huge computing pressure. Efficient large model deployment can achieve a good application experience under limited computing resources. The optimization of large models is mainly reflected in the following aspects: 1. Efficient parallel computing; mainly focusing on the performance of computing nodes and the collaborative capabilities of computing clusters; 2. Efficient data I / O capabilities: involving network system performance and storage system performance; 3. Efficient memory management; reduce memory usage and access bandwidth through model memory access optimization; 4. Optimize the model itself.

[0003] In response to points 1 and 2, namely, improving the efficiency of parallel computing and data I / O, this invention proposes a deployment solution that maximizes throughput under given computing resources (including computing nodes and network connections) by decomposing, copying, and combining the model to reconstruct the model's computing flow chart, so as to achieve operation optimization, including efficient use of computing resources and reduced I / O latency, thereby achieving a deployment solution that maximizes throughput under given resources and model types, rather than quantities. Summary of the invention

[0004] As mentioned above: the optimization of the model is the process of exploring the optimal solution in a complex optimization space, especially the decomposition, replication, combination of complex models, and the task allocation for computing resources, which leads to the need to explore the optimal solution in the optimization space of four dimensions. Therefore, this invention adopts the basic method based on Monte Carlo Tree Search (MCTS) to construct an optimization algorithm to achieve the throughput of the optimized deployment algorithm.

[0005] A machine learning model deployment optimization technology based on search optimization, characterized by searching for the optimal deployment solution to maximize the utilization of computing resources and maximize the throughput of machine learning algorithm models under given computing resources, machine learning algorithm model sets and concurrency requirements; The deployment solution search method includes: Constructing the search space, including computing task organization and computing node allocation; The optimization search process, including the allocation of computing resources and the expansion of the search tree; Throughput evaluation.

[0006] Preferably, the search space is through a data flow graph G D And the computational graph G C The mathematical expression of G D It represents the organization mode of computing tasks, Gc represents the topological structure of computing nodes, and the computing task organization includes operator unit definition, operator replication, operator splitting and operator combination.

[0007] Preferably, the operator unit definition is to define the minimum computing unit of a given algorithm model, and the minimum computing unit supports optimization by defining operator units of different granularities.

[0008] Preferably, the operator is copied to improve tensor parallelism, and the operator is copied in multiple copies.

[0009] Preferably, the operator splitting is to split non-minimum computing unit operators.

[0010] Preferably, the operator combination is to combine several operators into a composite operator to reduce data I / O delay between computing nodes.

[0011] Preferably, the optimization search process adopts a Monte Carlo Tree Search (MCTS) algorithm, and the computing resource allocation is to allocate nodes in the algorithm data flow graph to computing nodes to form a mapping relationship.

[0012] Preferably, the Monte Carlo Tree Search (MCTS) algorithm also includes an expansion method of the search tree, which reconstructs the data stream and defines a new mapping relationship.

[0013] Preferably, the throughput (cost) evaluation is determined by calculating the computation time required for each node in the data flow graph and the I / O delay between different nodes. The lower the cost, the higher the efficiency.

[0014] Preferably, the method also includes combining model segmentation with an optimization algorithm, and deploying the model modules on computing nodes through throughput evaluation.

[0015] Compared with the prior art, the present invention provides a machine learning model deployment optimization technology based on search optimization, which has the following beneficial effects: By decomposing, copying, and combining the model, the model's computational flowchart is reconstructed to achieve operational optimization, including efficient use of computing resources and reduced I / O latency, thereby achieving a deployment solution that maximizes throughput under given resources and model types rather than quantities. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The search process for the present invention; Figure 2 A search tree expansion method of the present invention; DETAILED DESCRIPTION

[0017] The implementation of the present invention includes the construction of a search space based on the replication, combination and allocation of operators on computing resources after model decomposition; an optimization process based on MCTS; an expansion method of the search tree during the optimization process (i.e., generating new operator replication, combination and computing resource allocation schemes at each step); and a calculation method for evaluating the throughput of each configuration. As described below: Search space construction: The search space consists of two parts: computing task organization and computing node allocation. The computing task organization is achieved by decomposing the given algorithm model in the form of a data flow graph, where nodes represent operators and the associations between nodes represent the data flow. The computing task organization includes operator unit definition, operator replication, and operator combination.

[0018] 1. Operator unit definition: It is to define the minimum computing unit of a given algorithm model, such as matrix calculation. By defining the minimum computing units of different granularities, the search optimization algorithm invented in this project can be optimized at different granularities; 2. Operator replication: To improve tensor parallelism, operators can be replicated multiple times; 3. Operator splitting: splitting non-minimum computing unit operators; 4. Operator combination: Combine several operators into a composite operator to reduce possible data I / O delays between computing nodes.

[0019] The above step 1 implements the initial data flow graph G D = {V D , E D}Build, V D = {v d0 , v d1 , … v dm}, v di represents the task node, E D = {e d0 , e d1 , … e dk}, the search algorithm will execute steps 2, 3, and 4 to reconstruct the data flow graph. Each node in the graph is considered as a computational task.

[0020] Computational resources to compute the graph G C = {V C , E C} indicates that V C = {v c0 , v c1 , … vcm}, v ci represents the computing node, E C = {e c0 , e c1 , … e ck}, e cj = < c j , c j > represents the computing node c i , c j Based on the data flow graph, the given computing resources are allocated computing nodes, that is, the nodes v in the algorithm data flow graph are allocated to the given computing resources. di Assign to compute node v cj Here we define the data flow graph G D To the computational graph G C The mapping relationship is Conf.

[0021] Optimization process: The optimization process is to continuously iterate the deployment configuration to find the optimal deployment solution, that is, to achieve the optimal computing task organization and computing node allocation Conf opt : Construct the optimal data flow graph and achieve the highest throughput computing node allocation. Figure 1 As shown, the input to the search process includes: 1. A collection of artificial intelligence algorithm models. Each model is represented by a data flow graph. Models that support multiple concurrency have multiple data flow graphs. Therefore, the model collection is represented as G DS = {G D0 , G D1 , … G Dn}, G DS All graphs in the representation set are merged into a unified data flow graph; 2. Computational resources to compute graph G C = {V C , E C}express; The search process is iterative, starting from the initial data flow graph G DS And the computational graph G C The mapping relationship Conf0 is gradually expanded by the new data flow graph G DS ' and build a new mapping Conf i relationship until the optimal mapping Conf is found opt Conf is represented as a tree structure, so Child(Conf i ) indicates the node Conf i The corresponding next layer node set, parent(Conf i ) means Conf iThe corresponding parent node. Eval(Conf i ) indicates that Conf i Perform throughput evaluation. expand(Conf i ) indicates that Conf i To expand the child nodes, during the expansion process (described in detail in the next section), the input data flow graph G DS Reconstruct G DS ' and with G C A new mapping relationship is formed.

[0022] The search process is as follows:

[0023] Each Conf will evaluate the throughput at the beginning of the iteration loop. If it exceeds the threshold, the search will be stopped and the results will be published. The output of the search process is the optimal mapping relationship Conf opt , and contains the reconstructed data flow graph G DS .

[0024] Extension methods for searching the tree: like Figure 2 The search tree shown is extended in two ways: 1. Reconstruct the data flow graph G DS ; 2. According to the data flow graph G DS And the computational graph G C , define a new mapping relationship; The reconstruction of the data flow graph includes: copying nodes and splitting / combining nodes. The selection of reconstruction actions is based on existing reconstruction schemes and random selection of new schemes that have not been constructed.

[0025] The mapping from data flow graph to computation graph must meet the following conditions: 1. Data flow graph node v di Mapped to the computational graph node v cj , v cj Must satisfy v di Requirements for computing power and storage, otherwise mapping cannot be performed; 2. Data flow graph node v di Mapped to the computational graph node v cr , data flow graph node v dj Mapped to the computational graph node v cm , if v di With v dj There is an edge e between dij , then v cr With v cm There must be an edge e between crm, or they are the same node, otherwise mapping cannot be performed.

[0026] The mapping relationship between the data flow graph and the computation graph nodes can be established by filling them one by one through a greedy algorithm, or by filling them through a heuristic algorithm, or by selecting them randomly. For the initial mapping Conf0 required when the search algorithm starts, it can also be generated by performing one of the above three methods on the initial data flow graph and the computation graph.

[0027] Throughput evaluation: The throughput rate of the mapping relationship Conf is Eval(Conf) = 1 / cost(G DS ,G C ), that is, the data flow graph G DS In the computational graph G C The calculation efficiency and I / O latency are determined by the following formula: .

[0028] The throughput is evaluated by calculating the value of v for each node in the data flow graph. di (Computational task) The required computational time cost(v di )(by v di Corresponding to v cr The computational efficiency of the node) and the I / O latency cost between different nodes (v di , v dj ) (by v di Corresponding to v cr to v dj Corresponding to v cm Here, the cost of computing task nodes and task associations (edges in the data flow graph) is represented by the processing time required for a given input task t, so the lower the cost, the higher the efficiency.

[0029] By combining model segmentation with optimization algorithms, the model modules are deployed on computing nodes, thereby effectively utilizing the computing power resources of the computing nodes and reducing the communication overhead between nodes, thereby achieving a deployment plan that maximizes the use of computing resources.

Claims

1. A machine learning model deployment optimization technology based on search optimization, characterized in that Given computing resources, a set of machine learning algorithm models, and concurrency requirements, search for the optimal deployment solution to maximize the utilization of computing resources and maximize the throughput of machine learning algorithm models; The deployment solution search method includes: Constructing the search space, including computing task organization and computing node allocation; The optimization search process, including the allocation of computing resources and the expansion of the search tree; Throughput evaluation.

2. The machine learning model deployment optimization technology based on search optimization according to claim 1, characterized in that: The search space is represented by a data flow graph G D And the computational graph G C The mathematical expression of G D It represents the organization mode of computing tasks, Gc represents the topological structure of computing nodes, and the computing task organization includes operator unit definition, operator replication, operator splitting and operator combination.

3. The machine learning model deployment optimization technology based on search optimization according to claim 2, characterized in that: The operator unit definition is to define the minimum computing unit of a given algorithm model, and the minimum computing unit supports optimization by defining operator units of different granularities.

4. The machine learning model deployment optimization technology based on search optimization according to claim 2, characterized in that: The operator is replicated to improve tensor parallelism, and the operator is replicated in multiple copies.

5. The machine learning model deployment optimization technology based on search optimization according to claim 2, characterized in that: The operator splitting is to split the non-minimum computing unit operator.

6. The machine learning model deployment optimization technology based on search optimization according to claim 2, characterized in that: The operator combination is to combine several operators into a composite operator to reduce the data I / O delay between computing nodes.

7. The machine learning model deployment optimization technology based on search optimization according to claim 1, characterized in that: The optimization search process adopts the Monte Carlo Tree Search (MCTS) algorithm, and the computing resource allocation is to allocate the nodes in the algorithm data flow graph to the computing nodes to form a mapping relationship.

8. The machine learning model deployment optimization technology based on search optimization according to claim 7, characterized in that: The Monte Carlo Tree Search (MCTS) algorithm also includes an expansion method of the search tree, which reconstructs the data stream and defines a new mapping relationship.

9. The machine learning model deployment optimization technology based on search optimization according to claim 1, characterized in that: The throughput (cost) evaluation is determined by calculating the computation time required for each node in the data flow graph and the I / O delay between different nodes. The lower the cost, the higher the efficiency.

10. The machine learning model deployment optimization technology based on search optimization according to claim 9, characterized in that: It also includes model segmentation combined with optimization algorithms, and deploys model modules on computing nodes through throughput evaluation.

Images