Core particle-oriented deep model fault-tolerant deployment optimization method and system

Abstract

The application discloses a kind of deep model fault-tolerant deployment optimization methods and systems for core particle, which comprises the following steps: obtaining the calculation graph of deep model, the division strategy of operator and the irregular topology structure of fault core particle;By optimizing the division strategy of operator and the number allocation strategy of hardware resources, the calculation overhead, communication overhead and random mapping link contention overhead of calculation graph are minimized, to obtain the best division strategy of operator and the best resource allocation number;By optimizing the hardware mapping strategy of operator based on the irregular topology structure of core particle, the link contention overhead of core particle fault network is minimized, to obtain the best mapping scheme of operator;The above-mentioned overhead is regarded as total execution overhead, and the total execution overhead is minimized by iterative optimization, and finally the best scheduling scheme is obtained.The method of the application can obtain a scheduling scheme with stronger inference performance and smaller execution overhead, and promote the development and application of fault-tolerant deployment and optimization technology of deep learning large model on core particle.

Description

Technical Field

[0001] This invention belongs to the field of computer system optimization technology, specifically relating to a method and system for fault-tolerant deployment optimization of deep large models oriented towards core particles. Background Technology

[0002] In recent years, the emergence of chip-particle technology has had a positive impact on the development of deep learning large model accelerators. Chip-particle technology is a technique that integrates and packages multiple small chip units into a modular chip that meets specific functions. This technology breaks the design limitations of traditional single-chip designs, making chip design more flexible and efficient. In the field of deep learning, deep learning large models typically require a large amount of computing resources and storage space, thus their deployment and inference processes often consume significant time and energy. The emergence of chip-particle technology provides a new solution for the deployment and inference of deep learning large models. Deploying deep learning large models on chips for inference brings multiple benefits, including greater flexibility, lower cost, and faster inference speed. By breaking down deep learning large models into multiple smaller models and deploying them on multiple chips, inference speed can be significantly improved while reducing energy consumption.

[0003] However, with the continuous development of chip technology, its integration and complexity are also constantly increasing. Since a chip is composed of multiple small chip units integrated and packaged, some nodes may fail during its assembly process. These failures may cause the chip to malfunction or even fail to work properly. Currently, although some methods exist for detecting and repairing chip failures, how to effectively schedule chip resources and achieve efficient inference when faulty nodes are known to exist remains a problem to be solved. Existing scheduling strategies often lack consideration for chip failures, resulting in the inability to fully utilize the advantages of the chip when failures are present.

[0004] Therefore, achieving fault-tolerant deployment and optimization of large deep learning models on chips has become an urgent problem to be solved. Meanwhile, with the continuous development of chip technology, how to achieve fault-tolerant deployment and optimization of large deep learning models on chips has also become a research hotspot in academia and industry. Several research works are currently exploring solutions to these problems. For example, some research groups are developing chip-based fault-tolerant mechanisms, improving system reliability by introducing backup chips or using redundant computation. In addition, some research focuses on optimizing chip scheduling strategies to achieve more efficient inference. These research works provide new ideas and methods for solving the problem of fault-tolerant deployment and optimization of large deep learning models on chips.

[0005] However, despite some progress, existing methods still have shortcomings. For example, some fault-tolerance mechanisms may lead to increased system complexity and performance degradation; while some scheduling strategies cannot effectively handle resource scheduling problems under fault conditions. Therefore, further research and exploration are needed on methods and techniques for fault-tolerant deployment and optimization of large deep learning models at the chip level. Summary of the Invention

[0006] In view of the above, the purpose of this invention is to provide a fault-tolerant deployment optimization method and system for deep large-scale models oriented towards core particles. By comprehensively considering the computational overhead, communication overhead, and link contention overhead of random mapping in the deep large-scale model computation graph, as well as the link contention overhead in the faulty core particle network, the invention utilizes integer linear programming optimization to perform local optimization and global iterative optimization respectively, thereby minimizing the overhead and obtaining a scheduling scheme with more powerful inference performance and lower execution overhead. This promotes the development and application of fault-tolerant deployment and optimization technology for deep learning large-scale models on core particles.

[0007] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0008] In a first aspect, the present invention provides a fault-tolerant deployment optimization method for deep large-scale models oriented towards core particles, comprising the following steps:

[0009] Obtain the computation graph of the deep large model, the partitioning strategy of the operators in the computation graph, and the irregular topology of the core particles under the case of core particle failure;

[0010] By optimizing the partitioning strategy of the operator and the hardware resource allocation strategy based on the partitioning strategy, the computational overhead, communication overhead and link contention overhead of random mapping in the deep large model computation graph are minimized, and the optimal partitioning strategy and optimal resource allocation of the operator after local optimization are obtained.

[0011] By optimizing the hardware mapping strategy of the operator based on the irregular topology of the core particle, the link contention overhead of the core particle fault network is minimized, and the optimal mapping scheme of the locally optimized operator is obtained.

[0012] The computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network in the deep large model computation graph are regarded as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization. Finally, the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization constitute the optimal scheduling scheme.

[0013] Preferably, the acquisition of the computational graph of the deep large model, the partitioning strategy of the operators in the computational graph, and the irregular topology of the core particles under the condition of core particle failure includes:

[0014] Obtain a directed acyclic graph (DAG) of a large deep model during the inference phase as a computation graph G(V,E). This DAG consists of a set of nodes and a set of directed edges, where V represents the set of nodes for each operator in the computation graph, and E represents the set of directed edges for the data dependencies of each operator in the computation graph.

[0015] Based on the computation graph, obtain the set T of all partitioning strategies for all operators, where the set of partitioning strategies for each operator v is t. v , where t v ∈T;

[0016] Obtain the irregular topology of the core particles: mesh = {m1, m2, ..., m} i ,...,m n}, where i∈[1,2,...,n], m i Indicates available core nodes, where i represents the label of the available node, and m... i The total number is the total number of fault-free cores and is represented by size.

[0017] Preferably, the step of minimizing the computational overhead, communication overhead, and link contention overhead of random mapping in the computational graph of a large-scale deep model by optimizing the partitioning strategy of the operator and the hardware resource allocation strategy based on the partitioning strategy, to obtain the optimal partitioning strategy and optimal resource allocation quantity of the locally optimized operator, includes:

[0018] Obtain all partitioning strategies for the current operator v and its preceding operators u. For each partitioning strategy of a single operator, use {t1,t2,...,t}. p} contains a set of p integers to represent the number of partitions t for each partitioning strategy across the p dimensions. p ,t1~t p The values ​​of are all greater than or equal to 1. Each part needs to be allocated one chip resource. Therefore, the hardware resource allocation strategy for each partitioning strategy is expressed as:

[0019]

[0020] In the formula, M represents the total number of core resources corresponding to each partitioning strategy, and t i This represents the number of divisions based on the i dimensions;

[0021] Treating the partitioning strategies chosen by the current operator v and its preceding operator u as variables, we model them as one-hot codes s. u,v , where s u,v ∈{0,1} kv*ku kv and ku represent the number of partitioning strategies for the current operator v and its preceding operator u, respectively;

[0022] Obtain the computational and communication overhead for each operator and each partitioning strategy, where the computational overhead includes the intra-operator computational overhead (Comp). v Communication overhead includes intra-operator communication overhead Comm. v Inter, the inter-operator communication transmission overhead corresponding to each partitioning strategy of operator v. u,v And the link contention overhead Z of the random mapping obtained by randomly mapping the computational task to the core array after partitioning according to each partitioning strategy. v The computational overhead within the operator, Comp v Intra-operator communication overhead Comm v Inter-operator communication transmission overhead u,v Link contention overhead Z with random mapping v The summation is considered as the overall execution overhead of this part, and represented as a linear mathematical function solution R1:

[0023] R1=∑ (u,v)∈G s u,v T (Inter u,v +Comm v +Comp v +Z v )

[0024] In the formula, G represents the computation graph, and T represents the transpose;

[0025] The integer linear programming method is used to optimize R1, with the optimization objective being to minimize the execution cost R1. The solution yields the optimal partitioning strategy t for each operator. vi * The set of optimal partitioning strategies for all operators is represented as T. * , t vi * ∈T * And the optimal resource allocation size (size) of the core resources to be mapped for each operator. i * The set of optimal resource allocation quantities for all operators is represented by size. * size i * ∈size * .

[0026] Preferably, the step of optimizing the hardware mapping strategy of the operator based on the irregular topology of the core particle to minimize the link contention overhead of the core particle faulty network and obtain the optimal mapping scheme of the locally optimized operator includes:

[0027] The link overhead incurred in computing the current operator v and in moving data from its preceding operator u before the computation begins.

[0028] Calculate the link contention overhead in the current operator v due to intra-operator communication.

[0029] Calculate the link contention overhead when performing weight shifting before computation for the current operator v.

[0030] The above three parts of link contention overhead and The link contention overhead of the core-fault network is summed and represented as a linear mathematical function R2:

[0031]

[0032] The R² value is optimized using integer linear programming. The optimization objective is to minimize the execution cost R². The solution yields the optimal mapping scheme Map for the operator in the faulty core network. * .

[0033] Preferably, the link overhead incurred in calculating the current operator v and moving data from its predecessor operator u before the calculation begins is... include:

[0034] The current operator node is represented by v, the preceding operator by u, and the link overhead incurred in moving data. Represented as:

[0035]

[0036]

[0037] In the formula, cs, ct, cp, cq are the available nodes in the core, cs, ct, cp, cq ∈ {1, 2, ..., size}, size is the total number of fault-free cores, lk ∈ {1, 2, ..., L}, L is the total number of available links; G is the computation graph; for the two pairs of communication tasks (cs, ct) and (cp, cq) generated during the data transfer process, representing the communication from the source node cs to the destination node ct and the communication from the source node cp to the destination node cq respectively, it is necessary to stipulate that the source nodes and destination nodes of the two pairs of communication tasks cannot be the same, therefore, the constraints are cs ≠ ct, cp ≠ cq; Calculate whether both pairs of communication tasks used a certain link lk, variable An integer with a value of 0 or 1, where 1 indicates that link lk is used and 0 indicates that link lk is not used.

[0038] Preferably, the calculation of the link contention overhead caused by intra-operator communication in the current operator v is... include:

[0039] The current operator node is denoted by v, representing the link contention overhead caused by intra-operator communication. Represented as:

[0040]

[0041]

[0042] In the formula, cs,ct,cp,cq are the available nodes in the core, cs,ct,cp,cq∈{1,2,...,size}, size is the total number of fault-free cores, lk∈{1,2,...,L}, L is the total number of available links; V represents the set of each operator node in the computation graph; for the two pairs of communication tasks (cs,ct) and (cp,cq) generated when operator node v performs intra-operator communication, it must be stipulated that the source node and destination node of the two pairs of communication tasks cannot be the same, therefore, the constraints are cs≠ct,cp≠cq, cs≠cp,ct≠cq; Calculate whether both pairs of communication tasks used a certain link lk, variable An integer with a value of 0 or 1, where 1 indicates that link lk is used and 0 indicates that link lk is not used.

[0043] Preferably, the link contention overhead during weight shifting before calculating the current operator v is included. include:

[0044] The current operator node is represented by v. Due to the link contention overhead during weight shifting before operator computation... Represented as:

[0045]

[0046]

[0047] In the formula, cs,ct,cp,cq are the available nodes in the core, cs,ct,cp,cq∈{1,2,...,size}, size is the total number of fault-free cores, lk∈{1,2,...,L}, L is the total number of available links; V represents the set of each operator node in the computation graph; for the two pairs of communication tasks (cs,ct) and (cp,cq) generated by operator node v when performing weight transfer, it must be stipulated that the source node and destination node of the two pairs of communication tasks cannot be the same, therefore, the constraints are cs≠ct,cp≠cq, cs≠cp,ct≠cq; Calculate whether both pairs of communication tasks used a certain link lk, variable An integer with a value of 0 or 1, where 1 indicates that link lk is used and 0 indicates that link lk is not used.

[0048] Preferably, the computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network are considered as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization, ultimately obtaining the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization, constituting the optimal scheduling scheme, including:

[0049] The computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core-particle fault network are considered as the total execution overhead of the scheduling task. The goal of iterative optimization is to minimize the total execution overhead of the deep large model computation graph in the faulty core-particle topology. A maximum number of iterations, MaxIter, is set, and optimization stops when it is reached. This yields the optimal scheduling scheme, which includes the optimal partitioning strategy, the optimal number of resource allocations, and the optimal mapping scheme of the overall optimized operators.

[0050] Secondly, in order to achieve the above-mentioned objectives, the present invention also provides a fault-tolerant deployment optimization system for deep large-scale models oriented towards core particles, including: a computation graph and core particle structure acquisition module, a partitioning strategy and resource allocation optimization module, a mapping scheme optimization module, and a total execution overhead iterative optimization module;

[0051] The computation graph and core structure acquisition module is used to acquire the computation graph of the deep large model, the partitioning strategy of the operators in the computation graph, and the irregular topology of the core under the condition of core failure.

[0052] The partitioning strategy and resource allocation optimization module is used to minimize the computational overhead, communication overhead, and link contention overhead of the deep large model computation graph by optimizing the partitioning strategy of the operator and the hardware resource quantity allocation strategy based on the partitioning strategy, so as to obtain the optimal partitioning strategy and optimal resource allocation quantity of the locally optimized operator.

[0053] The mapping scheme optimization module is used to optimize the hardware mapping strategy of the operator based on the irregular topology of the core particle, minimize the link contention overhead of the core particle fault network, and obtain the best mapping scheme of the locally optimized operator.

[0054] The total execution overhead iterative optimization module is used to regard the computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network of the deep large model computation graph as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization, and finally the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization constitute the optimal scheduling scheme.

[0055] Thirdly, to achieve the above-mentioned objectives, embodiments of the present invention also provide a chip-oriented deep large model fault-tolerant deployment optimization device, including a memory and a processor. The memory is used to store a computer program, and the processor is used to implement the above-mentioned chip-oriented deep large model fault-tolerant deployment optimization method when the computer program is executed.

[0056] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0057] (1) This invention optimizes the computational overhead, communication overhead, and link contention overhead of random mapping in the deep large model computation graph, as well as the link contention overhead under the faulty core topology, to obtain the best scheduling scheme. It can make full use of core resources, reduce execution overhead, achieve optimized scheduling, and accelerate the inference process of deep large models when there are faults in the core network.

[0058] (2) This invention uses integer linear programming to perform local optimization and global optimization respectively. By taking the computational overhead, communication overhead, link contention overhead of the large model computation graph of local optimization and the link contention overhead under the fault core topology as the total execution overhead of the scheduling task for global optimization, it solves the problem of difficulty in solving due to excessive number of parameters in the global optimization process, thereby reducing the number of parameters and computational load of model optimization and improving the generalization ability and overall performance of the model.

[0059] (3) The present invention takes into account the situation of faulty core particles, and can effectively handle the resource scheduling problem of deep large model under the condition of core particle failure, give full play to the advantages of core particles, and can effectively schedule core particle resources and achieve efficient inference. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart illustrating a chip-oriented deep large model fault-tolerant deployment optimization method provided in an embodiment of the present invention;

[0062] Figure 2 This is a flowchart illustrating the optimal partitioning strategy and optimal resource allocation quantity for the computational operator provided in this embodiment of the invention.

[0063] Figure 3 This is a flowchart illustrating the optimal mapping scheme for the computational operator provided in this embodiment of the invention;

[0064] Figure 4 This is a flowchart illustrating the process of minimizing the total execution overhead to obtain the optimal mapping scheme through iterative optimization, as provided in an embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram of the structure of a core-oriented deep large model fault-tolerant deployment optimization system provided in an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0067] The inventive concept of this invention is as follows: Addressing the urgent need in existing technologies for an effective method to achieve fault-tolerant deployment and optimization of large deep learning models on cores, this invention provides a core-oriented fault-tolerant deployment optimization method and system for large deep learning models. It comprehensively considers the computational overhead, communication overhead, and link contention overhead of the large deep learning model's computational graph, as well as the link contention overhead in the faulty core network. Integer linear programming is used for both local and global optimization to minimize these two overheads. Local optimization mainly includes two aspects: first, minimizing the computational overhead, communication overhead, and link contention overhead under random mapping of the large deep learning model's computational graph, solving for the optimal partitioning strategy and resource allocation quantity for operators; second, minimizing the link contention overhead in the faulty core network, solving for the optimal mapping scheme for operators in the faulty core network. Global optimization involves jointly iteratively optimizing the results of these two local optimizations to obtain a scheduling scheme with stronger inference performance and lower execution overhead. Ultimately, this enables full utilization of core resources, reduced execution overhead, optimized scheduling, and accelerated inference of large deep learning models even when the core network is faulty.

[0068] Figure 1 This is a flowchart illustrating a chip-oriented deep large model fault-tolerant deployment optimization method provided by an embodiment of the present invention. Figure 1 As shown, the embodiment provides a fault-tolerant deployment optimization method for deep large-scale models oriented towards core particles, including the following steps:

[0069] S1, obtain the computation graph of the deep large model, the partitioning strategy of the operators in the computation graph, and the irregular topology of the core particles under the condition of core particle failure.

[0070] Specifically, a directed acyclic graph (DAG) of a large deep model during the inference phase is obtained as a computation graph G(V,E). This DAG consists of a set of nodes and a set of directed edges, where V represents the set of nodes for each operator in the computation graph, and E represents the set of directed edges for the data dependencies of each operator in the computation graph.

[0071] Based on the computation graph, obtain the set T of all partitioning strategies for all operators, where the set of partitioning strategies for each operator v is t. v , where t v ∈T.

[0072] Obtain the irregular topology of the core particles: mesh = {m1, m2, ..., m} i ,...,m n}, where i∈[1,2,...,n], m i Indicates available core nodes, where i represents the label of the available node, and m... i The total number is the total number of fault-free cores, denoted by size. The set of fault node labels is denoted by F.

[0073] S2 minimizes the computational overhead, communication overhead, and link contention overhead of random mapping in the deep large model computation graph by optimizing the partitioning strategy of the operator and the hardware resource allocation strategy based on the partitioning strategy, thus obtaining the optimal partitioning strategy and optimal resource allocation quantity of the locally optimized operator.

[0074] Specifically, such as Figure 2 As shown, it includes the following steps:

[0075] S201, based on all partitioning strategies of the current operator v and its preceding operator u, obtain the hardware resource allocation strategy.

[0076] For each partitioning strategy of a single operator, {t1,t2,...,t} are used. p} contains a set of p integers to represent the number of partitions t for each partitioning strategy across the p dimensions. p ,t1~t p The values ​​of are all greater than or equal to 1. Each part needs to be allocated one chip resource. Therefore, the hardware resource allocation strategy for each partitioning strategy is expressed as:

[0077]

[0078] In the formula, M represents the total number of core resources corresponding to each partitioning strategy, and t i This represents the number of divisions based on the i dimensions.

[0079] S202, model the partitioning strategy selected by the current operator v and its predecessor operator u into one-hot encoding.

[0080] Treating the partitioning strategies chosen by the current operator v and its preceding operator u as variables, we model them as one-hot codes s. u,v , where s u,v ∈{0,1} kv*ku kv and ku represent the number of partitioning strategies for the current operator v and its preceding operator u, respectively.

[0081] S203, obtain the computational and communication overhead for each operator and each partitioning strategy, as well as the link contention overhead for random mapping.

[0082] Computational overhead includes operator-level computational overhead (Comp). v Communication overhead includes intra-operator communication overhead Comm. v Inter, the inter-operator communication transmission overhead corresponding to each partitioning strategy of operator v. u,v Among them, the computational overhead within the operator, Comp... v To estimate the computational cost of each partitioning strategy for each operator on each hardware resource, the intra-operator communication overhead Comm is calculated. v This mainly includes the summation and reduction overhead caused by operator partitioning in tensor multiplication computation, as well as the inter-operator communication and transmission overhead. u,v This includes the overhead of transporting inputs and weights before operator computation.

[0083] The link contention overhead Z is calculated based on the random mapping (RandMap) obtained after task partitioning according to each partitioning strategy, and then mapped to the core array. v .

[0084] S204 uses integer linear programming to minimize the overhead and obtain the optimal partitioning strategy and optimal resource allocation quantity of the locally optimized operator.

[0085] The computational overhead within the operator, Comp v Intra-operator communication overhead Comm v Inter-operator communication transmission overhead u,v Link contention overhead Z with random mapping v The summation is considered as the overall execution overhead of this part, and represented as a linear mathematical function solution R1:

[0086] R1=∑ (u,v)∈G s u,v T (Inter u,v +Comm v +Comp v +Z v )

[0087] In the formula, G represents the computation graph, and T represents the transpose;

[0088] The integer linear programming (ILP) method is used to optimize R1, with the objective of minimizing the execution cost R1. The solution yields the optimal partitioning strategy t for each operator. vi * The set of optimal partitioning strategies for all operators is represented as T. * , t vi * ∈T * And the optimal resource allocation size (size) of the core resources to be mapped for each operator. i * The set of optimal resource allocation quantities for all operators is represented by size. * size i * ∈size * .

[0089] S3 optimizes the hardware mapping strategy of the operator based on the irregular topology of the core particle, minimizes the link contention overhead of the core particle faulty network, and obtains the best mapping scheme of the locally optimized operator.

[0090] Specifically, such as Figure 3 As shown, it includes the following steps:

[0091] S301, calculate the link overhead of the current operator v and the data transported from its predecessor operator u before the calculation begins.

[0092] The current operator node is represented by v, the preceding operator by u, and the link overhead incurred in moving data. Represented as:

[0093]

[0094]

[0095] In the formula, cs, ct, cp, cq are the available nodes in the core, cs, ct, cp, cq ∈ {1, 2, ..., size}, size is the total number of fault-free cores, lk ∈ {1, 2, ..., L}, L is the total number of available links; G is the computation graph; for the two pairs of communication tasks (cs, ct) and (cp, cq) generated during the data transfer process, representing the communication from the source node cs to the destination node ct and the communication from the source node cp to the destination node cq respectively, it is necessary to stipulate that the source nodes and destination nodes of the two pairs of communication tasks cannot be the same, therefore, the constraints are cs ≠ ct, cp ≠ cq; Calculate whether both pairs of communication tasks used a certain link lk, variable An integer with a value of 0 or 1, where 1 indicates that link lk is used and 0 indicates that link lk is not used.

[0096] S302, calculate the link contention overhead caused by intra-operator communication in the current operator v.

[0097] The current operator node is denoted by v, representing the link contention overhead caused by intra-operator communication. Represented as:

[0098]

[0099]

[0100] In the formula, cs,ct,cp,cq are the available nodes in the core, cs,ct,cp,cq∈{1,2,...,size}, size is the total number of fault-free cores, lk∈{1,2,...,L}, L is the total number of available links; V represents the set of each operator node in the computation graph; for the two pairs of communication tasks (cs,ct) and (cp,cq) generated when operator node v performs intra-operator communication, it must be stipulated that the source node and destination node of the two pairs of communication tasks cannot be the same, therefore, the constraints are cs≠ct,cp≠cq, cs≠cp,ct≠cq; Calculate whether both pairs of communication tasks used a certain link lk, variable An integer with a value of 0 or 1, where 1 indicates that link lk is used and 0 indicates that link lk is not used.

[0101] S303, calculate the link contention overhead when the current operator v performs weight shifting before calculation.

[0102] The current operator node is represented by v. Due to the link contention overhead during weight shifting before operator computation... Represented as:

[0103]

[0104]

[0105] In the formula, cs,ct,cp,cq are the available nodes in the core, cs,ct,cp,cq∈{1,2,...,size}, size is the total number of fault-free cores, lk∈{1,2,...,L}, L is the total number of available links; V represents the set of each operator node in the computation graph; for the two pairs of communication tasks (cs,ct) and (cp,cq) generated by operator node v when performing weight transfer, it must be stipulated that the source node and destination node of the two pairs of communication tasks cannot be the same, therefore, the constraints are cs≠ct,cp≠cq, cs≠cp,ct≠cq; Calculate whether both pairs of communication tasks used a certain link lk, variable An integer with a value of 0 or 1, where 1 indicates that link lk is used and 0 indicates that link lk is not used.

[0106] S304 uses integer linear programming to minimize the overhead and obtain the optimal mapping scheme of the locally optimized operator.

[0107] The above three parts of link contention overhead and The link contention overhead of the core-fault network is summed and represented as a linear mathematical function R2:

[0108]

[0109] The integer linear programming (ILP solver2) method is used to optimize R2, with the optimization objective being to minimize the execution overhead R2. The solution yields the optimal mapping scheme Map for the operator in the faulty core network. * .

[0110] S4 considers the computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization, and the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization are obtained to form the optimal scheduling scheme.

[0111] Specifically, such as Figure 4 As shown, it includes the following steps:

[0112] S401, Input computation graph G(V,E), set of partitioning strategies for all operators T, set of fault node labels F, mesh of irregular core topology, size of the total number of fault-free cores, RandMap, and MaxIter of the maximum number of iterations.

[0113] S402, calculate the execution overhead R1.

[0114] S403, call the integer linear programming method ILP slover1 to optimize R1, and obtain the optimal partitioning strategy set T for the operator. * and the set size of the optimal resource allocation * .

[0115] S404, input T * and size * Calculate the execution overhead R2.

[0116] S405, call the integer linear programming method ILP slover2 to optimize R², and obtain the optimal mapping scheme Map of the operator in the faulty core network. * .

[0117] S406, Input Map * Repeat steps S402 to S403 to update T. * and size * .

[0118] S407, Repeat steps S404-S405 to update Map. * .

[0119] S408, Repeat steps S406-S407 until the set maximum number of iterations MaxIter is reached, then stop the optimization to obtain the overall optimal scheduling scheme and output the overall optimized T. * size * and Map * .

[0120] In summary, the fault-tolerant deployment optimization method for deep large-scale models oriented towards core particles provided by this invention minimizes overhead when core particles are faulty. This is achieved by comprehensively considering the computational overhead, communication overhead, and link contention overhead of the deep large-scale model computation graph, as well as the link contention overhead in the faulty core particle network. Integer linear programming optimization is used for both local and global iterative optimization, resulting in a scheduling scheme with stronger inference performance and lower execution overhead. This invention not only improves the inference speed and accuracy of deep large-scale models but also reduces their deployment and inference costs, enabling fault-tolerant deployment and optimization of deep learning large-scale models on core particles. This has significant theoretical and practical value, promoting the development and application of fault-tolerant deployment and optimization technologies for deep learning large-scale models on core particles.

[0121] Based on the same inventive concept, embodiments of the present invention also provide a core-oriented deep large model fault-tolerant deployment optimization system 500, such as... Figure 5 As shown, it includes: a computation graph and core structure acquisition module 510, a partitioning strategy and resource allocation optimization module 520, a mapping scheme optimization module 530, and a total execution overhead iterative optimization module 540.

[0122] The computation graph and core structure acquisition module 510 is used to acquire the computation graph of the deep large model, the partitioning strategy of the operators in the computation graph, and the irregular topology of the core under the condition of core failure.

[0123] The partitioning strategy and resource allocation optimization module 520 is used to minimize the computational overhead, communication overhead, and link contention overhead of the deep large model computation graph by optimizing the partitioning strategy of the operator and the hardware resource allocation strategy based on the partitioning strategy, so as to obtain the optimal partitioning strategy and optimal resource allocation quantity of the locally optimized operator.

[0124] The mapping scheme optimization module 530 is used to optimize the hardware mapping strategy of the operator based on the irregular topology of the core particle, minimize the link contention overhead of the core particle fault network, and obtain the best mapping scheme of the locally optimized operator.

[0125] The total execution overhead iterative optimization module 540 is used to regard the computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network of the deep large model computation graph as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization, and finally the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization constitute the optimal scheduling scheme.

[0126] Based on the same inventive concept, this embodiment of the invention also provides a chip-oriented deep large model fault-tolerant deployment optimization device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to implement the above-described chip-oriented deep large model fault-tolerant deployment optimization method when the computer program is executed.

[0127] It should be noted that the chip-oriented deep large model fault-tolerant deployment optimization system, chip-oriented deep large model fault-tolerant deployment optimization device, and chip-oriented deep large model fault-tolerant deployment optimization method provided in the above embodiments belong to the same concept. The specific implementation process is detailed in the embodiments of the chip-oriented deep large model fault-tolerant deployment optimization method, which will not be repeated here.

[0128] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A core particle-oriented deep model fault-tolerant deployment optimization method, characterized in that, Includes the following steps: Obtain the computation graph of the deep large model, the partitioning strategy of the operators in the computation graph, and the irregular topology of the core particles under the condition of core particle failure; By optimizing the partitioning strategy of the operator and the hardware resource allocation strategy based on the partitioning strategy, the computational overhead, communication overhead, and link contention overhead of random mapping in the computational graph of large deep models are minimized. This yields the optimal partitioning strategy and optimal resource allocation for the locally optimized operator, including: get current operator and its predecessor operator all partition strategies, for each partition strategy of a single operator, each partition strategy is represented by a set of integers each value is greater than or equal to 1, each share needs to be allocated a core resource, so the hardware resource quantity allocation strategy for each partition strategy is represented as:​​​​​ ， In the formula, This represents the total number of core resources corresponding to each partitioning strategy. Indicates from The number of divisions in each dimension; The current operator and its preorder operator The chosen partitioning strategy is treated as a variable and modeled as a one-hot encoding. ,in , Represent the current operator respectively and its preorder operator The number of partitioning strategies; Obtain the computational and communication overhead for each operator and each partitioning strategy, where the computational overhead includes intra-operator computational overhead. Communication overhead includes intra-operator communication overhead. and operators Inter-operator communication transmission overhead for each partitioning strategy And the link contention overhead of randomly mapping the computational tasks to the core array after partitioning them according to each partitioning strategy. Reduce the computational overhead within the operator. Intra-operator communication overhead Inter-operator communication transmission overhead Link contention overhead with random mapping The summation is considered as the overall execution overhead of this part, and represented as a linear mathematical function solution R1: R1 = ， In the formula, Represents a computation graph. Indicates transpose; The integer linear programming method is used to optimize R1, with the optimization objective being to minimize the execution cost R1. The solution yields the optimal partitioning strategy for each operator. The set of optimal partitioning strategies for all operators is represented as , And the optimal resource allocation quantity of the core resources that need to be mapped for each operator. The set of optimal resource allocation quantities for all operators is represented as , ; By optimizing the hardware mapping strategy of the operator based on the irregular topology of the core particle, and minimizing the link contention overhead of the core particle faulty network, the optimal mapping scheme of the locally optimized operator is obtained, including: calculating the current operator and from its preceding operator before the computation begins Link overhead caused by data transfer ; Calculate the current operator Link contention overhead caused by intra-operator communication ; Calculate the current operator Link contention overhead during weight shifting before computation The above three parts of link contention overhead , and The link contention overhead of the core-fault network is summed and represented as a linear mathematical function R2: R2 = The integer linear programming method is used to optimize R², with the objective of minimizing the execution overhead R². The solution yields the optimal mapping scheme of the operator in the faulty core network. ; The computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network in the deep large model computation graph are regarded as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization. Finally, the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization constitute the optimal scheduling scheme.

2. The method for fault-tolerant deployment optimization of deep large-scale models oriented towards core particles according to claim 1, characterized in that, The acquisition of the computational graph of the deep large model, the partitioning strategy of the operators in the computational graph, and the irregular topology of the core particles under the condition of core particle failure include: Obtain a directed acyclic graph (DAG) of a large deep model during the inference phase as the computation graph. This directed acyclic graph consists of a set of nodes and a set of directed edges, where This represents the set of each operator node in the computation graph. The set of directed edges representing the data dependencies of each operator in the computation graph; Obtain the set of all partitioning strategies for all operators based on the computation graph. Each operator The set of partitioning strategies is ,in ; Obtain the irregular topology of the core particles: mesh={ },in , Indicates available core nodes. The label indicating the available nodes. The total number is the total number of fault-free cores and is used in conjunction with express.

3. The method for fault-tolerant deployment optimization of large deep models oriented towards core particles according to claim 1, characterized in that, The calculation of the current operator and from its preceding operator before the computation begins Link overhead caused by data transfer ,include: Current operator node Indicates that the preorder operator is used This indicates the link overhead incurred during data transfer. Represented as: ， ， In the formula, { }, This represents the total number of fault-free cores. { }, This represents the total number of available links. For computational graphs; for those generated during data transfer. and Communication tasks, respectively representing the communication tasks from the source node Communication and from source node For communication to work, it must be stipulated that the source nodes and destination nodes of the two pairs of communication tasks cannot be the same, therefore, there are constraints. ; Calculate whether both pairs of communication tasks used a certain link. ,variable The value can be an integer of 0 or 1, where 1 indicates the use of a link. 0 indicates that the link is not used. .

4. The method for fault-tolerant deployment optimization of large deep models oriented towards core particles according to claim 1, characterized in that, The calculation of the current operator Link contention overhead caused by intra-operator communication ,include: Current operator node This indicates the link contention overhead caused by intra-operator communication. Represented as: ， ， In the formula, { }, This represents the total number of fault-free cores. { }, This represents the total number of available links. This represents the set of each operator node in the computation graph; for each operator node... Generated during operator intra-communication and For communication tasks, it must be stipulated that the source and destination nodes of the two pairs of communication tasks cannot be the same, therefore, there is a constraint. ; Calculate whether both pairs of communication tasks used a certain link. ,variable The value can be an integer of 0 or 1, where 1 indicates the use of a link. 0 indicates that the link is not used. .

5. The method for fault-tolerant deployment optimization of large deep models oriented towards core particles according to claim 1, characterized in that, The calculation of the current operator Link contention overhead during weight shifting before computation ,include: Current operator node This indicates that the link contention overhead occurs during weight shifting before computation. Represented as: ， ， In the formula, { }, This represents the total number of fault-free cores. { }, This represents the total number of available links. This represents the set of each operator node in the computation graph; for each operator node... Generated during weight transfer and For communication tasks, it must be stipulated that the source and destination nodes of the two pairs of communication tasks cannot be the same, therefore, there is a constraint. ; Calculate whether both pairs of communication tasks used a certain link. ,variable The value can be an integer of 0 or 1, where 1 indicates the use of a link. 0 indicates that the link is not used. .

6. The method for fault-tolerant deployment optimization of large deep models oriented towards core particles according to claim 1, characterized in that, The computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network in the deep large model computation graph are considered as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization, ultimately obtaining the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization, constituting the optimal scheduling scheme, including: The computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core-particle fault network are considered as the total execution overhead of the scheduling task. The goal of iterative optimization is to minimize the total execution overhead of the deep large model computation graph in the faulty core-particle topology. A maximum number of iterations, MaxIter, is set, and optimization stops when it is reached. This yields the optimal scheduling scheme, which includes the optimal partitioning strategy, the optimal number of resource allocations, and the optimal mapping scheme of the overall optimized operators.

7. A chip-oriented fault-tolerant deployment optimization system for large deep models, used to implement the chip-oriented fault-tolerant deployment optimization method for large deep models according to any one of claims 1 to 6, characterized in that, include: The module includes a computation graph and core structure acquisition module, a partitioning strategy and resource allocation optimization module, a mapping scheme optimization module, and a total execution overhead iteration optimization module. The computation graph and core structure acquisition module is used to acquire the computation graph of a large-depth model, the partitioning strategy of operators in the computation graph, and the irregular topology of cores under core failure conditions. The partitioning strategy and resource allocation optimization module is used to minimize the computational overhead, communication overhead, and link contention overhead of the deep large model computation graph by optimizing the partitioning strategy of the operator and the hardware resource quantity allocation strategy based on the partitioning strategy, so as to obtain the optimal partitioning strategy and optimal resource allocation quantity of the locally optimized operator. The mapping scheme optimization module is used to optimize the hardware mapping strategy of the operator based on the irregular topology of the core particle, minimize the link contention overhead of the core particle fault network, and obtain the best mapping scheme of the locally optimized operator. The total execution overhead iterative optimization module is used to regard the computational overhead, communication overhead, link contention overhead of random mapping, and link contention overhead of the core fault network of the deep large model computation graph as the total execution overhead of the scheduling task. The total execution overhead is minimized through iterative optimization, and finally the optimal partitioning strategy, optimal resource allocation quantity, and optimal mapping scheme after overall optimization constitute the optimal scheduling scheme.

8. A fault-tolerant deployment optimization device for deep large-scale models oriented towards core particles, comprising a memory and a processor, wherein the memory is used to store computer programs, characterized in that, The processor is configured to implement the chip-oriented deep large model fault-tolerant deployment optimization method according to any one of claims 1-6 when executing the computer program.

Images