Parallel strategy search method for efficient training of artificial intelligence large model

By building a bilayer policy network and distributed training performance evaluation model, combining importance sampling and stochastic gradient rise algorithm, parallel strategies are optimized, and the problem of low efficiency of parallel strategy search and tuning in the existing technology is solved, and the problem of low efficiency of parallel strategy search and tuning is realized. More efficient parallel training performance.

CN118966321AInactive Publication Date: 2024-11-15CHENGDU BITNUM TECH +3

Patent Information

Application Number
CN202411442290.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-16
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing distributed parallel strategy automatic search and tuning technology based on machine learning has limited performance improvement in training time and communication time, and the search and tuning process generates huge extra overhead, which cannot offset the benefits of parallel training performance optimization.

Method used

The TensorFlow framework is used to build a two-layer policy network for segmentation and scheduling, and predict the calculation and communication overhead through the simulator. The reward value is calculated using the distributed training performance evaluation model. The parallel strategy is optimized by combining importance sampling and stochastic gradient rise algorithm to generate the most training performance. Excellent parallel strategy.

Benefits of technology

More accurately evaluates the comprehensive performance of parallel strategies, reduces overhead during the search process, avoids memory overflow, and maintains steady convergence when parameters change, improving the efficiency and performance of parallel training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966321A_ABST
    Figure CN118966321A_ABST
Patent Text Reader

Abstract

The invention discloses a parallel strategy search method for efficient training of an artificial intelligence large model, and relates to the technical field of artificial intelligence, and the method specifically comprises the following steps: S1, inputting the artificial intelligence large model; s2, visualizing the artificial intelligence large model by utilizing a TensorFlow framework, generating a calculation graph g of the large model, and obtaining a topological graph D of cluster equipment; according to the parallel strategy search method oriented to efficient training of the artificial intelligence large model, the comprehensive performance of a parallel strategy is evaluated more accurately by mining calculation and communication characteristics in the parallel training process of a neural network model and establishing a distributed training performance evaluation model, and an intelligent agent is constructed by adopting a double-layer strategy network, so that the parallel strategy search efficiency is improved. The overhead in the search process is reduced, memory overflow is prevented, and the stochastic gradient rising algorithm is adopted to facilitate the algorithm to keep stable convergence when the change of parameters possibly leads to severe change of probability distribution of new and old strategies.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a parallel strategy search method for efficient training of large artificial intelligence models. Background Art

[0002] In recent years, the number of parameters in large AI models has increased rapidly, and they have achieved the best results in most tasks in various fields such as computer vision and natural language processing. In contrast, the development of computing hardware is very slow. Therefore, it is necessary to use a single machine with multiple cards or a cluster for parallel training.

[0003] A parallel strategy is a computing strategy used to process multiple computing tasks or data sets simultaneously on multiple processors or computers. The main purpose of this strategy is to accelerate processing speed and improve efficiency. In different fields and applications, parallel strategies have many forms, including data parallelism, model parallelism, pipeline parallelism, and hybrid parallelism.

[0004] Existing distributed parallel strategy automatic search and tuning technologies based on machine learning usually use training time or communication time as a single performance evaluation indicator to implement distributed parallel strategy search and tuning under specific deep learning models, thus limiting the overall training performance improvement. In addition, the parallel strategy search and tuning process itself generates huge additional overhead, which cannot even offset the performance optimization benefits brought by parallel training. Summary of the invention

[0005] The purpose of the present invention is to provide a parallel strategy search method for efficient training of large artificial intelligence models to solve the above-mentioned deficiencies in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a parallel strategy search method for efficient training of large artificial intelligence models, specifically comprising the following steps: S1. Input the artificial intelligence big model; S2. Use the TensorFlow framework to visualize the artificial intelligence big model, generate the calculation graph g of the big model, obtain the topology graph D of the cluster device, and build a two-layer strategy network for segmentation and scheduling based on the calculation graph g of the big model and the topology graph D of the cluster device; S3. The execution simulator simulates the execution of the complete split scheduling strategy according to the cluster device topology diagram D, predicts the computing cost f(.), communication and memory cost g(.), and calculates the reward value according to the distributed training performance evaluation model. , and then execute the simulator to transfer the reward value R to the agent; S4. The intelligent agent receives the reward value R signal, and based on the importance sampling principle, uses the stochastic gradient ascent algorithm to implement iterative optimization of the policy network, and outputs the parallel strategy with the best training performance of the large model composed of one or more parallel strategies under the current system.

[0007] Furthermore, the large model computation graph g in S2 is specifically g(O, E), where vertex O represents the large model network operator to be trained, denoted as O={O 1 , O 2 , ..., O N}, E is the directed edge connecting vertex O, representing the communication data between computing nodes, denoted as E={e 11 , e 12 ,……,e ij}: The cluster device topology D is obtained by abstracting the cluster computing devices, which consists of M available devices {d 1 , d2, ..., d M}composition.

[0008] Furthermore, in S2, the splitting and scheduling two-layer strategy network is introduced into the shared layer to form an intelligent agent.

[0009] Furthermore, the execution simulator described in S3 is used to simulate the GPU operator calculation and communication process in the form of a queue on a single node.

[0010] Furthermore, the distributed training performance evaluation model described in S3 includes a computational cost model, a communication cost model, and a memory cost model; The computational cost model is used to measure the i-th device d i The computational load during training is expressed as , where N is the total number of tensors involved in the calculation, Respectively represent the size of the kth dimension of the current tensor, C i For the i-th device d i The floating-point operands involved in the operation, c i is the computing density of the i-th device; The communication cost model is used to measure the device d i and d j The communication load between during the training process is expressed as , where N is the total number of tensors involved in the communication, T n Indicates that in device d i , d j The tensor transmitted between them, A is the scale function of the calculated tensor size, Indicates the communication bandwidth between device i and device j; The memory cost model is used to measure the memory cost of the i-th device d iThe memory load during training is expressed as , where N represents the storage on the current device d i The total number of tensors on T n Represents the current tensor, and A is the function for calculating the tensor size scale.

[0011] Furthermore, the reward value in S3 is constructed according to the computational cost model, the communication cost model and the memory cost model, and can be expressed as ,in It is represented as the kth parallel strategy, α and β are weight parameters, E i Indicates the computational load during large model training. Indicates the communication load during large model training, M i Indicates the memory load during large model training.

[0012] Furthermore, S4 specifically includes the following steps: make and Respectively represent the segmentation and scheduling strategies, and the corresponding network parameters are Represents the probability distribution of generating parallel strategies ; After further separation of the segmentation and scheduling strategy network parameters, it can be refined into the objective function , the TF framework defines the objective function and Finding partial derivatives and ; Finally, the stochastic gradient ascent algorithm is used to iteratively optimize the network parameters of the segmentation and scheduling strategies, and return the parallel strategy with the best training performance and the corresponding reward value R.

[0013] Furthermore, the importance sampling principle in S4 is used to solve the problem of The severe jitter of the extraction strategy causes low sample efficiency and large variance between samples.

[0014] Furthermore, the stochastic gradient ascent algorithm is specifically: Enter the available device serial group , split and schedule network initialization parameters ; Set the minimum reward min to infinity and the reward value R to 0; The algorithm loops from i=1 to N; In each loop, define a Includes several subgroups ,Will Add to a list group; Utilize to pair group with to obtain ; Apply the pairing result to the evaluation model to calculate the reward value R i ; If R i < min, then update the optimal policy Π * to , where min is the current R i , parameter is the original plus , parameter is the original plus ; If R i = min, then break out of the loop and output the current R i and the corresponding optimal policy Π * ; If the algorithm loops from i = 1 to N and still there is no R i = min, then output R N and the corresponding optimal policy Π * .

[0015] Compared with the prior art, a parallel policy search method for efficient training of artificial intelligence large models provided by the present invention has the following beneficial effects: 1. By mining the computing and communication characteristics in the parallel training process of the neural network model, a distributed training performance evaluation model is established to more accurately evaluate the comprehensive performance of the parallel policy; 2. By adopting a double-layer policy network to construct an intelligent agent, the overhead in the search process is reduced, and memory overflow is prevented; 3. By adopting the stochastic gradient ascent algorithm, it is beneficial for the algorithm to maintain stable convergence when the剧烈变化 of the probability distribution of the old and new policies may be caused by parameter changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0017] Figure 1 is the overall flowchart provided by the embodiment of the present invention; Figure 2 is the flowchart of the stochastic gradient ascent algorithm provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0019] See also Figure 1-Figure 2 , a parallel strategy search method for efficient training of large artificial intelligence models, including the following steps: S1. Input the artificial intelligence large model, which is the BERT-Large model; S2. Use the TensorFlow framework to visualize the artificial intelligence big model, generate the calculation graph g of the big model, obtain the topology graph D of the cluster device, and build a two-layer strategy network for segmentation and scheduling based on the big model calculation graph g and the cluster device topology graph D. The two-layer strategy network architecture of the intelligent agent is the core of this method. S3. The execution simulator simulates the execution of the complete split scheduling strategy according to the cluster device topology diagram D, predicts the computing cost f(.), communication and memory cost g(.), and calculates the reward value according to the distributed training performance evaluation model. , and then execute the simulator to transmit the reward value R to the agent. The setting of the reward value can affect the agent's trade-off between exploring new strategies and using known strategies, thereby avoiding falling into a local optimal solution; S4. The intelligent agent receives the reward value R signal, and based on the importance sampling principle, uses the stochastic gradient ascent algorithm to implement iterative optimization of the policy network, and outputs the parallel strategy with the best training performance of the large model composed of one or more parallel strategies under the current system.

[0020] In this example, the hardware environment is a cluster equipped with a high-performance GPU accelerator, and the software environment uses the TensorFlow open source framework and related dependency packages.

[0021] The large model computation graph g in S2 is specifically g(O, E), where vertex O represents the large model network operator to be trained, denoted as O={O 1 , O 2 , ..., O N}, E is the directed edge connecting vertex O, representing the communication data between computing nodes, denoted as E={e 11 , e 12 ,……,e ij}: The cluster device topology D is obtained by abstracting the cluster computing devices, which consists of M available devices {d 1 , d2, ..., d M}composition.

[0022] In S2, the splitting and scheduling two-layer strategy network is introduced into the shared layer and constitutes an intelligent agent. The splitting and scheduling two-layer strategy network is designed to effectively manage and optimize the use of network resources to improve network performance, reliability and resource utilization.

[0023] The execution simulator in S3 is used to simulate the GPU operator calculation and communication process in the form of queues on a single node.

[0024] The distributed training performance evaluation model in S3 includes the computational cost model, the communication cost model, and the memory cost model; The computational cost model is used to measure the i-th device d i The computational load during training is expressed as , where N is the total number of tensors involved in the calculation, Respectively represent the size of the kth dimension of the current tensor, C i For the i-th device d i The floating-point operands involved in the operation, c i is the computing density of the i-th device; The communication cost model is used to measure the device d i and d j The communication load between during the training process is expressed as , where N is the total number of tensors involved in the communication, T n Indicates that in device d i , d j The tensor transmitted between them, A is the scale function of the calculated tensor size, Indicates the communication bandwidth between device i and device j; The memory cost model is used to measure the memory cost of the i-th device d i The memory load during training is expressed as , where N represents the storage on the current device d i The total number of tensors on T n Represents the current tensor, and A is the function for calculating the tensor size scale.

[0025] The reward value in S3 is constructed based on the computational cost model, communication cost model, and memory cost model, which can be expressed as ,in It is represented as the kth parallel strategy, α and β are weight parameters, E i Indicates the computational load during large model training. Indicates the communication load during large model training, M i It indicates the memory load during the training of large models. The distributed training performance evaluation model uses multiple indicators as performance evaluation criteria to measure the comprehensive performance of parallel strategies from multiple angles.

[0026] S4 specifically includes the following steps: Let and represent the splitting and scheduling strategies respectively, and the corresponding network parameters are represented by to generate the probability distribution of the parallel strategy ; After further separating the network parameters of the splitting and scheduling strategies, it can be refined into the objective function , and the TF framework calculates the partial derivatives of the objective function with respect to and to obtain and ; Finally, use the stochastic gradient ascent algorithm to achieve iterative optimization of the network parameters of the splitting and scheduling strategies, and return the parallel strategy with the optimal training performance and the corresponding reward value R.

[0027] The importance sampling principle in S4 is used to solve the problems of low efficiency of sampling the extraction strategy and large variance between samples caused by the drastic jitter caused by parameter changes, and can use the existing data for efficient estimation without changing the sample distribution.

[0028] The stochastic gradient ascent algorithm is specifically as follows: Input the available device sequence group , and the initial parameters of the splitting and scheduling network ; Set the minimum reward min to infinity and the reward value R to 0; The algorithm loops from i = 1 to N; In each loop, define a including several subgroups , and add to a list group; Use to pair group with to obtain ; Apply the pairing result to the evaluation model to calculate the reward value R i ; If R i < min, then update the optimal strategy Π * to , min to the current R i , the parameter is the original plus , and the parameter is the original plus ; If R​i =min, then jump out of the loop and output the current R i and the corresponding optimal strategy Π * ; If the algorithm ends the cycle from i=1 to N, there is still no R i =min, then output R N and the corresponding optimal strategy Π * ,The stochastic gradient ascent algorithm helps the algorithm maintain smooth convergence when parameter changes may lead to drastic changes in the probability distribution of the new and old strategies.

[0029] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A parallel strategy search method for efficient training of large artificial intelligence models, characterized in that: The specific steps include: S1. Input the artificial intelligence big model; S2. Use the TensorFlow framework to visualize the artificial intelligence big model, generate the calculation graph g of the big model, obtain the topology graph D of the cluster device, and build a two-layer strategy network for segmentation and scheduling based on the calculation graph g of the big model and the topology graph D of the cluster device; S3. The execution simulator simulates the execution of the complete split scheduling strategy according to the cluster device topology diagram D, predicts the computing cost f(.), communication and memory cost g(.), and calculates the reward value according to the distributed training performance evaluation model. , and then execute the simulator to transfer the reward value R to the agent; S4. The intelligent agent receives the reward value R signal, and based on the importance sampling principle, uses the stochastic gradient ascent algorithm to implement iterative optimization of the policy network, and outputs the parallel strategy with the best training performance of the large model composed of one or more parallel strategies under the current system.

2. According to claim 1, a parallel strategy search method for efficient training of large artificial intelligence models is characterized in that: The large model calculation graph g in S2 is specifically g(O, E), where vertex O represents the large model network operator to be trained, denoted as O={O1, O2, ..., O N }, E is the directed edge connecting vertex O, representing the communication data between computing nodes, denoted as E={e 11 , e 12 ,……,e ij }; The cluster device topology D is obtained by abstracting the cluster computing devices, and is composed of M available devices {d1, d2, ..., d M }composition.

3. A parallel strategy search method for efficient training of large artificial intelligence models according to claim 1, characterized in that: In S2, the splitting and scheduling two-layer strategy network is introduced into the shared layer to form an intelligent agent.

4. The parallel strategy search method for efficient training of large artificial intelligence models according to claim 1 is characterized in that: The execution simulator described in S3 is used to simulate the GPU operator calculation and communication process in the form of a queue on a single node.

5. The parallel strategy search method for efficient training of large artificial intelligence models according to claim 1 is characterized in that: The distributed training performance evaluation model described in S3 includes a computational cost model, a communication cost model, and a memory cost model; The computational cost model is used to measure the i-th device d i The computational load during training is expressed as , where N is the total number of tensors involved in the calculation, Respectively represent the size of the kth dimension of the current tensor, C i For the i-th device d i The floating-point operands involved in the operation, c i is the computing density of the i-th device; The communication cost model is used to measure the device d i and d j The communication load between during the training process is expressed as , where N is the total number of tensors involved in the communication, T n Indicates that in device d i , d j The tensor transmitted between them, A is the scale function of the calculated tensor size, Indicates the communication bandwidth between device i and device j; The memory cost model is used to measure the memory cost of the i-th device d i The memory load during training is expressed as , where N represents the storage in the current device d i The total number of tensors on T n Represents the current tensor, and A is the function for calculating the tensor size scale.

6. The parallel strategy search method for efficient training of large artificial intelligence models according to claim 1 is characterized in that: The reward value described in S3 is constructed based on the computational cost model, the communication cost model, and the memory cost model, which can be expressed as ,in It is represented as the kth parallel strategy, α and β are weight parameters, E i Indicates the computational load during large model training. Indicates the communication load during the large model training process, M i Indicates the memory load during large model training.

7. A parallel strategy search method for efficient training of large artificial intelligence models according to claim 1, characterized in that: S4 specifically includes the following steps: make and Respectively represent the segmentation and scheduling strategies, and the corresponding network parameters are Represents the probability distribution of generating parallel strategies ; After further separation of the segmentation and scheduling strategy network parameters, it can be refined into the objective function , the TF framework defines the objective function and Finding partial derivatives and ; Finally, the stochastic gradient ascent algorithm is used to iteratively optimize the network parameters of the segmentation and scheduling strategies, and return the parallel strategy with the best training performance and the corresponding reward value R.

8. The parallel strategy search method for efficient training of large artificial intelligence models according to claim 1 is characterized in that: The importance sampling principle in S4 is used to solve the problem of parameter changes. The severe jitter of the extraction strategy causes low sample efficiency and large variance between samples.

9. A parallel strategy search method for efficient training of large artificial intelligence models according to claim 7, characterized in that: The stochastic gradient ascent algorithm is specifically: Enter the available device sequence groups , split and schedule network initialization parameters ; Set the minimum reward min to infinity and the reward value R to 0; The algorithm loops from i=1 to N; In each loop, define a Includes several subgroups ,Will Add to a list group; use Group and Pairing ; The pairing results Apply to the evaluation model to calculate the reward value R i ; If R i < min, then update the optimal policy Π * to , where min is the current R i , and the parameter is the original plus , and the parameter is the original plus ; If R i =min, then jump out of the loop and output the current R i and the corresponding optimal strategy Π * ; If the algorithm ends the cycle from i=1 to N, there is still no R i =min, then output R N and the corresponding optimal strategy Π * .

Citation Information

Patent Citations

  • Adaptive distributed parallel training method for neural network based on reinforcement learning

    CN113128702A

  • Multi-agent reinforcement learning acceleration method for cluster unmanned aerial vehicle decision

    CN115454646A

  • Elastic training of machine learning models via re-partitioning based on feedback from the training algorithm

    US20200356893A1

Cited By

  • Large model computing cluster service reliability evaluation and optimization method and device

    CN119806973A

  • Parallel strategy search method for efficient training of artificial intelligence large model

    CN120012879A

  • Automatic tuning method and device for distributed parallel strategy, medium and program product

    CN120449991A

  • Parallel strategy generation method, model training method and equipment

    CN121599162A