Distributed gradient compression training method and device, equipment and storage medium

Abstract

The application discloses a distributed gradient compression training method and device, equipment and a medium. The method comprises the following steps: when it is determined that a policy matrix starting learning condition is met, controlling each work node in a distributed cluster to perform first stage gradient compression training; in each round of training in the first stage gradient compression training process, dynamically updating a policy matrix according to a loss value and a training time difference of each round compared with at least one previous round; when the first stage gradient compression training is completed, controlling each work node in the distributed cluster to perform second stage gradient compression training; in each round of training in the second stage gradient compression training process, querying the updated policy matrix according to a training state corresponding to each round, and determining a gradient compression ratio used by each work node in each round. The technical scheme of the embodiment of the application realizes dynamic balance of convergence accuracy and convergence speed in the distributed gradient compression training process.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to distributed training technology of machine learning models, and particularly relate to a distributed gradient compression training method and device, equipment and a storage medium. BACKGROUND

[0002] With the increasing size of deep learning models, the complete model training process is mainly implemented using a distributed training manner. In the distributed training process, in order to reduce the collective communication amount between the worker nodes, a distributed training method based on a gradient compression method is proposed.

[0003] At present, a commonly used gradient compression method is gradient sparsification, that is, a small part of gradients participating in parameter updating is determined by an algorithm, and most of the gradients with small changes are temporarily updated by reducing the number of transmitted gradients to achieve gradient compression. The top-k random sampling method, as a kind of gradient sparsification, obtains the gradients of the top k percentage points in all gradients in a manner of random sampling in the model parameters, and each worker node approximately obtains the gradients of the top k percentage points in all gradients in descending order of gradient values, and performs parameter updating through collective communication.

[0004] The inventor found in the process of implementing the present application that the prior art has the following defects: Generally, the gradient compression ratio k is determined by human experience, and the k value is generally fixed during the entire distributed training process. This way of determining the k value will affect the convergence accuracy of the deep learning model. SUMMARY

[0005] Embodiments of the present application provide a distributed gradient compression training method, device, equipment and storage medium to dynamically determine the gradient compression ratio of each round according to the actual training state in the distributed training process.

[0006] In a first aspect, embodiments of the present application provide a distributed gradient compression training method, executed by a control node in a distributed cluster, comprising:

[0007] When it is determined that the learning condition of the strategy matrix is met, the first phase gradient compression training of each worker node in the distributed cluster is controlled;

[0008] In each round of training in the first phase gradient compression training process, the strategy matrix is dynamically updated according to the loss value and the training time difference of each round compared with at least one previous round;

[0009] The strategy matrix is used to describe the matrix score value of different gradient compression ratios under each training state, and the matrix score value is directly proportional to the distributed training performance.

[0010] When the first-stage gradient compression training is completed, the second-stage gradient compression training of each worker node in the distributed cluster is controlled to be executed;

[0011] In each round of training in the second-stage gradient compression training process, the strategy matrix updated according to the training state corresponding to each round is queried to determine the gradient compression ratio used by each worker node in each round.

[0012] In a second aspect, the embodiments of the present application also provide a distributed gradient compression training device, which is executed by a control node in a distributed cluster, and the device comprises:

[0013] The first-stage compression training module is configured to control each worker node in the distributed cluster to execute the first-stage gradient compression training when it is determined that the strategy matrix starting learning condition is met;

[0014] The strategy matrix updating module is configured to dynamically update the strategy matrix according to the loss value and the training time difference of each round compared with at least one previous round in each round of training in the first-stage gradient compression training process;

[0015] The strategy matrix is used to describe the matrix score value under different gradient compression ratios in each training state, and the matrix score value is directly proportional to the distributed training performance;

[0016] The second-stage compression training module is configured to control each worker node in the distributed cluster to execute the second-stage gradient compression training when the first-stage gradient compression training is completed;

[0017] The gradient compression ratio determining module is configured to query the updated strategy matrix according to the training state corresponding to each round in each round of training in the second-stage gradient compression training process to determine the gradient compression ratio used by each worker node in each round.

[0018] In a third aspect, the embodiments of the present application also provide a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to realize the distributed gradient compression training method according to any embodiment of the present application.

[0019] In a fourth aspect, the embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the distributed gradient compression training method according to any embodiment of the present application.

[0020] The technical scheme of the embodiment of the present application is that in each round of training in the first-stage gradient compression training process, the loss value and the training time difference of each round compared with at least one previous round are used to dynamically update the strategy matrix, and in each round of training in the second-stage gradient compression training process, the strategy matrix is queried according to the training state corresponding to each round to determine the gradient compression ratio used by each working node in each round. Based on the traditional top-k random sampling method, the strategy matrix for describing the distributed training performance under different gradient compression ratios under different training states is generated based on the dynamic programming method, the problem of poor convergence accuracy of the deep learning model caused by using a fixed k value or selecting a k value by human experience is solved, and the dynamic balance of convergence accuracy and convergence speed is achieved in the whole distributed gradient compression training process. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1a is a structural schematic diagram of a distributed training process of each working node in a distributed cluster suitable for the technical scheme of the embodiment of the present application;

[0022] Figure 1b is a schematic diagram of a global reduction operation of a compressed gradient using top-k suitable for the technical scheme of the embodiment of the present application;

[0023] Figure 1c is a flowchart of a distributed gradient compression training method in the first embodiment of the present application;

[0024] Figure 2a is a flowchart of a distributed gradient compression training method in the second embodiment of the present application;

[0025] Figure 2b is a flowchart of a method for calculating a reward score value and a next round training state suitable for the second embodiment of the present application;

[0026] Figure 3a is a flowchart of a distributed gradient compression training method in the third embodiment of the present application;

[0027] Figure 3b is a flowchart of a specific application scenario suitable for the technical scheme of the embodiment of the present application;

[0028] Figure 4 is a structural diagram of a distributed gradient compression training device in the fourth embodiment of the present application;

[0029] Figure 5 is a structural diagram of a computer device in the fifth embodiment of the present application. DETAILED DESCRIPTION

[0030] The application will be described in further detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the application and not to limit the application. In addition, it should be noted that, for the sake of description, only the parts related to the application are shown in the drawings and not all the structures.

[0031] To facilitate the description hereinafter, first, the principle of implementing the distributed gradient compression training using the distributed cluster will be simply described. In the principle, Figure 1a a structural schematic diagram of the distributed training process of each work node in the distributed cluster is shown.

[0032] As shown in Figure 1a , the distributed cluster 100 includes five work nodes, which are the first work node 100-1, the second work node 100-2, the third work node 100-3, the fourth work node 100-4 and the fifth work node 100-5. The above five work nodes perform three rounds of distributed training, that is, the first training, the second training and the third training. In each round of distributed training, each work node has four stages of tasks, which are loading data, gradient calculation, gradient synchronization and updating parameters. In the data loading stage, each work node obtains sample data to be trained, in the gradient calculation stage, each work node calculates a plurality of model gradients corresponding to the sample data, in the gradient synchronization stage,

[0033] After the collection communication of each work node, the plurality of model gradients calculated by the plurality of work nodes are subjected to an all-reduce operation, and each work node obtains the same mean value of the model gradient, and updates the local model parameter to complete one round of distributed training.

[0034] The gradient compression operation described in the background art occurs in the gradient synchronization stage in each round of distributed training. At this time, instead of performing all-reduce on all the model gradients calculated locally by each work node, only the top-k model gradients are transmitted to reduce the communication data volume.

[0035] Correspondingly, Figure 1b a structural schematic diagram of the gradient all-reduce operation using the top-k compressed gradient suitable for the technical scheme of the embodiment of the application is shown. As Figure 1bAs shown, after each worker node completes the calculation of the gradients of each model, it selects top-k compressed gradients to perform a global reduction operation through collective communication. Among them, top-k can be understood as the top k percentage, and k can be understood as the compression ratio, such as 1% or 2%, etc. For example, if the total number of model gradients calculated by each worker node is 1000, and top-k is 1%, only the top 10 model gradients need to be obtained for transmission.

[0036] It should be noted that when the value of k in top-k is uniquely determined, the number of model gradients sent by each worker node can be the same or different, and the specific number is associated with the selection algorithm of the model gradient. In an optional implementation, the worker node can perform full ordering on all the calculated model gradients in descending order, and then obtain the top 10 to perform collective communication. In this case, the number of model gradients sent by each worker node is the same, but the amount of calculation is large. In another optional implementation, each worker node can obtain a small sample set of model gradients, for example, 200, and obtain 1% of the model gradients in it, for example, the top 2, and the smaller one is used as a threshold. In all model gradients, all model gradients greater than or equal to the threshold are used as top-k model gradients for collective communication. In this case, the number of model gradients sent by each worker node is not fixed, but the amount of calculation is less.

[0037] In the prior art, a fixed k value is used in each round of distributed training process, and the k value cannot be dynamically updated according to the actual situation of each training round. In contrast, the scheme of the embodiments of the present application divides the entire distributed training process into multiple stages. First, a strategy matrix for describing the optimal k value under different training states is obtained in a stage, and then in each round of distributed training process in the next stage, the optimal k value in the training round can be dynamically selected according to the different training states of different training rounds, so as to achieve a dynamic balance between convergence accuracy and convergence speed.

[0038] Embodiment one

[0039] Figure 1c A flowchart of a distributed gradient compression training method provided by embodiment one of the present application. This embodiment can be applied to dynamically determine the gradient compression ratio in the distributed gradient compression training process. The method can be executed by a distributed gradient compression training device, which can be realized by software and / or hardware, and generally can be integrated in the control node of the distributed cluster. Accordingly, the method specifically includes the following steps:

[0040] S110, when it is determined that the policy matrix start learning condition is met, controlling each worker node in the distributed cluster to perform first-stage gradient compression training.

[0041] In this embodiment, the policy matrix start learning condition can be understood as the timing at which the policy matrix learning can be started. Generally, the first few rounds of training of the distributed training will cause the calculated loss function to fluctuate abnormally, and the effect of gradient compression training is not good if these rounds of training are performed. Correspondingly, the machine learning model to be trained can be first subjected to preliminary gradient-free compression training, so that the machine learning model has a preliminary accurate output result, and then the learning of the policy matrix is started.

[0042] Correspondingly, in an optional implementation of this embodiment, determining that the policy matrix start learning condition is met can include:

[0043] When the gradient-free compression training of the first iteration round number is performed by each worker node in the distributed cluster, it is determined that the policy matrix start learning condition is met.

[0044] Specifically, referring to Figure 1a The gradient-free compression training can be understood as performing global reduction operation on all model gradients calculated by each worker node through collective communication. The first iteration round number can be preset according to actual conditions, for example, 5, 10 or 15, etc., and this embodiment does not limit this.

[0045] The purpose of controlling each worker node in the distributed cluster to perform first-stage gradient compression training is to learn a policy matrix matching the current machine learning model. The policy matrix is used to determine the gradient compression ratio selected by each worker node in each round in the subsequent distributed model training process.

[0046] S120, in each round of training in the first-stage gradient compression training process, dynamically updating the policy matrix according to the loss value and training time difference of each round compared with at least one previous round.

[0047] The policy matrix is used to describe the matrix score value under different gradient compression ratios in each training state, and the matrix score value is proportional to the distributed training performance.

[0048] In this embodiment, a policy matrix for dynamically determining the gradient compression ratio is defined, and the form of the policy matrix Q[S,K] can be as shown in Table 1.

[0049] Table 1

[0050] Q[S,K] K[1]＝0.1 K[2]＝1％ K[3]＝2％ K[4]＝4％ K[5]＝8％ S[1] Q＝0 Q＝0 Q＝0 Q＝0 Q＝0 S[2] Q＝0 Q＝0 Q＝0 Q＝0 Q＝0 S[3] Q＝0 Q＝0 Q＝0 Q＝0 Q＝0 S[4] Q＝0 Q＝0 Q＝0 Q＝0 Q＝0 S[5] Q＝0 Q＝0 Q＝0 Q＝0 Q＝0

[0051] As shown in Table 1, the strategy matrix describes the matrix score value Q under different training states S and different gradient compression ratios K. The training state can be calculated by the loss value difference of each round compared with at least one previous round, and the gradient compression ratio is the k value in the top-k strategy, that is, the gradient that needs to be processed by global reduction through collective communication accounts for the top k percentage of all gradients. The matrix score value is used to reflect the distributed training performance, and the higher the matrix score value, the better the corresponding distributed training performance. The matrix score value can be calculated by the loss value and training time difference of each round compared with at least one previous round.

[0052] S130, when the first phase gradient compression training is completed, each worker node in the distributed cluster is controlled to perform the second phase gradient compression training.

[0053] In this embodiment, the completion condition of the first phase compression training can be predetermined, for example, the round condition, or the update rate condition of the strategy matrix, etc., and the present embodiment does not limit this.

[0054] After the completion of the first phase gradient compression training, it is determined that the strategy matrix has been updated, and then the strategy matrix can be used to dynamically determine the gradient compression ratio used in each round in the subsequent distributed model training process.

[0055] S140, in each round of the second phase gradient compression training process, the strategy matrix updated is queried according to the training state corresponding to each round to determine the gradient compression ratio used by each worker node in each round.

[0056] As described above, the training state of each round can be calculated by the loss value and training time difference of each round compared with at least one previous round. Through the training state, a matrix row can be located in the strategy matrix. Then, the gradient compression ratio corresponding to the maximum matrix score value can be selected in the matrix row as the gradient compression ratio used in the current training round to achieve the best training performance, that is, to meet the dynamic balance of convergence speed and convergence accuracy.

[0057] The technical scheme of the embodiment of the present application dynamically updates the strategy matrix according to the loss value and the training time difference of each round compared with at least one previous round in each training round of the first-stage gradient compression training process, and then queries the updated strategy matrix according to the training state corresponding to each round in each training round of the second-stage gradient compression training process, to determine the gradient compression ratio used by each working node in each round, thereby generating a strategy matrix for describing the distributed training performance under different gradient compression ratios under different training states based on the dynamic programming method on the basis of the traditional top-k random sampling method, solving the problem of poor convergence accuracy of the deep learning model caused by using a fixed k value or selecting a k value by human experience, and achieving dynamic balance between convergence accuracy and convergence speed in the entire distributed gradient compression training process.

[0058] Embodiment Two

[0059] Figure 2a A flowchart of a distributed gradient compression training method in the second embodiment of the present application is shown in FIG. 2. This embodiment is refined based on the above-mentioned embodiment. In this embodiment, the specific process of dynamically updating the strategy matrix according to the loss value and the training time difference of each round compared with at least one previous round in each training round of the first-stage gradient compression training process is completely refined.

[0060] Correspondingly, the method specifically includes the following operations:

[0061] S210, when it is determined that the strategy matrix start learning condition is met, controlling each working node in the distributed cluster to perform first-stage gradient compression training.

[0062] The first-stage gradient compression training includes gradient compression training of the second iteration round number. The second iteration round number can be preset according to actual conditions, for example, can be 500 times, 1000 times or 5000 times, etc., and the present embodiment does not limit this.

[0063] S220, acquiring a first current round in the first-stage gradient compression training process.

[0064] In this embodiment, the learning process of the strategy matrix under a specific round (i.e., the first current round) in the first-stage gradient compression training process is described.

[0065] S230, controlling each working node to perform model training under the first current round, and calculating a first current loss value and a first current training time corresponding to the first current round according to the model training result of each working node.

[0066] Optionally, the control node controls each working node to perform model training in the first current round, specifically, the control node instructs each working node to input one or more training data into the machine learning model currently trained by the working node for training, and each working node calculates a loss value of a loss function of the working node for the training data according to a difference between a calculation result and a label result of the training data.

[0067] It can be understood that after completing the model training in the first current round, each working node can calculate a node loss value and a node training time consumption of the working node and report the same to the control node accordingly. After obtaining the node loss value and the node training time consumption reported by each working node, the control node can correspondingly calculate a first current loss value and a first current training time consumption matched with the first current round.

[0068] Each working node can record a starting time point when the model training in the first current round starts, record an ending time point when the model training in the first current round ends, and take a difference between the ending time point and the starting time point as the node training time consumption of the working node in the first current round.

[0069] In an optional embodiment of the present embodiment, according to the model training results of each working node, the first current loss value and the first current training time consumption matched with the first current round can be calculated by:

[0070] Obtaining the node loss value and the node training time consumption reported by each working node for the model training in the first current round; and calculating the first current loss value and the first current training time consumption matched with the first current round according to the node loss value and the node training time consumption corresponding to each working node.

[0071] Optionally, the control node can calculate the first current loss value and the first current training time consumption matched with the first current round by calculating an average value of the node loss value and an average value of the node training time consumption of each working node.

[0072] Specifically, the first current loss value Loss[t] in the first phase gradient compression training in the t-th round can be calculated by the formula:

[0073]

[0074] the first current loss value Loss[t] in the first phase gradient compression training in the t-th round, wherein N is the total number of working nodes, Loss[t] i is the node loss value calculated by the i-th working node in the t-th round;

[0075] and the first current training time consumption T[t] in the first phase gradient compression training in the t-th round can be calculated by the formula:

[0076]

[0077] computing a first current training latency Latency[t] in the tth round of the first-stage gradient compression training, wherein N is the total number of worker nodes, Latency[t] i is the node training latency of the ith worker node calculated in the tth round.

[0078] S240, obtaining a first current training state matched with the first current round, wherein the first round of training in the first-stage gradient compression training process has an initialized training state.

[0079] In the embodiment, the training state of each round in the first-stage gradient compression training process is calculated according to the difference value of the loss value compared with at least one previous round after the completion of the compressed training of the previous round. Since the first round of training in the first-stage gradient compression training process has no previous round, the initial value can be set for the training state of this round.

[0080] As shown in Table 1, the number of rows of the strategy matrix (S[1]-S[5]) is the total number of training states that can be selected in each round.

[0081] S250, selecting a first current gradient compression ratio matched with the first current training state according to the currently updated strategy matrix.

[0082] It should be noted that the purpose of learning the strategy matrix in the embodiment is to update the matrix score values in the strategy matrix. Therefore, it is definitely desirable to be able to obtain the distributed gradient compression training of each gradient compression ratio under each training state, and then the loss function and training latency of the above compressed training can be used to evaluate the corresponding matrix score values.

[0083] Further, when selecting the first current gradient compression ratio matched with the first current training state according to the currently updated strategy matrix, the gradient compression ratio with the maximum matrix score value cannot be selected in general. This is because if the selection is made according to such a strategy every time, only the matrix score values of one or a few gradient compression ratios under each training state can be updated each time, which may filter out a better gradient compression ratio under a certain training state, and is not conducive to the comprehensiveness of strategy matrix learning.

[0084] In order to solve the above problem, in an optional embodiment of the embodiment, selecting the first current gradient compression ratio matched with the first current training state according to the currently updated strategy matrix can include:

[0085] obtaining all gradient compression ratios matched with the first current training state in the currently updated strategy matrix; and selecting the first current gradient compression ratio from all gradient compression ratios by using an ε-greedy algorithm strategy.

[0086] In this embodiment, by using the ε-greedy algorithm strategy to select the first current gradient compression ratio among all gradient compression ratios, it is possible to ensure that the gradient compression ratio with the highest matrix score is selected to the greatest extent possible, while also randomly selecting other gradient compression ratios with different matrix scores (ε) for distributed gradient compression training.

[0087] S260. Control each working node to report the first compression gradient information according to the first current gradient compression ratio, and update the model parameters according to the first compression gradient information reported by each working node.

[0088] In this process, after the control node receives the first compression gradient information reported by each worker node according to the first current gradient compression ratio, it can perform a global reduction operation on the aforementioned first compression gradient information and use the global compression gradient information after the global reduction operation to update the model parameters. After completing the model parameter update, the control node can feed back the aforementioned model parameters to each worker node, so that each worker node can use the new model parameters to conduct a new round of model training.

[0089] S270. Based on the first current loss value and the first current training time, and the difference in loss value and training time compared with at least one previous round, calculate the first reward score and the first target training state matching the next round.

[0090] The first reward score can be understood as the benefit value that can be achieved by updating the model parameters by reporting the first compression gradient information according to the first current gradient compression ratio at each working node. The benefit value can be understood as the contribution value of the model training in the first current round to the entire model training process. The larger the benefit value, the larger the above contribution value, which in turn indicates that the matrix score value corresponding to the first current training state and the first current gradient compression ratio should also be larger.

[0091] Among them, Figure 2b The diagram shows a flowchart of a method for calculating the reward score and the training state for the next round, as shown below. Figure 2b As shown, the method includes:

[0092] S2701. Obtain the historical loss value of at least one previous round that matches the preset sliding window size.

[0093] The sliding window size can be an integer greater than or equal to 1. The sliding window can be understood as the number of preceding rounds selected starting from the first current round. For example, if the first current round is round t, when the sliding window is 1, the historical loss value of round t-1 is selected; when the sliding window is 2, the historical loss values ​​of rounds t-1 and t-2 are selected simultaneously.

[0094] S2702、According to the first current loss value and each historical loss value, a sliding window loss slope value and a sliding window loss standard deviation are calculated.

[0095] It can be understood that when multiple rounds of distributed model training are performed, the ultimate goal is to gradually converge the loss value of the loss function corresponding to each round to a stable value (i.e., a convergence value). The higher the speed of reaching this convergence value, the better the performance of the model training.

[0096] Based on this model training goal, the inventors have found that the standard deviation of the sample represents the degree of deviation of the sample, and thus the degree of oscillation of the convergence curve can be measured by calculating the standard deviation of the loss value in the sliding window. The greater the standard deviation, the more violent the fluctuation of the convergence curve, which means that the return at this time is lower.

[0097] In addition, since the environment state (feature space) of the distributed training system of the machine learning model changes rapidly and has no dependent relationship, the environment state can be simplified as an approximation of the trend of the expected value of the convergence curve. Accordingly, the approximate slope in the sliding window of the convergence curve can be used to measure the training state.

[0098] Based on this, after determining the size of the sliding window, the first reward score value and the first target training state matched with the next round can be finally calculated by calculating the sliding window loss slope value and the sliding window loss standard deviation.

[0099] Optionally, according to the first current loss value and each historical loss value, the sliding window loss slope value and the sliding window loss standard deviation can be calculated, which can include:

[0100] According to the formula:

[0101] Slope t =Rolling_slope(Loss[t,…,t-L+1],window=L)t≥L;

[0102] Std t =Rolling_std(Loss[t,…,t-L+1],window=L);

[0103] The sliding window loss slope value Slop t and the sliding window loss standard deviation Std t corresponding to the first current round t are calculated, and L is the number of previous rounds corresponding to the first current round t.

[0104] wherein the Rolling function is a sliding window function.

[0105] S2703, query the preset training state mapping table to obtain a first target training state corresponding to the sliding window slope value.

[0106] As described previously, the training state can be measured using the approximate slope within the convergence curve sliding window, and a training state mapping table can be established in advance to store the mapping relationship between the sliding window slope value range and the next round training state.

[0107] Optionally, the mapping relationship between the sliding window slope value range and the next round training state stored in the training state mapping table can include:

[0108] When the sliding window slope value range is [-0.01, 0.01], the next round training state is 0;

[0109] When the sliding window slope value range is (0.01, +∞), the next round training state is 1;

[0110] When the sliding window slope value range is (-10, -0.01), the next round training state takes a value in [2, …, 1+N];

[0111] Wherein, N is an integer greater than or equal to 1, when N is determined, the interval range (-10, -0.01) is equally divided into N sub-interval ranges, and one sub-interval range corresponds to one value in [2, …, 1+N].

[0112] In this embodiment, the training state has 2+N different training states, and by querying the training state mapping table according to the sliding window loss slope value calculated in the current calculation, the next round training state can be obtained in the above 2+N training states.

[0113] S2704, according to the sliding window loss standard deviation, the first current training time, the previous round training time and the sliding window loss slope value, a first reward score value is calculated.

[0114] Optionally, the first reward score value can be calculated according to the sliding window loss standard deviation, the first current training time, the previous round training time and the sliding window loss slope value, which can include:

[0115] According to the formula:

[0116]

[0117] The first reward score value Rt is calculated.

[0118] Wherein, Latency[t] is the first current training time, Latency[t-1] is the previous round training time, and sigmoid(.) is a sign function.

[0119] In the embodiment, a reward score value calculation formula determined by the sliding window loss standard deviation, the first current training time consumption, the previous round training time consumption and the sliding window loss slope value is constructed. The reason for such setting is that in addition to considering the standard deviation of the loss value in the sliding window to measure the convergence speed of the whole model training process, the training time consumption is further introduced as another evaluation index to ensure that the reward score value is a compromise between convergence speed and training time consumption.

[0120] S280, according to the first reward score value and the first target training state, updating the matrix score value in the policy matrix corresponding to the first current training state and the first current gradient compression ratio, and continuing to perform the first stage gradient compression training of the new round until completing the gradient compression training of the second iteration number of rounds.

[0121] Optionally, according to the first reward score value and the first target training state, updating the matrix score value in the policy matrix corresponding to the first current training state and the first current gradient compression ratio can include:

[0122] According to the formula:

[0123]

[0124] The first current training state S t and the first current gradient compression ratio K t corresponding to the matrix score value Q(S t , K t )′ of the first current round t are calculated.

[0125] Wherein, Q(S t , K t ) is the matrix score value before updating, Rt is the first reward score value, S t+1 is the first target training state, max k Q(S t+1 , k) is the maximum matrix score value in the policy matrix corresponding to S t+1 , and a, β are preset proportion coefficients.

[0126] S290, when the first stage gradient compression training is completed, controlling each working node in the distributed cluster to perform the second stage gradient compression training.

[0127] S2100, in each round of training in the second stage gradient compression training process, querying the updated policy matrix according to the training state corresponding to each round to determine the gradient compression ratio used by each working node in each round.

[0128] The technical scheme of the embodiment of the application is that in each round of training in the first-stage gradient compression training process, the strategy matrix is dynamically updated according to the loss value and the training time difference of each round compared with at least one previous round, and in each round of training in the second-stage gradient compression training process, the strategy matrix is queried according to the training state corresponding to each round to determine the gradient compression ratio used by each working node in each round. Based on the traditional top-k random sampling method, the strategy matrix for describing the distributed training performance under different gradient compression ratios in different training states is generated based on the dynamic programming method, thereby solving the problem of poor convergence accuracy of the deep learning model caused by using a fixed k value or selecting a k value by human experience, and achieving dynamic balance between convergence accuracy and convergence speed in the whole distributed gradient compression training process.

[0129] Embodiment three

[0130] Figure 3a A flowchart of a distributed gradient compression training method in the third embodiment of the application is shown in FIG. 3. This embodiment is refined based on the above-mentioned embodiments. In this embodiment, the operation of querying the updated strategy matrix according to the training state corresponding to each round to determine the gradient compression ratio used by each working node in each round in each round of training in the second-stage gradient compression training process is further refined.

[0131] Correspondingly, the method specifically includes the following operations:

[0132] S310, when it is determined that the strategy matrix starting learning condition is met, controlling each working node in the distributed cluster to perform first-stage gradient compression training.

[0133] S320, in each round of training in the first-stage gradient compression training process, dynamically updating the strategy matrix according to the loss value and the training time difference of each round compared with at least one previous round.

[0134] The strategy matrix is used to describe the matrix score value under different gradient compression ratios in each training state, and the matrix score value is proportional to the distributed training performance.

[0135] S330, when the first-stage gradient compression training is completed, controlling each working node in the distributed cluster to perform second-stage gradient compression training.

[0136] The second-stage gradient compression training includes gradient compression training of a third iteration number. The third iteration number can be preset according to actual conditions, for example, can be 5 times, 10 times or 15 times, etc., and the present embodiment does not limit this.

[0137] S340, a second current round in the second-stage gradient compression training process is obtained. S350, each worker node is controlled to perform model training of the second current round, and a second current loss value matched with the second current round is calculated according to the model training results of each worker node.

[0138] S360, a second current training state matched with the second current round is obtained.

[0139] The training state of the first training round of the second-stage gradient compression training process is a first target training state obtained by the last training round in the first-stage gradient compression training process.

[0140] In the embodiment, the training state of each round in the second-stage gradient compression training process is obtained according to the difference value of the loss value compared with at least one previous round after the completion of the compressed training of the previous round. Since the previous round of the first training round of the second-stage gradient compression training process is the last round (the last round) in the first-stage gradient compression training process, the training state of the next round calculated by the last round in the first-stage gradient compression training process can be used as the training state of the first training round of the second-stage gradient compression training process.

[0141] S370, a second current gradient compression ratio matched with the second current training state is selected according to the updated policy matrix.

[0142] In the first-stage gradient compression training process, the complete updating process of the policy matrix has been completed, and then, after obtaining the second current training state corresponding to the second current round, the gradient compression ratio corresponding to the maximum matrix score value can be obtained in the matrix row corresponding to the second current training state by querying the policy matrix, as the second current gradient compression ratio.

[0143] S380, each worker node is controlled to report second compressed gradient information according to the second current gradient compression ratio, and the model parameters are updated according to the second compressed gradient information reported by each worker node.

[0144] S390, a second target training state matched with the next round is calculated according to the loss value difference of the second current loss value compared with at least one previous round.

[0145] Specifically, the method of the foregoing embodiment can be used to calculate the second target training state matched with the next round for use in the next round.

[0146] That is, at least one historical loss value of a previous round matching a preset sliding window size is obtained, a sliding window loss slope value is calculated according to the first current loss value and each historical loss value, a preset training state mapping table is queried, and a second target training state corresponding to the sliding window slope value is obtained.

[0147] S3100, continue to perform the second phase gradient compression training of the new round until the gradient compression training of the third iteration number is completed.

[0148] The technical scheme of the embodiment of the application updates the strategy matrix dynamically according to the loss value and the training time difference of each round compared with at least one previous round in the first phase gradient compression training process, and then queries the updated strategy matrix according to the training state corresponding to each round in the second phase gradient compression training process to determine the gradient compression ratio used by each work node in each round. Based on the traditional top-k random sampling method, the strategy matrix for describing the distributed training performance under different gradient compression ratios under different training states is generated based on the dynamic programming method, which solves the problem of poor convergence accuracy of the deep learning model caused by using a fixed k value or selecting a k value by human experience. In the whole distributed gradient compression training process, the dynamic balance of convergence accuracy and convergence speed is achieved.

[0149] Specific application scenarios

[0150] In Figure 3b a specific application scenario suitable for the embodiment of the application is shown, as shown in Figure 3b the whole distributed gradient compression training process includes four steps;

[0151] Step 1, distributed training initialization phase

[0152] In the distributed training initialization phase, the initialization configuration of each parameter required in the whole model training process is completed. In Table 2, the parameter initialization description related to gradient compression in the distributed system startup phase is shown.

[0153] Table 2

[0154]

[0155]

[0156] Step 2, distributed system gradient compression training phase

[0157] 1) Determine whether the current training number n is <= Skip_num? If it is satisfied, go to 3); otherwise, go to step 3.

[0158] 2), control each work node to read each batch of training data, obtain the node loss value and the complete node model gradient reported by each work node, and calculate the forward loss value and the reverse gradient of the distributed training system according to the node loss value and the node model gradient.

[0159] 3), performing global reduction operation on the loss value and the gradient, and then updating the model weight parameter.

[0160] 4), n = n + 1, jump to 1) in step two.

[0161] Step three, Q[S, K] learning phase of the strategy matrix

[0162] 1), judge whether the current learning number t is greater than N_max? If satisfied, jump to step four; otherwise, enter 2) in step three.

[0163] 2), control each work node to read batch training data, and calculate the forward loss value, the reverse gradient value, and the training time value; after receiving the loss value and the training time value sent by each work node, perform global reduction operation on each loss value to calculate the average loss value.

[0164] 3), according to the training state, based on Q[S t ,K]ε-greedy method is used to sample Kt (gradient compression ratio in the strategy matrix)

[0165] 4), control each work node to perform top-k gradient sampling based on Kt to obtain the compressed gradient for reporting, and perform global reduction operation on each reported compressed gradient to update the weight parameter of the machine learning model.

[0166] 5), calculate the training state of the next round.

[0167] 6), after updating the Q[S, K] matrix, n = n + 1, t = t + 1, jump to 1) in step three.

[0168] Step four, distributed training phase based on Q[S, K]

[0169] 1), control each work node to read batch training data, and calculate the forward loss value and the reverse gradient value; after receiving the loss value sent by each work node, perform global reduction operation on each loss value to calculate the average loss value.

[0170] 2), if it is the first training, according to the Q[St=N_max+1, K] of the previous stage, select the maximum Q value to obtain Kn, otherwise, according to the Sn+1 of the previous step cycle, select the maximum Q value to obtain Kn.

[0171] 3), control each work node based on Kn top-k gradient sampling, get the compressed gradient for reporting, and perform global reduction operation on each reported compressed gradient, and update the weight parameters of the machine learning model, and calculate the next round of training state Sn+1.

[0172] 4), n = n + 1, if the training number n>global_step is satisfied, the training is ended, otherwise jump to step four 1).

[0173] In the specific application scenario of the embodiment of the application, on the basis of the traditional top-K sampling, a K value candidate list is introduced, the optimal strategy matrix Q[s,k] is obtained through difference iteration based on a dynamic programming method combined with the sliding variance and slope of the convergence curve; based on the technical solution of the embodiment of the application, the selection of the K value strategy can be dynamically performed according to the feedback of the distributed system, the reliability problem of the calculation threshold caused by the random sampling method is optimized, and a dynamic balance is reached in terms of convergence speed and convergence accuracy; in addition, the technical solution of the embodiment of the application has a calculation complexity O(n) through the introduced dynamic programming method, n is the size of the Q matrix, compared with other gradient compression methods such as low-rank decomposition, the calculation complexity is significantly reduced.

[0174] Embodiment four

[0175] In Figure 4 a structural diagram of a distributed gradient compression training device in the embodiment of the application is shown, the device is applied to a control node in a distributed cluster, as shown in Figure 4 the device comprises a first-stage compression training module 410, a strategy matrix updating module 420, a second-stage compression training module 430 and a gradient compression ratio determination module 440, wherein:

[0176] The first-stage compression training module 410 is configured to control each work node in the distributed cluster to perform first-stage gradient compression training when it is determined that a strategy matrix starting learning condition is met.

[0177] The strategy matrix updating module 420 is configured to dynamically update the strategy matrix according to the loss value and the training time difference of each round compared with at least one previous round in each round of training in the first-stage gradient compression training process.

[0178] The strategy matrix is used to describe the matrix score value under different gradient compression ratios in each training state, and the matrix score value is proportional to the distributed training performance.

[0179] The second-stage compression training module 430 is configured to control each work node in the distributed cluster to perform second-stage gradient compression training when the first-stage gradient compression training is completed.

[0180] The gradient compression ratio determination module 440 is configured to determine, in each round of training of the second-stage gradient compression training process, a gradient compression ratio used by each work node in each round according to the strategy matrix updated in correspondence with the training state of each round.

[0181] The technical scheme of the embodiment of the present application dynamically updates the strategy matrix according to the loss value and the training time difference between each round and at least one previous round in each round of the first-stage gradient compression training process, and then determines, in each round of the second-stage gradient compression training process, a gradient compression ratio used by each work node in each round according to the updated strategy matrix corresponding to the training state of each round. Based on the dynamic programming method, the strategy matrix describing the distributed training performance under different gradient compression ratios in different training states is generated on the basis of the traditional top-k random sampling method, thereby solving the problem of poor convergence accuracy of the deep learning model caused by using a fixed k value or selecting a k value based on human experience, and achieving a dynamic balance between convergence accuracy and convergence speed in the entire distributed gradient compression training process.

[0182] On the basis of the above-mentioned embodiments, the first-stage compression training module 410 can be specifically configured to:

[0183] When the gradient-free compression training of the first iteration number is completed by each work node in the distributed cluster, it is determined that the strategy matrix start learning condition is met.

[0184] On the basis of the above-mentioned embodiments, the first-stage gradient compression training includes gradient compression training of the second iteration number.

[0185] Correspondingly, the strategy matrix updating module 420 can include:

[0186] The current round acquisition unit is configured to acquire a first current round in the first-stage gradient compression training process.

[0187] The loss time calculation unit is configured to control each work node to perform model training in the first current round, and calculate a first current loss value and a first current training time matched with the first current round according to the model training result of each work node.

[0188] The current training state acquisition unit is configured to acquire a first current training state matched with the first current round, wherein the first round of training of the first-stage gradient compression training process has an initialized training state.

[0189] The current compression ratio acquisition unit is configured to select a first current gradient compression ratio matched with the first current training state according to the currently updated strategy matrix.

[0190] a model parameter updating unit, configured to control each working node to report first compressed gradient information according to a first current gradient compression ratio, and update model parameters according to the first compressed gradient information reported by each working node;

[0191] a score state calculating unit, configured to calculate a first reward score value and a first target training state matched with a next round according to a first current loss value and a first current training time consumption, and a loss value and a training time consumption difference compared with a previous at least one round;

[0192] a repeating executing unit, configured to continue to perform the first-stage gradient compression training of the new round according to the first reward score value and the first target training state after updating a matrix score value in the policy matrix corresponding to the first current training state and the first current gradient compression ratio, until the gradient compression training of the second iteration number of rounds is completed.

[0193] On the basis of each of the above embodiments, the loss time consumption calculating unit can be specifically configured to:

[0194] obtain a node loss value and a node training time consumption reported by each working node for the first current round of model training;

[0195] calculate a first current loss value and a first current training time consumption matched with the first current round according to the node loss value and the node training time consumption corresponding to each working node respectively.

[0196] On the basis of each of the above embodiments, the current compression ratio obtaining unit can be specifically configured to:

[0197] obtain all gradient compression ratios matched with the first current training state in the currently updated policy matrix;

[0198] select the first current gradient compression ratio from the all gradient compression ratios by using an ε-greedy algorithm policy.

[0199] On the basis of each of the above embodiments, the score state calculating unit can specifically include:

[0200] a historical loss value obtaining sub-unit, configured to obtain a historical loss value of at least one previous round matched with a preset sliding window size;

[0201] a slope standard deviation calculating sub-unit, configured to calculate a sliding window loss slope value and a sliding window loss standard deviation according to the first current loss value and each historical loss value;

[0202] The training state query subunit is configured to query a preset training state mapping table to obtain a first target training state corresponding to the sliding window slope value, wherein the training state mapping table stores a mapping relationship between a sliding window slope value range and a next round training state.

[0203] The reward score value calculation subunit is configured to calculate a first reward score value according to the sliding window loss standard deviation, the first current training time consumption, the previous round training time consumption, and the sliding window loss slope value.

[0204] On the basis of the above embodiments, the mapping relationship between the sliding window slope value range and the next round training state stored in the training state mapping table specifically includes:

[0205] When the sliding window slope value range is [-0.01, 0.01], the next round training state is 0.

[0206] When the sliding window slope value range is (0.01, +∞), the next round training state is 1.

[0207] When the sliding window slope value range is (-10, -0.01), the next round training state takes a value in [2, …, 1+N].

[0208] wherein N is an integer greater than or equal to 1, when N is determined, the interval range (-10, -0.01) is equally divided into N sub-interval ranges, and one sub-interval range corresponds to one value in [2, …, 1+N].

[0209] On the basis of the above embodiments, the slope standard deviation calculation subunit is specifically configured to:

[0210] According to the formula:

[0211] Slope t = Rolling_slope(Loss[t,…,t-L+1], window=L) t≥L;

[0212] Std t = Rolling_std(Loss[t,…,t-L+1], window=L);

[0213] The sliding window loss slope value Slope t and the sliding window loss standard deviation Std t corresponding to the first current round t are calculated, and L is a quantity value of a previous round corresponding to the first current round t.

[0214] On the basis of the above embodiments, the reward score value calculation subunit can be specifically configured to:

[0215] According to the formula:

[0216]

[0217] The first reward score value Rt is calculated.

[0218] Wherein, Latency[t] is the first current training time consumption, Latency[t-1] is the previous round training time consumption, and sigmoid(.) is a sign function.

[0219] On the basis of the above embodiments, the repeating execution unit can be specifically used for:

[0220] According to the formula:

[0221]

[0222] The first current training state S t and the first current gradient compression ratio K t corresponding to the matrix score value Q(S t , K t )′ is calculated.

[0223] Wherein, Q(S t , K t ) is the matrix score value before updating, Rt is the first reward score value, S t+1 is the first target training state, max k Q(S t+1 , k) is the maximum matrix score value corresponding to S t+1 in the policy matrix, and a and β are preset proportion coefficients.

[0224] On the basis of the above embodiments, the second stage gradient compression training includes gradient compression training of a third iteration number.

[0225] Correspondingly, the gradient compression ratio determination module 440 can be specifically used for:

[0226] Obtain the second current round in the second stage gradient compression training process;

[0227] Control each work node to execute model training of the second current round, and calculate a second current loss value matched with the second current round according to the model training results of each work node;

[0228] Obtain a second current training state matched with the second current round, wherein the training state of the first round of training in the second stage gradient compression training process is the first target training state obtained by the last round of training in the first stage gradient compression training process;

[0229] According to the updated policy matrix, a second current gradient compression ratio matching the second current training state is selected;

[0230] The control node controls each work node to report second compressed gradient information according to the second current gradient compression ratio, and updates the model parameters according to the second compressed gradient information reported by each work node;

[0231] According to the loss value difference of the second current loss value compared with the previous at least one round, a second target training state matching the next round is calculated;

[0232] The second phase gradient compression training of the new round is continued to be performed until the gradient compression training of the third iteration number of rounds is completed.

[0233] The distributed gradient compression training device provided by the embodiment of the application can execute the distributed gradient compression training method provided by any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0234] Embodiment five

[0235] Figure 5 A structural schematic diagram of a computer device provided by the fifth embodiment of the application. The computer device can be a control node in a distributed cluster. As shown in the figure, the computer device includes a processor 50, a memory 51, an input device 52 and an output device 53; the number of processors 50 in the computer device can be one or more, and one processor 50 is taken as an example in the figure; the processor 50, the memory 51, the input device 52 and the output device 53 in the computer device can be connected through a bus or other means, and the connection through the bus is taken as an example in the figure. Figure 5 Figure 5 Figure 5

[0236] The memory 51 is a kind of computer readable storage medium, which can be used to store software programs, computer executable programs and modules, such as program instructions / modules (for example, first phase compression training module 410, policy matrix updating module 420, second phase compression training module 430 and gradient compression ratio determination module 440) corresponding to the distributed gradient compression training method in the embodiment of the application. The processor 50 executes the software program, instruction and module stored in the memory 51, thereby performing various function applications and data processing of the computer device, i.e. realizing the distributed gradient compression training method described above.

[0237] That is, when it is determined that the policy matrix starting learning condition is met, the control node controls each work node in the distributed cluster to perform first phase gradient compression training;

[0238] ​​​In each round of training in the first-stage gradient compression training process, the strategy matrix is dynamically updated according to the loss value and the training time difference of each round compared with at least one previous round.

[0239] The strategy matrix is used to describe the matrix score value under different gradient compression ratios in each training state, and the matrix score value is proportional to the distributed training performance.

[0240] When the first-stage gradient compression training is completed, the working nodes in the distributed cluster are controlled to perform the second-stage gradient compression training.

[0241] In each round of training in the second-stage gradient compression training process, the strategy matrix is queried according to the training state corresponding to each round to determine the gradient compression ratio used by each working node in each round.

[0242] The memory 51 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 51 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 51 can further include a memory remotely arranged with respect to the processor 50, which can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0243] The input device 52 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device. The output device 53 can include a display device such as a display screen.

[0244] Embodiment six

[0245] Embodiment six of the present application also provides a storage medium containing computer executable instructions, which when executed by a computer processor, are used to perform a distributed gradient compression training method, the method comprising:

[0246] When it is determined that the strategy matrix starting learning condition is met, the working nodes in the distributed cluster are controlled to perform the first-stage gradient compression training;

[0247] In each round of training in the first-stage gradient compression training process, the strategy matrix is dynamically updated according to the loss value and the training time difference of each round compared with at least one previous round.

[0248] The policy matrix is used to describe the matrix score value of different gradient compression ratios under each training state, and the matrix score value is proportional to the distributed training performance.

[0249] When the first-stage gradient compression training is completed, the control unit controls each work node in the distributed cluster to perform the second-stage gradient compression training.

[0250] In each round of training in the second-stage gradient compression training process, the policy matrix updated according to the training state corresponding to each round is queried to determine the gradient compression ratio used by each work node in each round.

[0251] Of course, the storage medium provided by the embodiment of the present application contains computer executable instructions, which are not limited to the method operations described above, and can also perform related operations in the distributed gradient compression training method provided by any embodiment of the present application.

[0252] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH, a hard disk or an optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment of the present application.

[0253] It is worth noting that in the above embodiment of the search device, each unit and module included is only divided according to functional logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for easy mutual distinction, and does not limit the protection scope of the present application.

[0254] Note that the above merely describes preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, modifications and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for distributed gradient compression training, executed by a control node in a distributed cluster, characterized in that, The method comprises the following steps: When it is determined that the policy matrix starting learning condition is met, control each work node in the distributed cluster to perform first-stage gradient compression training; In each round of training in the first-stage gradient compression training process, the policy matrix is dynamically updated according to the loss value and the training time difference of each round compared with at least one previous round; Wherein, the policy matrix is used to describe the matrix score value under different gradient compression ratios in each training state, and the matrix score value is proportional to the distributed training performance; When the first-stage gradient compression training is completed, control each work node in the distributed cluster to perform second-stage gradient compression training; In each round of training in the second-stage gradient compression training process, the policy matrix is queried according to the training state corresponding to each round to determine the gradient compression ratio used by each work node in each round; Wherein, the first-stage gradient compression training includes gradient compression training of the second iteration number; In each round of training in the first-stage gradient compression training process, the policy matrix is dynamically updated according to the loss value and the training time difference of each round compared with at least one previous round, comprising: Obtain the first current round in the first-stage gradient compression training process; Control each work node to perform model training in the first current round, and calculate the first current loss value and the first current training time according to the model training results of each work node; Obtain the first current training state matched with the first current round, wherein the first round of training in the first-stage gradient compression training process has an initialized training state; According to the current updated policy matrix, select the first current gradient compression ratio matched with the first current training state; Control each work node to report the first compressed gradient information according to the first current gradient compression ratio, and update the model parameters according to the first compressed gradient information reported by each work node; Obtain the historical loss value of at least one previous round matched with the preset sliding window size; According to the first current loss value and each historical loss value, the sliding window loss slope value and the sliding window loss standard deviation are calculated; Query the preset training state mapping table to obtain the first target training state corresponding to the sliding window slope value, wherein the training state mapping table stores the mapping relationship between the sliding window slope value range and the next round training state; According to the sliding window loss standard deviation, the first current training time, the previous round training time and the sliding window loss slope value, the first reward score value is calculated; After updating the matrix score value corresponding to the first current training state and the first current gradient compression ratio in the policy matrix according to the first reward score value and the first target training state, the first-stage gradient compression training of the new round is continued until the gradient compression training of the second iteration number is completed.

2. The method of claim 1, wherein the policy matrix starting learning condition is met when each work node in the distributed cluster completes the first iteration number of gradient-free compression training. ​ 3. The method of claim 1, wherein, According to the model training result of each work node, a first current loss value and a first current training time period matched with the first current round are calculated, including: Obtaining the node loss value and the node training time period reported by each work node for the first current round of model training; According to the node loss value and the node training time period corresponding to each work node, a first current loss value and a first current training time period matched with the first current round are calculated.

4. The method of claim 1, wherein, According to the current updated strategy matrix, a first current gradient compression ratio matched with the first current training state is selected, including: In the current updated strategy matrix, all gradient compression ratios matched with the first current training state are obtained; Using an ε-greedy algorithm strategy, the first current gradient compression ratio is selected from all gradient compression ratios.

5. The method of claim 1, wherein, The mapping relationship between the sliding window slope value range and the next round training state stored in the training state mapping table specifically includes: When the sliding window slope value range is [-0.01, 0.01], the next round training state is 0; When the sliding window slope value range is (0.01, +∞), the next round training state is 1; When the sliding window slope value range is (-10, -0.01), the next round training state takes a value in [2, …, 1+N]; Wherein, N is an integer greater than or equal to 1, when N is determined, (-10, -0.01) interval range is equally divided into N sub-interval ranges, and one sub-interval range corresponds to a value in [2, …, 1+N].

6. The method of claim 5, wherein, According to the first current loss value and each historical loss value, a sliding window loss slope value and a sliding window loss standard deviation are calculated, including: According to the formula: Slope t = Rolling_slope(Loss[t,…,t-L+1], window=L) t≥L; Std t = Rolling_std(Loss[t,...,t-L+1], window=L); The sliding window loss slope value Slope corresponding to the first current round t is calculated t and the sliding window loss standard deviation Std t , L is the number value of the previous rounds corresponding to the first current round t.

7. The method of claim 6, wherein, According to the sliding window loss standard deviation, the first current training time period, the previous round training time period and the sliding window loss slope value, a first reward score value is calculated, including: According to the formula: The first reward score value Rt is calculated; Wherein, Latency[t] is the first current training time period, Latency[t-1] is the previous round training time period, and sigmoid(.) is a sign function.

8. The method of claim 1, wherein, According to the first reward score value and the first target training state, the matrix score value corresponding to the first current training state and the first current gradient compression ratio in the strategy matrix is updated, including: According to the formula: a first current training state S of a first current round t is calculated t and a first current gradient compression ratio K t a jointly corresponding matrix score value Q(S t , K t )′; Among them, Q(S) t K t Rt represents the matrix score before the update, Rt represents the first reward score, and S represents the matrix score before the update. t+1 For the first target training state, max k Q(S t+1 k) is the strategy matrix related to S t+1 The corresponding maximum matrix score, where α and β are preset scaling factors.

9. The method according to any one of claims 1 to 8, characterized in that, The third iteration number of gradient compression training is included in the second stage gradient compression training. In each round of training in the second stage gradient compression training process, the updated strategy matrix is queried according to the training state corresponding to each round to determine the gradient compression ratio used by each work node in each round, including: Obtaining the second current round in the second stage gradient compression training process; Controlling each work node to perform model training of the second current round, and calculating a second current loss value matched with the second current round according to the model training result of each work node; Obtaining a second current training state matched with the second current round, wherein the training state of the first round of training in the second stage gradient compression training process is the first target training state obtained in the last round of training in the first stage gradient compression training process; According to the updated policy matrix, a second current gradient compression ratio matching the second current training state is selected; The control each work node reports second compressed gradient information according to second current gradient compression ratio, and carries out model parameter update according to the second compressed gradient information reported by each work node; According to the loss value difference of the second current loss value compared with the previous at least one round, a second target training state matching the next round is calculated; Continue to execute the second phase gradient compression training of the new round until the gradient compression training of the third iteration round is completed.

10. A device for distributed gradient compression training, applied to a control node in a distributed cluster, characterized in that, Comprise: The first phase compression training module is used for controlling each work node in the distributed cluster to execute the first phase gradient compression training when it is determined that the policy matrix starting learning condition is met; The policy matrix updating module is used for dynamically updating the policy matrix according to the loss value and the training time difference of each round compared with the previous at least one round in the process of the first phase gradient compression training; Wherein, the policy matrix is used for describing the matrix score value under different gradient compression ratios in each training state, and the matrix score value is proportional to the distributed training performance; The second phase compression training module is used for controlling each work node in the distributed cluster to execute the second phase gradient compression training when the first phase gradient compression training is completed; The gradient compression ratio determination module is used for querying the updated policy matrix according to the training state corresponding to each round in the process of the second phase gradient compression training to determine the gradient compression ratio used by each work node in each round; Wherein, the first phase gradient compression training includes the gradient compression training of the second iteration round; The policy matrix updating module is specifically used for: Obtain the first current round in the process of the first phase gradient compression training; Control each work node to execute model training under the first current round, and calculate the first current loss value and the first current training time according to the model training result of each work node; Obtain the first current training state matching the first current round, wherein the first round of the first phase gradient compression training process has an initialized training state; According to the current updated policy matrix, a first current gradient compression ratio matching the first current training state is selected; Control each work node to report first compressed gradient information according to the first current gradient compression ratio, and update the model parameters according to the first compressed gradient information reported by each work node; Obtain the historical loss value of at least one previous round matching the preset sliding window size; According to the first current loss value and each historical loss value, a sliding window loss slope value and a sliding window loss standard deviation are calculated; Query the preset training state mapping table to obtain the first target training state corresponding to the sliding window slope value, wherein the training state mapping table stores the mapping relationship between the sliding window slope value range and the next round training state; According to the sliding window loss standard deviation, the first current training time, the previous round training time and the sliding window loss slope value, a first reward score value is calculated. After updating the matrix score value in the policy matrix corresponding to the first current training state and the first current gradient compression ratio according to the first reward score value and the first target training state, the first phase of gradient compression training of the new round is continued to be performed until the gradient compression training of the second iteration number of rounds is completed.

11. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the distributed gradient compression training method of any one of claims 1-9 when executing the program.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the distributed gradient compression training method of any one of claims 1-9. The program is executed by the processor to implement the distributed gradient compression training method of any one of claims 1-9.

Images