Federal learning ternary quantization method based on momentum acceleration and error feedback

Abstract

The invention belongs to the field of federated learning communication overhead optimization, and particularly relates to a federated learning ternary quantization method based on momentum acceleration and error feedback, which comprises the following steps that: a central server issues model parameters, a client of a random sampling part receives a model, the client locally updates the model, and the model is updated; the difference delta between the updated model and the model before updating is calculated, the difference delta and the error e are calculated through a Ternary quantification method to obtain a final compression result, the new error of the round is updated, finally, the compression result is uploaded to a central server, the server updates the global model through a momentum acceleration algorithm after parameters are aggregated, a plurality of rounds are executed, and the compression efficiency is improved. And finally, a trained target model is obtained. According to the method, a Ternary quantification method is used on the federated learning baseline model, so that the overhead of a transmission model is greatly reduced, and meanwhile, the Adam momentum acceleration and error feedback mechanism is combined, so that the model can still converge quickly under the condition of reducing the communication overhead.

Description

Technical Field

[0001] The present invention relates to the field of intelligent question answering technology, and in particular to a federated learning ternary quantization method based on momentum acceleration and error feedback. Background Art

[0002] With the advent of the era of massive data, sufficient data provides a good training opportunity for the model for the traditional centralized machine learning method. However, massive data comes from independent clients. As society pays more and more attention to privacy protection and data security issues, Google proposed a new framework of federated learning in 2016. In this context, since it involves communication and collaboration between multiple devices, the performance of federated learning is limited by the efficiency and bandwidth of communication. Therefore, researchers need to explore efficient communication strategies to reduce communication overhead and improve training efficiency. There are two main methods to reduce communication overhead. The first is to reduce communication overhead by reducing the number of communications between the central service and the client. For example, the Federated Averaging algorithm mainly increases the computational workload of the local client, thereby reducing the number of updated parameters submitted, and ultimately reducing communication overhead. The One-Shot Averaging algorithm only communicates once during the entire training process, which can be proven to have good results in some strongly convex problems, but it does not perform well in some non-convex problems and is extremely dependent on the computing power of the client.

[0003] Another way to reduce communication overhead is to reduce the amount of data transmitted in each round of communication. Since uncompressed models are generally large in dimension, they will incur huge overhead during transmission. The idea of ​​solving this problem is mainly to use a small amount of encoded tag information, that is, to compress the model. There are currently two main methods to reduce uplink communication overhead: one is to reduce the number of parameters in the model through low rank and random masking methods, and the other is to compress model parameters through quantization, random rotation, subsampling and other methods. For example, in terms of model sparsification, not all information in the gradient is important. For some model parameters, even if 99% of the parameter information is mapped to zero and the sparse matrix is ​​uploaded to the server, good results can be achieved. However, only a few model parameters meet the conditions, so this method is not objective. In terms of quantization, it is mainly through discretizing continuous values ​​and replacing the original 32-bit, 64-bit and other high-precision parameters with low-precision parameters such as 8 bits, 2 bits or even 1 bit. This is a proven method of quantizing the parameters in each gradient into binary symbols. The amount of data transmitted can be reduced by 32 times, but the compression model will collapse in the downlink, causing large errors. The final model accuracy depends on the choice of quantization accuracy.

[0004] Although the above two methods are effective in reducing the communication overhead of the federated learning framework, they do not take into account the loss of model accuracy and the residual information in the previous gradient that also affects the model to a certain extent. The final model effect is an important criterion that cannot be ignored. Based on this, historical information needs to be considered. Existing research uses an error feedback mechanism to compensate for the loss caused by compression. The main idea is to accumulate the compressed gradient information of the previous rounds to the current round, and then upload it to the central server after compression. It is considered to store the accumulated gradient second-order momentum on the server to improve the learning ability of the neural network. However, if the second-order momentum changes abnormally, it will cause fluctuations in the training process. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a ternary quantization method for federated learning based on momentum acceleration and error feedback. It includes the following steps: the central server initializes the model parameters to be trained, and in each round of training, a fixed number of clients are randomly sampled to receive the global model sent from the central server. After these clients perform multiple steps of stochastic gradient descent locally, they calculate the difference Δ between the trained model and the model sent by the central server, and calculate the difference Δ together with the error e through the ternary quantization method to obtain the final compression result, and at the same time update the new error generated by the current training round of the client, while for those clients that did not participate in this training, the error remains unchanged from the previous round. Finally, the clients participating in the training upload the compression results to the central server, and then the server aggregates these parameters and updates the global model through the momentum acceleration algorithm, and then sends the new global model to each client participating in the next round of training, so as to perform multiple rounds and finally obtain the trained target model;

[0006] The training process includes the following steps:

[0007] S1. Get the original data set, divide it into training set and test set, and preprocess the data of the training set: take the first 80% of the data in the data set as the training data set, and in each epoch, shuffle the training data set and distribute it in fixed batch units; take the last 20% of the data as the test data set, and in each epoch, distribute the data in sequence in fixed batch units;

[0008] S2, initialize the neural network model to be trained, the global iteration round epoch, the global learning rate learningrate, the number of all clients participating in the training num_users, the ratio of the number of clients participating in each round of training frac, the local iteration round local_epoch, the local learning rate local rate and other parameters;

[0009] S3, the central device sends the global model;

[0010] S4. Randomly sample clients with a fraction of frac from all clients, and assign training data and test data loaders to the selected clients by index;

[0011] S5. Each client updates the model locally using the stochastic gradient descent optimizer based on the allocated data, calculates the difference between the local updated model and the model sent by the server, aggregates the model difference and the quantization error (initial 0), uses ternary quantization to aggregate the results to obtain the compressed model, and updates the quantization error of the current round to the difference between before and after quantization. The clients participating in the training upload the quantized model.

[0012] S6, the central server aggregates the model updates from the clients, accelerates the update of the global model through the Adam optimization algorithm, and returns to step S3;

[0013] S7. Verify the effect of model parameters trained in each epoch through the test data set;

[0014] S8. Repeat the above steps. When the global number of iterations is met, end the training and obtain the final training model.

[0015] Furthermore, the central server aggregates the difference between the model of each client participating in the training and the model before training, and then averages them to update the global model. The formula is:

[0016]

[0017] In the standard paradigm of federated learning, the central server updates the global model by aggregating the average of the models of each client participating in the training after each round of training. The formula is:

[0018]

[0019] It can be proved that

[0020] Furthermore, the adaptive gradient optimization Adam formula is:

[0021] The first-order momentum update rule is:

[0022] The second-order momentum update rule is:

[0023] Furthermore, the client local gradient descent formula is:

[0024]

[0025] Furthermore, the loss function formula is:

[0026]

[0027] Furthermore, the ternary quantization operator formula is:

[0028]

[0029] Furthermore, b t Indicates that it is signed(g t ) The binary mask matrix of the gradient after processing satisfies the following distribution, and the formula is:

[0030]

[0031] Furthermore, the error feedback mechanism formula is:

[0032]

[0033] Furthermore, the global model update rule calculation formula is:

[0034]

[0035] Beneficial effects of the present invention:

[0036] (1) The present invention uses the ternary quantization method to compress the model parameters on the FedAvg model, and randomly quantizes the floating-point gradient to three values ​​of {-1, 0, +1}. While ensuring the accuracy of the model, this method only uses 3 sign bits, which can theoretically improve Times the communication efficiency and greatly reduces the gradient communication volume.

[0037] (2) The present invention utilizes an error feedback mechanism on the FedAvg model to compensate for the model accuracy loss caused by quantization.

[0038] (3) The present invention uses the Adam optimization algorithm on the FedAvg model to accelerate model convergence. While maintaining a high degree of learning rate adaptability, it significantly reduces communication overhead. The client uploads the quantized model differences to the server instead of the model itself. The server aggregates these differences and updates the next round of models through Adam. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Figure 1 It is the overall architecture diagram of the present invention;

[0041] Figure 2 This is a flow chart of the client local model update process in the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] A ternary quantization method for federated learning based on momentum acceleration and error feedback, such as Figure 1 The overall architecture diagram of the method of the present invention is shown. First, the central server initializes the model parameters to be trained. In each round of training, the clients with a random sampling fixed ratio of frac receive the global model sent from the central server. These clients use the stochastic gradient descent optimizer locally. After iteratively updating the local_epoch step, the difference Δ between the model after training and the model sent by the central server is calculated. The difference Δ and the error e (also called historical information, with an initial value of 0) are calculated together through the ternary quantization method to obtain the final compression result. At the same time, the new error generated by the current training round of the client is updated. For those clients who did not participate in this training, the error remains unchanged from the previous round. The clients participating in the training upload the compression results to the central server, and then the server aggregates these parameters and updates the global model through the Adam algorithm, and then sends the new global model to each client participating in the next round of training. When the number of training times meets the epoch, the trained target model is finally obtained.

[0044] Figure 2 The following is a flowchart of the client local model update process in the present invention:

[0045] S1. Get the original data set, divide it into training set and test set, and preprocess the data of the training set: take the first 80% of the data in the data set as the training data set, and in each epoch, shuffle the training data set in fixed batch units for distribution; then take 10% of the data as the test data set, and finally use 10% of the data as the validation set, and in each epoch, distribute the data in order in fixed batch units;

[0046] The input of the data set is:

[0047] The dataset used is CIFAR-10, an image classification dataset widely used in machine learning and computer vision. The CIFAR-10 dataset contains 60,000 32*32 pixel color images, divided into 10 different categories, each category has 6,000 images, and the dataset is divided into 50,000 training images and 10,000 test images. For the data in the test set, the images are randomly cropped and randomly flipped horizontally to increase the diversity of the data, and then the image data is normalized to make the distribution of the input data closer to the standard normal distribution, which is helpful for model training.

[0048] S2, initialize the neural network model to be trained, the global iteration round epoch, the global learning rate learningrate, the number of all clients participating in the training num_users, the ratio of the number of clients participating in each round of training frac, the local iteration round local_epoch, the local learning rate local rate and other parameters;

[0049] The model input is:

[0050] The training model is ResNet, a deep learning model architecture used to solve the problem of model performance degradation in deep neural network training.

[0051] S3, the central server sends the global model;

[0052] Specifically, during the first round of training, the central server will send the determined model to be trained to the clients participating in the training. Before the global iteration epoch is met, the central server needs to aggregate the model difference updates uploaded by all clients participating in the training and calculate the global model required for the next round of training.

[0053] S4. Randomly sample clients with a fraction of frac from all clients, and assign training data and test data loaders to the selected clients by index;

[0054] S5. Each client updates the model locally using the stochastic gradient descent optimizer based on the allocated data. The formula is:

[0055]

[0056] in, represents the gradient, represents the loss function, α represents the local learning rate of each client, It represents the model of the i-th client in the k-th round during the t-th round of training. Then the difference between the local updated model and the model sent by the server is calculated using the formula:

[0057] Δt =W t+1 -W t #(3)

[0058] The compression model is obtained by adding the model difference and the quantization error (initialized to 0), and the result after ternary quantization aggregation is used. At the same time, the quantization error of the current round is updated to the difference before and after quantization. The client participating in the training uploads the quantized model. The formula is:

[0059]

[0060] C represents the quantization of the model difference and the quantization error of the previous round through the ternary quantization operation, which is called pseudo gradient. The quantization steps include:

[0061] Step 1: Calculate the symbolic matrix of pseudo gradient:

[0062] Usually, floating point numbers are used to store gradient information, and the sign of each item in the pseudo gradient is calculated by the sign function. Specifically:

[0063] If the current item value is negative, it is represented by -1; if the value is 0, it remains unchanged; if the value is 0, it is represented by 1. Ternary is an orthogonal sparse method that quantizes the pseudo gradient into a ternary level {-1, 0, 1}.

[0064] Step 2: Calculate the scaling factor:

[0065] In order to minimize the number of levels, a globally shared scalar, called a scaling factor, is needed to control the change in the value of each item in the final pseudo gradient and restore the influence of the real gradient as much as possible. Therefore, a very small amplitude is changed by calculating the infinite norm of the real gradient parameter, that is, the absolute maximum value item in the gradient information. The specific calculation formula is:

[0066]

[0067] Among them, s t represents the scaling factor, g t represents the true gradient.

[0068] Step 3: Calculate the binary mask matrix of pseudo gradient:

[0069] The parameter distribution of the binary mask matrix is ​​{0, 1}. For each value in the pseudo gradient, its absolute value and scaling factor s are calculated. t The ratio of , which indicates the probability of its value being 1, otherwise, it indicates the probability of its value being 0.

[0070] Specifically, the binary mask matrix calculation formula is:

[0071]

[0072] Among them, b tk Indicates that it is signed(g t ) is the value of the tth row and kth column in the gradient matrix after processing, g tk Represents the value of the tth row and kth column in the true gradient parameter matrix.

[0073] Step 4: Calculate the pseudo gradients that need to be uploaded:

[0074] The meaning of the pseudo gradient matrix obtained by Hadamard product of the sign matrix and the binary mask matrix is ​​expressed as follows: for each value, if the sign of the true gradient is 0, the corresponding item is still 0; if the sign of the true gradient is 1, refer to the value of the binary mask matrix corresponding to the item. If the ratio to the scaling factor is larger, it means that the influence of the item is greater, then the probability of retaining the item is greater (marked as 1), otherwise, the influence of the item is smaller, then the probability of retaining the item is smaller (marked as 0); if the sign of the true gradient is -1, the calculation method is the same as the sign of 1, but the opposite effect is produced. In order to restore the influence of the true gradient as much as possible, it is necessary to adjust the amplitude change of each item through the scaling factor. The pseudo gradient calculation formula is:

[0075]

[0076] in, Identify pseudo gradients, Identifies the Hadamard product.

[0077] S6. The central server aggregates the model updates from the clients, accelerates the update of the global model through the Adam optimization algorithm, and returns to step S3. The aggregation formula for the global model update is:

[0078]

[0079] Where n represents the number of clients participating in the training. represents the model update after quantization of the i-th client in the t-th round of training, Represents the approximate pseudo gradient, and the global model formula is updated by the Adam optimization algorithm:

[0080]

[0081] in, represents the corrected first-order moment estimate, represents the corrected second-order moment estimate, ∈, and η represents the parameters that need to be adjusted during training. The adaptive gradient optimization Adam formula is:

[0082]

[0083] S7. Verify the effect of model parameters trained in each epoch through the test data set; the verification indicators are calculation loss and model accuracy. The formula for calculating loss is:

[0084]

[0085] Among them, LOSS avg represents the average loss, n represents the total number of batches in the test data set, Loss i Represents the loss of the i-th batch, batch i represents the number of samples in the i-th batch, labels represents the total number of samples; the expression of accuracy is as follows:

[0086]

[0087] Acc represents the accuracy, C represents the number of samples predicted correctly, and labels represents the total number of samples.

[0088] S8. Repeat the above steps. When the global number of iterations is met, end the training and obtain the final training model.

[0089] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A federated learning ternary quantization method based on momentum acceleration and error feedback, characterized in that: The following steps are involved: The central server initializes the parameters of the model to be trained. In each round of training, a fixed number of clients are randomly selected to receive the global model sent from the central server. After performing multiple steps of stochastic gradient descent locally, these clients calculate the difference Δ between the trained model and the model sent by the central server. The difference Δ is calculated together with the error e through the ternary quantization method to obtain the final compression result. At the same time, the new error generated by the current training round of the client is updated. For those clients that did not participate in this training, the error remains unchanged from the previous round. Finally, the clients participating in the training upload the compression results to the central server, and then the server aggregates these parameters and updates the global model through the momentum acceleration algorithm, and then sends the new global model to each client participating in the next round of training. Multiple rounds are performed to finally obtain the trained target model; Training process The process includes: S1. Get the original data set, divide it into training set and test set, and preprocess the data of the training set: take the first 80% of the data in the data set as the training data set, and in each epoch, shuffle the training data set and distribute it in fixed batch units; take the last 20% of the data as the test data set, and in each epoch, distribute the data in sequence in fixed batch units; S2, initialize the neural network model to be trained, the global iteration round epoch, the global learning rate learning rate, the number of all clients participating in the training num_users, the ratio of the number of clients participating in each round of training frac, the local iteration round local_epoch, the local learning rate local rate and other parameters; S3, the central device sends the global model; S4. Randomly sample clients with a fraction of frac from all clients, and assign training data and test data loaders to the selected clients by index; S5. Each client updates the model locally using the stochastic gradient descent optimizer based on the allocated data, calculates the difference between the local updated model and the model sent by the server, aggregates the model difference and the quantization error (initial 0), uses ternary quantization to aggregate the results to obtain the compressed model, and updates the quantization error of the current round to the difference between before and after quantization. The clients participating in the training upload the quantized model. S6, the central server aggregates the model updates from the clients, accelerates the update of the global model through the Adam optimization algorithm, and returns to step S3; S7. Verify the effect of model parameters trained in each epoch through the test data set; S8. Repeat the above steps. When the global number of iterations is met, end the training and obtain the final training model.

2. According to claim 1, a federated learning ternary quantization method based on momentum acceleration and error feedback is characterized in that: The central server aggregates the difference between the model of each client participating in the training and the model before training, and then averages them to update the global model. The formula is: Among them, W t+1 represents the global model of the t+1th round, represents the difference between the i-th client trained model and the global model before training, represents the model of the ith client after the tth round of training, and n represents the number of clients. In the standard paradigm of federated learning, the central server updates the global model by aggregating the average of the models of each client participating in the training after each round of training. The formula is: It can be proved that 3. According to claim 1, a federated learning ternary quantization method based on momentum acceleration and error feedback is characterized in that: The adaptive gradient optimization Adam formula is: The first-order momentum update rule is: The second-order momentum update rule is: Among them, β1 and β2 represent the hyperparameters that control the decay rate. represents the gradient. The first-order momentum m t Represents the average value of the gradient, which is used to estimate the overall trend of the gradient; the second-order momentum v t Represents the variance of the gradient, which is used to estimate the degree of gradient fluctuation.

4. According to claim 1, a federated learning ternary quantization method based on momentum acceleration and error feedback is characterized in that: The client local gradient descent formula is: Among them, α represents the local learning rate, θ i+1 Represents the local model after the i+1th iteration. Represents the gradient at the i-th iteration.

5. The method for federated learning ternary quantization based on momentum acceleration and error feedback according to claim 1, characterized in that: The loss function formula is: in, represents the true label of the i-th sample, It represents the predicted probability distribution of the model for the i-th sample, and C represents the number of sample classifications in the data set.

6. According to claim 3, a federated learning ternary quantization method based on momentum acceleration and error feedback is characterized in that: The ternary quantization operator formula is: in, represents the quantized gradient, s t Represents the scaling factor, which is used to adjust the size of the quantized gradient, g t represents the original gradient, sign(g t ) means g t The symbol, represents the Hadamard product, b t Indicates that it is signed(g t ) The binary mask matrix of the gradient after processing satisfies the following distribution, and the formula is: Among them, g tk represents the value of the kth item in the gradient matrix, b tk Represents the value of the kth item in the binary mask matrix, in the range of {0, 1}.

7. The method for federated learning ternary quantization based on momentum acceleration and error feedback according to claim 4, characterized in that: The error feedback mechanism formula is: Δ t =W t+1 -W t Among them, for each client, e t Indicates the difference between the last round of model quantization and before quantization, Δ t Indicates the difference between the model updated locally on the client and the model sent by the server. Indicates the model difference after ternary quantization. For clients that did not participate in this round of training, the quantization error update formula is: And t+1 =and t 。 8. The method for federated learning ternary quantization based on momentum acceleration and error feedback according to claim 1, characterized in that: The global model update rule calculation formula is: in, represents the corrected first-order moment estimate, represents the corrected second-order moment estimate, ∈, and η represents the parameters that need to be adjusted during the training process.

Images