A federated learning training method, device, equipment and storage medium

Abstract

The application provides a federated learning training method and device, equipment and a storage medium. The method comprises the following steps: obtaining a teacher sample data set; training a preset agent according to the teacher sample data set, so that the preset agent learns to train a node model by adjusting a model level bit width at different training stages; and deploying the trained preset agent to a plurality of to-be-trained nodes of a federated learning network, so that the trained preset agent trains each to-be-trained node. The application distinguishes the characteristic differences of different levels of the model, and performs bit width quantization accordingly, thereby avoiding the transmission of a large amount of data in the traditional uniform quantization strategy. Furthermore, the model level bit width is dynamically adjusted at different training stages, which can guarantee the minimum requirement of model training under the premise of minimizing the training data volume, and avoids the damage to the training accuracy caused by model compression.

Description

Technical Field

[0001] This application relates to the field of deep learning, and more particularly to a federated learning training method, apparatus, device, and storage medium. Background Technology

[0002] With the increasing importance of data privacy protection, federated learning, as a distributed machine learning paradigm, can effectively solve data security problems in applications such as healthcare and finance that require data privacy protection. Its core goal is to build a federated learning model based on a distributed dataset, in which multiple nodes participate in distributed training and finally aggregate to obtain a global model. The common workflow of federated averaging algorithms is that each node uploads the parameters of its local model to the server, the server calculates a weighted average of the model parameters of each node, and then broadcasts the calculated average back to each node device for repeated iterations until the model converges.

[0003] However, traditional federated learning models have significant limitations. Frequent transmission of model parameters between nodes and servers results in enormous uplink and downlink communication overhead, severely impacting model training efficiency. Therefore, focusing on compressing model updates through techniques such as quantization and sparsity, low-bit training techniques have emerged. This technique uses low-bit integer matrix computation operators to replace floating-point computation operators in traditional convolutional or linear layers during model training.

[0004] Existing low-bit training methods can be broadly categorized into three types. The first type utilizes low-bit floating-point numbers or new data types for training. While this can achieve performance similar to full-precision networks, the speedup is less significant than with integer quantization due to the floating-point type. The second type performs integer quantization on the entire network. Although the quantization level is high, it results in a significant performance loss compared to full-precision networks. The third type quantizes only the convolutional layers within the network. This method can accelerate training while maintaining relatively stable performance, but it doesn't fully consider the differences between different stages of model training and between different layers within the network. Therefore, existing training methods have not fundamentally solved the problem of high communication overhead and are prone to performance degradation. Summary of the Invention

[0005] The purpose of this invention is to provide a federated learning training method, apparatus, device, and storage medium to solve the above-mentioned technical problems.

[0006] This invention provides a federated learning training method, comprising: acquiring a teacher sample dataset, the teacher sample dataset including sample data of model layer bit widths corresponding to multiple nodes at different training stages; training a preset agent based on the teacher sample dataset, so that the preset agent learns to train node models by adjusting the model layer bit width at different training stages; and deploying the trained preset agent to multiple nodes to be trained in the federated learning network, so that the trained preset agent trains each of the nodes to be trained.

[0007] In one embodiment of the present invention, the model layer bit width includes weight bit width, activation bit width, and gradient bit width. The construction of the preset agent includes: constructing an initial neural network; constructing a state space for the initial neural network, the state space including a time step, the current model layer bit width, the quantization error of each layer in the current neural network, and historical model layer bit width change time information; constructing an action space for the initial neural network, the action items in the action space including maintaining bit width, increasing weight bit width, increasing activation bit width, and increasing gradient bit width; setting a reward function for the initial neural network, the reward function being obtained by calculating the product of efficiency reward and model accuracy reward; and determining the initial neural network with constructed state space and action space, and with set reward function, as the preset agent.

[0008] In one embodiment of the present invention, the reward function is obtained by weighted summation of computational efficiency reward and model accuracy reward, including: determining the current computational cost based on the product of the current weight bit width and the current activation bit width, and determining the computational cost reduction rate between the current computational cost and the computational cost of the preset reference bit width, so as to determine the computational efficiency reward based on the computational cost reduction rate, wherein the computational efficiency reward is inversely proportional to the computational cost reduction rate; determining the numerical difference between the accuracy of the preset reference bit width and the current training accuracy at the same training time node as the accuracy reduction amount, so as to determine the model accuracy reward based on the accuracy reduction amount, wherein the model accuracy reward is inversely proportional to the accuracy reduction amount.

[0009] In one embodiment of the present invention, the calculation of the quantization error of each layer in the current neural network includes: obtaining the original weight data and current weight bit width, the original activation data and current activation bit width, and the original gradient data and current gradient bit width of each layer of the neural network; calculating the relative deviation values ​​of the original weight data and current weight bit width, the relative deviation values ​​of the original activation data and current activation bit width, and the relative deviation values ​​of the original gradient data and current gradient bit width in each layer of the neural network respectively; and determining the relative deviation values ​​of the original weight data and current weight bit width, the relative deviation values ​​of the original activation data and current activation bit width, and the relative deviation values ​​of the original gradient data and current gradient bit width in each layer of the neural network as the quantization error of each layer in the current neural network.

[0010] In one embodiment of the present invention, deploying a pre-trained agent to multiple training nodes of a federated learning network, so that the pre-trained agent trains each training node, includes: adjusting the model layer bit width of the multiple training nodes of the federated learning network to a first target bit width to obtain an initialized training node; when the initialized training node reaches a preset trigger condition during the training process, the pre-trained agent adjusts the model layer bit width of the initialized training node, wherein the preset trigger condition includes the quantization error of each layer in the current neural network being greater than or equal to a quantization error threshold, the time step reaching a preset step size adjustment threshold, or the time interval between the current time and the last bit width change in the historical model layer bit width change time information being greater than or equal to a preset time interval threshold.

[0011] In one embodiment of the present invention, the preset agent adjusting the model layer bit width of the initialized training node includes: if the quantization error of each layer in the current neural network is greater than or equal to a quantization error threshold, then increasing at least one of the weight bit width, activation bit width, and gradient bit width of the initialized training node; if the time step reaches a preset step size adjustment threshold, then increasing the activation bit width and gradient bit width; if the time interval between the current time and the last bit width change in the historical model layer bit width change time information is greater than or equal to a preset time interval threshold, then increasing at least one of the activation bit width and gradient bit width.

[0012] In one embodiment of the present invention, after the preset intelligent agent that has completed training trains each of the nodes to be trained, the method further includes: when the nodes to be trained communicate, configuring the weight bit width of the data transmission during the communication process to maintain a first target bit width throughout the training process, or adjusting it from the first target bit width to a second target bit width during the training process, wherein the first target bit width is smaller than the second target bit width.

[0013] This invention also provides a federated learning training device, comprising: a sample data acquisition module for acquiring a teacher sample dataset, the teacher sample dataset including sample data of model layer bit widths corresponding to multiple nodes at different training stages; an agent training module for training a preset agent based on the teacher sample dataset, so that the preset agent learns to train node models by adjusting the model layer bit width at different training stages; and a node training module for deploying the trained preset agent to multiple nodes to be trained in the federated learning network, so that the trained preset agent trains each of the nodes to be trained.

[0014] This invention also provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the federated learning training method as described in any of the above embodiments.

[0015] This invention also provides a computer-readable storage medium storing computer-readable instructions that, when executed by a computer's processor, cause the computer to perform the federated learning training method as described in any of the above embodiments.

[0016] This invention provides a federated learning training method, apparatus, device, and storage medium. It acquires a teacher sample dataset and trains a pre-defined agent based on this dataset. The pre-defined agent learns by adjusting the model layer bit width at different training stages to train node models. The trained pre-defined agent is then deployed to multiple training nodes in the federated learning network, enabling the trained agent to train each node. This application differentiates the characteristics of different model layers and implements targeted bit width quantization, avoiding the large data transmission required by uniform quantization strategies in traditional methods. Furthermore, by dynamically adjusting the model layer bit width at different training stages, it ensures the minimum requirements for model training while minimizing the amount of training data, avoiding the loss of training accuracy due to model compression.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0019] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating a federated learning training method as shown in an exemplary embodiment of this application;

[0021] Figure 3 This is an exemplary embodiment of the present application illustrating a specific federated learning training diagram;

[0022] Figure 4 This is a schematic diagram of a federated learning training device shown in an exemplary embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the structure of a computer system for an electronic device, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0024] The embodiments of the present invention will be described below with reference to the accompanying drawings and specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0025] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0026] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0027] The term "and / or" used in this application describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0028] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0029] Reference Figure 1 As shown, the system architecture may include a computer device 110, training nodes 120, and a server 130. The computer device 110 acquires a teacher sample dataset, which includes sample data of model layer bit widths corresponding to multiple nodes at different training stages. Based on the teacher sample dataset, a pre-defined agent is trained, enabling the agent to learn to adjust the model layer bit width at different training stages for node model training. The trained pre-defined agent is then deployed to multiple training nodes 120 in the federated learning network, allowing the trained agent to train each training node 120. The computer device 110 refers to a computing power support device used to support the program or model implementation environment for the federated learning training method, including but not limited to microcomputers, server clusters, industrial control computers, and cloud virtual machines. The training nodes 120 include at least mobile phones, tablets, personal computers, industrial control computers, and microservers.

[0030] In a schematic manner, computer device 110 acquires a teacher sample dataset and trains a preset agent based on the teacher sample dataset. This allows the preset agent to learn to train node models by adjusting the model layer bit width at different training stages. The trained preset agent is then deployed to multiple training nodes 120 in the federated learning network so that the trained preset agent can train each training node 120. This application distinguishes the characteristic differences of different model layers and implements targeted bit width quantization, avoiding the large amount of data transmission required by the uniform quantization strategy in traditional methods. Furthermore, by dynamically adjusting the model layer bit width at different training stages, the minimum requirements for model training can be guaranteed while minimizing the amount of training data, thus avoiding the loss of training accuracy due to model compression.

[0031] First, it's important to clarify that low-bit training represents weights, activations, and gradients using low-bit integers during model training. This allows low-bit integer matrix operators to replace the floating-point operators in the original convolutional or linear layers. Specifically, during the forward computation, before entering a convolutional or linear layer, the weights and activations are quantized into low-bit integers, which can be represented as follows: Where s is the scaling factor, s = (Mm) / (2^N-1), m and M are the minimum and maximum values ​​after truncation, N is the quantization bit width, z is the equivalent zero point, clamp is the truncation function, and round is the rounding function. f Represents an unquantized floating-point number. This represents the quantized low-order integer. After quantization to low-order integers, matrix multiplication using low-order integers is performed, and the result is then dequantized back to a floating-point number for input to the next layer. During the backward propagation process, symmetric quantization is first used to quantize the gradient to low-order integers. Then, backward propagation is performed with the weights to obtain the gradient of the activation, and with the activation to obtain the gradient of the weight. This is then dequantized back to a floating-point number, and backward propagation continues. In the following embodiments, the weight bit width, activation bit width, and gradient bit width are all represented using low-order integers during model training via low-bit training methods, thus obtaining the weight bit width, activation bit width, and gradient bit width.

[0032] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present application of a federated learning training method, which can be implemented in... Figure 1 It can be executed in the implementation environment described above, but it can also be implemented in other implementation environments. No specific limitations are imposed on the aforementioned implementation environments here. (See also...) Figure 2 As shown, the flowchart of this federated learning training method includes at least steps S210 to S230, which are described in detail below:

[0033] In step S210, the teacher sample dataset is obtained.

[0034] In one embodiment of this application, the aforementioned teacher sample dataset includes sample data of the model layer bit width corresponding to multiple nodes at different training stages.

[0035] Specifically, multiple teacher sample data sequences are collected. Each teacher sample data sequence includes training state information during the training phase and correlation information including the adjustment actions of model layer bit width. Reward information is set for each teacher sample data sequence, including computational efficiency rewards and model accuracy rewards.

[0036] For example, in a specific training process, the time step for the gradient bit width to change to INT8 in the teacher sample dataset is 0.10 to 0.45, the time step for the activation bit width to change to INT8 is 0.50 to 0.90, and the time step for the weight bit width to change to INT8 is 0.50 to 0.95. Compared with random bit width allocation, this teacher sample dataset can achieve a better balance between computational efficiency and model accuracy. In some other embodiments, the weight bit width change also includes keeping INT4 throughout.

[0037] In one embodiment of this application, during each round of federated learning training in the history or sample construction process, the training time progress of nodes, the current bit width of weights and activations, the quantization error of each neural network layer, and the time node of the most recent bit width adjustment are recorded. Sample labels are generated according to an expert strategy. An exemplary sample label generated by the expert strategy includes, but is not limited to, the gradient bit width increasing to the highest bit width when the training progress is 10% to 45%, the activation bit width being dynamically adjusted based on the quantization error when the training progress is 50% to 90%, and the weight bit width being fixed at the lowest bit width throughout the process.

[0038] In step S220, the preset agent is trained based on the teacher sample dataset, so that the preset agent learns to train the node model by adjusting the bit width of the model layer at different training stages.

[0039] In one embodiment of this application, the model layer bit width includes weight bit width, activation bit width, and gradient bit width.

[0040] In one embodiment of this application, the construction of the pre-defined agent includes first constructing an initial neural network. In some specific application environments, an exemplary initial neural network is constructed using a Q-network. A Q-network is a method combining deep learning and reinforcement learning, primarily used to solve decision-making problems in complex environments. It approximates the Q-value function using a deep neural network, thereby enabling it to handle problems in high-dimensional state spaces. A specific example of this initial neural network consists of three linear layers, with 128 neurons in the hidden layers. The number of neurons in the first and last layers corresponds to the number of states in the state space and the number of actions in the action space, respectively.

[0041] In one embodiment of this application, a state space is constructed for the initial neural network, the state space including the time step, the current model layer bit width, the quantization error of each layer in the current neural network, and the historical model layer bit width change time information; and an action space is constructed for the initial neural network, the action items in the action space including maintaining bit width, increasing weight bit width, increasing activation bit width, and increasing gradient bit width.

[0042] The calculation of quantization error for each layer in the current neural network includes obtaining the original weight data and current weight bit width, the original activation data and current activation bit width, and the original gradient data and current gradient bit width for each layer of the neural network. The relative deviations between the original weight data and current weight bit width, the original activation data and current activation bit width, and the original gradient data and current gradient bit width for each layer of the neural network are then calculated. These relative deviations are determined as the quantization error for each layer of the current neural network. In some specific embodiments, the quantization error for each layer of the current neural network is calculated by separately calculating the relative deviations of the first-order magnitude values ​​of the original weight data and current weight bit width, the original activation data and current activation bit width, and the original gradient data and current gradient bit width for each layer of the neural network. Specifically, the raw weight data refers to the original weight observation data of the current weight bit width before quantization, the raw activation data refers to the original activation observation data of the current activation bit width before quantization, and the raw gradient data refers to the original observation data of the current gradient bit width before quantization. It's important to note that after calculating the relative deviations between the raw weight data and the current weight bit width, the relative deviations between the raw activation data and the current activation bit width, and the relative deviations between the raw gradient data and the current gradient bit width for each layer of the neural network, the calculation also includes averaging the results from each layer of the neural network.

[0043] In one embodiment of this application, a reward function is further set for the initial neural network. The reward function includes a computational efficiency reward and a model accuracy reward, which are obtained by multiplying the computational efficiency reward and the model accuracy reward. Specifically, the current computational cost is determined based on the product of the current weight bit width and the current activation bit width, and the computational cost reduction rate between the current computational cost and the computational cost of a preset reference bit width is determined. The computational efficiency reward is determined based on the computational cost reduction rate, and the computational efficiency reward is inversely proportional to the computational cost reduction rate. The numerical difference between the accuracy of the preset reference bit width and the current training accuracy at the same training time node is determined as the accuracy reduction amount. The model accuracy reward is determined based on the accuracy reduction amount, and the model accuracy reward is inversely proportional to the accuracy reduction amount.

[0044] The calculation cost of the preset reference bit width is determined by multiplying the weight bit width, activation bit width, and gradient bit width under full training conditions, such as INT8 bit width; the accuracy of the preset reference bit width is the model training accuracy recorded under full training conditions, where the weight bit width, activation bit width, and gradient bit width are all the same.

[0045] Specifically, the reward function `reward` can be expressed as `reward = q_idx * a - idx * t - step`, where `q_idx` is the computational efficiency reward, `a - idx` is the model accuracy reward, and `t - step` is the current time step. The computational efficiency reward can be expressed as `q - idx = θ / (1 - λ)`, where `λ` is the rate of reduction in computational cost between the current computational cost and the computational cost of the preset reference bit width, and `θ` is an adjustable hyperparameter, sometimes set to 0.25 in specific embodiments. The model accuracy reward can be expressed as `a_idx = 1 / (1 + (a...)`. r -a)*δ), where a r A preset reference bit width accuracy is set for the same training time node, where 'a' is the current training accuracy at the same training time node, and δ is an adjustable hyperparameter, which is set to 0.5 in some specific embodiments.

[0046] In one embodiment of this application, an initial neural network with a constructed state space and action space, and a set reward function, is determined as a preset intelligent agent.

[0047] In some embodiments, the pre-defined agent is trained on a collected teacher sample dataset through supervised learning, and the training loss is calculated by the sum of the reward difference between the agent policy and the teacher samples and the KL divergence of the output distribution. By utilizing the teacher sample dataset in behavior cloning, a pre-defined agent can be trained quickly.

[0048] In step S230, the trained preset agent is deployed to multiple training nodes of the federated learning network so that the trained preset agent can train each training node.

[0049] In one embodiment of this application, the model layer bit width of multiple nodes to be trained in a federated learning network is adjusted to a first target bit width to obtain an initialized node to be trained. When the initialized node to be trained reaches a preset trigger condition during the training process, a preset agent adjusts the model layer bit width of the initialized node to be trained. The preset trigger condition includes the quantization error of each layer in the current neural network being greater than or equal to a quantization error threshold, the time step reaching a preset step size adjustment threshold, or the time interval between the current time and the last bit width change in the historical model layer bit width change time information being greater than or equal to a preset time interval threshold.

[0050] In one embodiment of this application, if the quantization error of each layer in the current neural network is greater than or equal to the quantization error threshold, then at least one of the weight bit width, activation bit width and gradient bit width of the initial node to be trained is increased.

[0051] In one embodiment of this application, if the time step reaches a preset step adjustment threshold, the activation bit width and gradient bit width are increased.

[0052] In one embodiment of this application, if the time interval between the current time and the last bit width change in the historical model level bit width change time information is greater than or equal to a preset time interval threshold, then at least one of the activation bit width and gradient bit width is increased.

[0053] In one embodiment of this application, when the node to be trained communicates, the weight bit width of the data transmission during the communication process is configured to maintain a first target bit width throughout the training process, or to adjust from the first target bit width to a second target bit width during the training process, wherein the first target bit width is smaller than the second target bit width. In a specific embodiment, the first target bit width is INT4, and the second target bit width is INT8.

[0054] It should be noted that in the above embodiments, for example, in the bit width representations such as INT4 and INT8, INT (Integer) means integer. For example, INT4 represents a four-byte integer data type, while INT8 represents an eight-byte integer data type. In the above embodiments, the numerical examples of bit widths such as INT4 and INT8 are merely examples of implementable numerical values ​​in the embodiments of this application, and do not limit the numerical representation and substitution of bit widths in the specific implementation process of this application.

[0055] In one embodiment of this application, when the trained preset agent changes to a new federated learning training scenario, i.e. a new model or a new dataset, the trained preset agent can be directly applied to the new federated learning training scenario, or a small portion of the local dataset can be used to update and optimize the trained preset agent through reinforcement learning.

[0056] In some feasible embodiments, the first target bit width can be set to INT4. During training of the node to be trained, the weights and activation bit width are adjusted according to the training state, gradually increasing from INT4 to a maximum of INT8. In other embodiments, the gradient bit width reaches a higher bit width of INT8 early in training, the activation bit width is adjusted according to the training state (including quantization error) during training, while the weight bit width remains at INT4. Therefore, the weights for both uplink and downlink communication are represented by INT4, reducing communication overhead to 1 / 8 of that for full-precision training.

[0057] In some other embodiments of this application, in addition to the above-mentioned training of node models by adjusting the bit width of model layers at different training stages using a preset agent trained through reinforcement learning, the method also includes directly adjusting the bit width of each layer in the neural network of each node to be trained based on sensitivity metrics. Specifically, the loss function perturbation introduced into each layer of the neural network in each node to be trained is subjected to Taylor expansion to calculate the sensitivity metrics of weights, activations, and gradients. After each preset period, the sensitivity metrics of weights, activations, and gradients of each layer in the neural network of each node to be trained are sorted based on sensitivity, and the quantization bit width of high-sensitivity layers is increased, while the bit width of low-sensitivity layers remains low.

[0058] The following describes a specific embodiment of this application; please refer to it. Figure 3 , Figure 3 This is an exemplary embodiment of the present application illustrating a specific federated learning training diagram, with reference to... Figure 3 As shown, the specific federated learning training process includes a first step, Agent Training, which trains a pre-defined agent. This training is conducted offline. In this specific embodiment, the training sample data comes from a locally constructed sample database. 10% of the data is randomly sampled as the training dataset to train the pre-defined agent, resulting in a trained agent. The specific federated learning training process also includes a second step, Model Training, which deploys the trained agent to multiple training nodes in the federated learning network. At least one of the weight bit width, activation bit width, and gradient bit width of the training nodes is dynamically adjusted. When each training node communicates with the server or cloud, the weight bit width is used as the uplink and downlink communication bit width. Specifically, each training node transmits the weight bit width as the uplink and downlink communication bit width representation to the server or cloud, performs aggregation with the server or cloud, re-widths the data, and then sends it back to each training node. The specific training model is a deep residual network ResNet-20, with 100 training epochs and 200 rounds of teacher samples collected, i.e., a total of 200 training iterations.

[0059] This invention provides a federated learning training method, apparatus, device, and storage medium. It acquires a teacher sample dataset and trains a pre-defined agent based on this dataset. The pre-defined agent learns by adjusting the model layer bit width at different training stages to train node models. The trained pre-defined agent is then deployed to multiple training nodes in the federated learning network, enabling the trained agent to train each node. This application differentiates the characteristics of different model layers and implements targeted bit width quantization, avoiding the large data transmission required by uniform quantization strategies in traditional methods. Furthermore, by dynamically adjusting the model layer bit width at different training stages, it ensures the minimum requirements for model training while minimizing the amount of training data, avoiding the loss of training accuracy due to model compression.

[0060] Experimental data obtained after specific testing shows that, compared to existing federated learning methods, this reduces the computational cost during training with limited dropout rates. Based on the classic convolutional neural network LeNet, the number of nodes to be trained is set to 100, with 2000 training epochs. In each epoch, 10% of the nodes to be trained are randomly selected for specific updates and weight uploads. For other models such as ResNet-18, ViT-S, or MobileNet-V2, the number of nodes to be trained is 10, with 200 training epochs. Each node to be trained is updated in each epoch, with a local update epoch of 2 for each node and a learning rate decay coefficient of 1. It is clear that, compared to training in a full-scale training state (where the weight bit width, activation bit width, and gradient bit width are all INT8 bits), the proposed solution can reduce the computational cost by more than 50% while maintaining a performance loss of less than 1.5%.

[0061] The following describes an apparatus embodiment of this application, which can be used to execute the federated learning training method in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the federated learning training method described above.

[0062] Figure 4 This is a schematic diagram illustrating a federated learning training device as an exemplary embodiment of this application. The device can be applied to… Figure 2 The method implementation process shown can be based on the device Figure 1 The implementation environment shown can be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0063] like Figure 4 As shown, the exemplary federated learning training device includes: a sample data acquisition module 401, an agent training module 402, and a node training module 403.

[0064] The sample data acquisition module 401 is used to acquire a teacher sample dataset, which includes sample data of model layer bit width corresponding to multiple nodes at different training stages; the agent training module 402 is used to train a preset agent based on the teacher sample dataset, so that the preset agent learns to train the node model by adjusting the model layer bit width at different training stages; the node training module 403 is used to deploy the trained preset agent to multiple nodes to be trained in the federated learning network, so that the trained preset agent trains each node to be trained.

[0065] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the federated learning training method provided in the above embodiments.

[0066] Figure 5 This is a schematic diagram illustrating the structure of a computer system for an electronic device, as shown in an exemplary embodiment of this application. It should be noted that... Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0067] like Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on a program stored in Read-Only Memory (ROM) 502 or a program loaded from storage into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus. An I / O interface 505 is also connected to the bus 504, where the I / O interface 505 refers to an input / output interface.

[0068] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0069] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0070] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0071] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0072] In the corresponding figures of the above embodiments, connecting lines can represent the connection relationship between various components, indicating more constitutive signal paths and / or one or more ends of some lines having arrows to indicate the main information flow direction. Connecting lines serve as an identifier and are not a limitation on the scheme itself, but rather, using these lines in conjunction with one or more exemplary embodiments helps to more easily connect circuits or logic units. Any signal represented (determined by design requirements or preferences) can actually include one or more signals that can be transmitted in any direction and can be implemented in any suitable type of signal scheme.

[0073] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0074] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0075] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements a federated learning training method as described in any of the above embodiments.

[0076] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0077] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0078] It should be noted that this application can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0079] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0080] It should be understood that the above content of this application is only a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be the scope of protection claimed in the claims.

Claims

1. A federated learning training method, characterized in that, The federated learning training method includes: Obtain a teacher sample dataset, which includes sample data of the model layer bit width corresponding to multiple nodes at different training stages; The preset agent is trained based on the teacher sample dataset so that the preset agent learns to train the node model by adjusting the model layer bit width at different training stages. The pre-trained agent is deployed to multiple nodes in the federated learning network so that the pre-trained agent can train each node.

2. The federated learning training method according to claim 1, characterized in that, The model layer bit width includes the weight bit width, activation bit width, and gradient bit width. The construction of the preset agent includes: Construct the initial neural network; A state space is constructed for the initial neural network, the state space including the time step, the current model layer bit width, the quantization error of each layer in the current neural network, and the time information of the historical model layer bit width changes; An action space is constructed for the initial neural network, and the action items in the action space include maintaining bit width, increasing weight bit width, increasing activation bit width, and increasing gradient bit width; A reward function is set for the initial neural network, the reward function including a computational efficiency reward and a model accuracy reward, the reward function being obtained by multiplying the computational efficiency reward and the model accuracy reward; The initial neural network with its state space and action space already constructed and its reward function already set is identified as the preset agent.

3. The federated learning training method according to claim 2, characterized in that, The reward function is obtained by weighted summation of efficiency reward and model accuracy reward, including: The current computation cost is determined based on the product of the current weight bit width and the current active bit width, and the computation cost reduction rate between the current computation cost and the computation cost of the preset reference bit width is determined. A computation efficiency reward is determined based on the computation cost reduction rate, and the computation efficiency reward is inversely proportional to the computation cost reduction rate. The difference between the accuracy of the preset reference bit width and the current training accuracy at the same training time node is determined as the accuracy reduction amount. The model accuracy reward is determined based on the accuracy reduction amount, and the model accuracy reward is inversely proportional to the accuracy reduction amount.

4. The federated learning training method according to claim 2, characterized in that, The calculation of quantization error at each layer in the current neural network includes: Obtain the original weight data and current weight bit width, the original activation data and current activation bit width, and the original gradient data and current gradient bit width for each layer of the neural network; Calculate the relative deviation between the original weight data and the current weight bit width, the relative deviation between the original activation data and the current activation bit width, and the relative deviation between the original gradient data and the current gradient bit width in each layer of the neural network. Determine the relative deviation between the original weight data and the current weight bit width, the relative deviation between the original activation data and the current activation bit width, and the relative deviation between the original gradient data and the current gradient bit width in each layer of the neural network as the quantization error of each layer in the current neural network.

5. The federated learning training method according to claim 1, characterized in that, Deploying the trained preset agent to multiple training nodes in the federated learning network, so that the trained preset agent can train each of the training nodes, includes: The model layer bit width of multiple nodes to be trained in the federated learning network is adjusted to the first target bit width to obtain the initialized nodes to be trained. When the initial training node reaches a preset trigger condition during the training process, the preset agent adjusts the model layer bit width of the initial training node. The preset trigger condition includes the quantization error of each layer in the current neural network being greater than or equal to the quantization error threshold, the time step reaching a preset step size adjustment threshold, or the time interval between the current time and the last bit width change in the historical model layer bit width change time information being greater than or equal to a preset time interval threshold.

6. The federated learning training method according to claim 5, characterized in that, The preset agent adjusts the bit width of the model layer of the initialized training node, including: If the quantization error of each layer in the current neural network is greater than or equal to the quantization error threshold, then increase at least one of the weight bit width, activation bit width and gradient bit width of the initialized training node. If the time step reaches the preset step adjustment threshold, the activation bit width and gradient bit width will be increased. If the time interval between the current time and the last bit width change in the historical model level bit width change time information is greater than or equal to the preset time interval threshold, then increase at least one of the activation bit width and gradient bit width.

7. The federated learning training method according to any one of claims 1-6, characterized in that, After the pre-trained agent has trained each of the nodes to be trained, the training process further includes: When the node to be trained communicates, the weight bit width of the data transmission during the communication process is configured to maintain the first target bit width throughout the training process, or to be adjusted from the first target bit width to the second target bit width during the training process, wherein the first target bit width is smaller than the second target bit width.

8. A federated learning training device, characterized in that, The federated learning training device includes: The sample data acquisition module is used to acquire the teacher sample dataset, which includes sample data of the model layer bit width corresponding to multiple nodes at different training stages. The agent training module is used to train a preset agent based on the teacher sample dataset, so that the preset agent learns to train the node model by adjusting the model layer bit width at different training stages. The node training module is used to deploy the trained preset agent to multiple training nodes in the federated learning network, so that the trained preset agent can train each of the training nodes.

9. An electronic device, characterized in that, It includes a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the federated learning training method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that enables the computer to perform the federated learning training method as described in any one of claims 1-7.

Images