Method and system for training a distributed model

Abstract

The present disclosure provides computer-implemented method for training a model using a central node and a plurality of client nodes, comprising, at one or more of the plurality of client nodes: receiving a set of model parameters and an initial seed from the central node; determining an update direction based on the initial seed; computing scalar reflecting a sign and a size of an approximation of a projection of a gradient of a loss function on the update direction, using the set of model parameters; sending the scalar of the approximated gradient projection to the central node; receiving a model update scalar from the central node; and updating the set of model parameters in the update direction based on the model update scalar.

Description

METHOD AND SYSTEM FOR TRAINING A DISTRIBUTED MODEL TECHNICAL FIELD

[0001] The present disclosure is in the field of machine learning, specifically a method andsystem for training a model using a central node and a plurality of client nodes. BACKGROUND

[0002] Federated Learning is a distributed machine learning approach where a sharedmodel is trained across multiple devices or servers, each holding their own local training data, and a central server, called the federator. Rather than transferring the training data itself, these devices (called clients) collaboratively update and improve a central model by exchanging model parameters with the federator. Federated Learning (FL) faces several challenges, for example, security against Byzantine clients and communication efficiency.

[0003] Security against Byzantine clients, i.e., clients deliberately corrupting theircomputation to disrupt the process, is paramount. Only one Byzantine client inserting errors can be enough to prevent the learning algorithm from converging. Furthermore, to train the distributed model, clients can run backpropagation on their local data and communicate a d- dimensional vector (usually d is in the order of 106-109) to the federator, making communication and computation two crucial bottlenecks, especially in settings where the clients are edge devices with limited computation and communication capabilities.

[0004] McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A.“Communication-Efficient Learning of Deep Networks from Decentralized Data” presents a practical method for the federated learning of deep networks based on iterative model averaging, and conduct an extensive empirical evaluation, considering five different model architectures and four datasets.

[0005] Yin, D., Chen, Y., Kannan, R., and Bartlett, P. “Byzantine - Robust DistributedLearning: Towards Optimal Statistical Rates” develops distributed optimization algorithms that are provably robust against Byzantine failures—arbitrary and potentially adversarial behavior, in distributed computing systems, with a focus on achieving optimal statistical performance. SUMMARY

[0006] According to a first aspect of the present disclosure, a computer-implementedmethod for training a model using a central node and a plurality of client nodes includes, atone or more of the plurality of the client nodes: receiving a set of model parameters and an initial seed from the central node; determining an update direction based on the initial seed; computing a scalar reflecting a sign and a size of an approximation of a projection of a gradient of a loss function on the update direction, using the set of model parameters; sending the scalar of the approximated gradient projection to the central node; receiving a model update scalar from the central node; and updating the set of model parameters in the update direction based on the model update scalar.

[0007] The method can be used in any kind of distributed scenarios, regardless of theownership of data and devices being centralized or decentralized, as long as the training topography is the same - a central node and a set of client (worker) nodes. The distributed scenario can be a federated scenario. The presented method addresses various concerns related to privacy, security, and data centralization.

[0008] The central node, also referred to as the federator or central server in federatedlearning, can orchestrate the learning process. It can initiate the training by sending the current model to the client nodes and aggregating the updates received from a subset of all of them. The central node can be responsible for ensuring the integrity and efficiency of the learning process, implementing mechanisms to optimize communication and / or protect privacy.

[0009] The plurality of client nodes may be the decentralized entities that hold their ownlocal training data. These nodes can perform computations on their training data and share model updates, in the form of gradients, rather than the data itself. This approach can significantly enhance privacy and security, as sensitive information cannot leave the client node.

[0010] An initial phase of this method includes a first step of receiving a set of modelparameters and an initial seed from the central node. The set of model parameters may reflect characteristics of the model. For example, if the model is a neural network model, the set of model parameters may comprise the weights and biases of the set of model parameters. The seed may be shared between the central node and client nodes. The seed may be a random seed, including but not limited to a random piece of text or a random number that can be used to synchronize the pseudo-random number generators (PRNG) across client nodes and the central node. Preferably, central node and client nodes have a same PRNG. The random seed thus allows the central node and client nodes to have the same set of random values without needing to transmit the random values to each other.

[0011] The initial step needs to be executed only once in a training procedure. At this step,the model parameters can be transmitted in full. However, as this is a step to be taken only once throughout the entire process, the impact on overall transmission load is small. To further save communication costs during the initialization phase, it is possible to generate the same random initializations at the federator and the clients’ by using a common seed. In that way,the same random weights can be generated and only the architecture of the model has to be communicated.

[0012] The initialization step is followed preferably by an iterative execution of the trainingsteps, which can be referred to as training phase. The update direction is determined based on the initial seed. After receiving the initial seed, a client node can set the initial seed as the seed in its own PRNG. In some cases, if steps of the training phase are iterative, the client node can construct a current seed for each iteration. For example, the current seed can be the initial seed followed by the current repetition number and the number of scalars to be communicated. With the same initial number, a current seed constructed by each client node can stay unchanged for each iteration. Thus, values sampled from each client node’s PRNG can be the same. A vector (i.e., the update direction) can be determined.

[0013] After determining the update direction, the client nodes can compute a scalarreflecting a sign and a size of an approximation of a projection of a gradient of a loss function on the update direction. Specifically, the size may be the norm of the approximated projection of the gradient and the sign is positive if the projection is in the same direction as the update direction and negative if the projection is in the opposite direction as the update direction. The federator can determine a learning rate which determines how much to adjust the model parameters in a specific update direction based on the aggregated scalar (see below). For example, the Adam optimizer may be used. Each participating client node may send the scalar it computed to the central node and receive the model update scalar from the central node to update the set of model parameters. In preferred embodiments, a client node can run multiple computations in multiple update directions and transmit the corresponding multiple scalars to the federator in one message.

[0014] In this phase, the central node can utilize the scalars sent by the client nodes toupdate the model parameters of the model. Specifically, there may be a plurality of update directions. The central node can also transmit back the updated model to the client nodes so that these steps can be repeated. In particular, these steps can be repeated a number of times, until the model achieves good accuracy or until one or more convergence criteria are met.

[0015] The presented method can allow client nodes to send approximations of theirgradients by using random linear projections. High communication efficiency can be achieved by utilizing a shared seed concept. As a result, client nodes only need to send k real numbers (scalars), for a parameter k > 0, instead of high-dimensional vectors. In addition, this can be coupled with a robust aggregation technique specifically tailored to the zero-order optimization context. Furthermore, this method can achieve a high accuracy in the presence of Byzantine clients.

[0016] In a further implementation of the method according to the first aspect, thedetermining the update direction based on the initial seed may include constructing a current seed for each iteration during training the model based on the initial seed; designating thecurrent seed for a pseudo-random number generator, PRNG; sampling a PRNG result from the PRNG; and generating the update direction based on the PRNG result.

[0017] In a further implementation of the method according to the first aspect, the currentseed may be the same in each iteration during training the model.

[0018] A single seed value can be transmitted from the central node to the client nodes atthe start of the initial phase. During each iteration, the current seed can be the same. This enables every client node to independently reconstruct update directions, dismissing the need for direct transmission from the central node. The seed can also be used to synchronize the pseudo-random number generation (PRNG) across client nodes and the central node. That allows the central node and client nodes to generate the same set of random values without needing to transmit the information to each other. The use of a shared seed can achieve a much better communication efficiency.

[0019] Furthermore, different variants of PRNGs (apart from uniform) can be suited forgenerating the random projections. There is no reason to limit this approach to a specific random number generator. Which randomness is most suitable can be application-specific and depends on the dataset and model used, although the proposed method may always be a preferred example.

[0020] In a further implementation of the method according to the first aspect, thecomputing the scalar of the approximation of the projection of the gradient of the loss function on the update direction may include determining the gradient using backpropagation; and projecting the gradient on the update direction to obtain the scalar of the approximation of the gradient projection.

[0021] This training method is capable of reducing communication between the centralnodes and the client nodes, and vice-versa, by sending only a limited number of scalars per iteration. This method can use backpropagation, but project the calculated gradient on the update direction generated by the current seed to obtain the scalar of the approximation of the gradient projection. The result can be a scalar, which can be the only information necessary to be communicated. To achieve better performances, this step can be repeated for multiple random projections (^^) in a plurality of update directions, hence more than one scalar to be transmitted. While the gradient projection approach requires to calculate the true gradientthrough backpropagation, it is more computationally efficient if ^^ is big.

[0022] In a further implementation of the method according to the first aspect, thecomputing the scalar of the approximation of the projection of the gradient of the loss function on the update direction may include evaluating the loss function using the set of model parameters perturbed into the update direction.

[0023] In a further implementation of the method according to the first aspect, theevaluating the loss function using the set of model parameters perturbed into the updatedirection may include perturbing the set of model parameters; updating the set of model parameters to a set of perturbed model parameters; calculating a first perturbed value and a second perturbed value based on the set of perturbed model parameters; and determining the approximated gradient of the loss function in the update direction based on the first perturbed value and the second perturbed value.

[0024] The client nodes can compute an approximation of the true gradient by randomperturbations of the model. The random perturbations are sampled by the client nodes at each iteration and used to compute an approximation of the true gradient, update the local model, and send the updated model to the central node.

[0025] This approach (e.g., Zero-order Optimization) can be pursued if computing thegradients is impossible, e.g., in black-box optimization or when the objective is not differentiable. The versatility of zero-order optimization as a memory-efficient technique allows thefine-tuning of billion-parameter models, while only using a fraction of the memory required by thefirst-order counterparts. The federated counterpart can be made efficient in terms of communication while effectively resisting to Byzantine behavior of a fraction of the clients.

[0026] In addition, this approach may not need backpropagation. Backpropagationrequires more memory resources compared to the approximation approach where nobackpropagation is required. The ^^ scalars are obtained through model perturbations in arandom direction instead of the projection onto the true gradient.

[0027] In a further implementation of the method according to the first aspect, the modelmay include at least one of Linear Regression, Logistic Regression, Support Vector Machine, Decision Tree, Ensemble Methods, Generalized Linear Model, Neural Networks, Deep Learning Model, and fine-tuning Large Language Models.

[0028] The method can be utilized for the training of any machine learning model thatrelies on gradient descent, this includes Neural Networks, Linear Regression, Logistic Regression, Support Vector Machines (SVMs), Decision Trees and Ensemble Methods (like Gradient Boosting), Generalized Linear Models (GLMs), Deep Learning Models (beyond standard neural networks, like CNNs and RNNs), and / or fine-tuning of Large Language Models.

[0029] According to a second aspect of the present disclosure, a computer-implementedmethod for orchestrating training a model using a central node and a plurality of client nodes, includes, at the central node: sending a set of model parameters and an initial seed to the plurality of client nodes; and iteratively executing, until a preset condition is satisfied. The iteratively executing, until a preset condition is satisfied, includes receiving a plurality of scalars of approximated gradient projections from the plurality of client nodes; aggregating them to obtain a model update scalar; and sending the model update scalar to the plurality ofclient nodes.

[0030] The method can be adapted to account for partial client node participation. In somecases, the learning algorithm may not suffer from the occasional participation of client nodes instead of each client node computing a partial gradient at each iteration. While this approach often may not hurt the performance of the model, it can bring significant communication savings on top of the efficiency provided by the situation with participation of each client node. To ensure this, the central node can keep track of all gradients a client node needs to update its local model to the global state after it did not contribute for a certain number of global iterations. More specifically, the central node can send the averaged gradients for each iteration and each projection together with the information of which of those belong to which iteration. This information may be sufficient to bring all client nodes up to speed in every iteration, even if some of them did not participate in any number of preceding iterations.

[0031] In a further implementation of the method according to the second aspect, themethod may further include computing the updated model at the central node at every iteration or at the end.

[0032] In a further implementation of the method according to the second aspect, themethod may further include determining the update direction of the model based on the initial seed. The determining the update direction of the model based on the initial seed may include constructing a current seed for each iteration during training the model based on the initial seed; designating the current seed for a pseudo-random number generator, PRNG; sampling a PRNG result from the PRNG; and generating the update direction based on the PRNG result.

[0033] In a further implementation of the method according to the second aspect, thecurrent seed may be the same in each iteration during training the model.

[0034] A single seed value can be transmitted from the central node to the client nodes atthe start of the initial phase. During each iteration, the current seed can be the same. This enables every client node to independently reconstruct update directions, dismissing the need for direct transmission from the central node. The seed can also be used to synchronize the pseudo-random number generation (PRNG) across client nodes and the central node. That allows the central node and client nodes to generate the same set of random values without needing to transmit the information to each other. The use of a shared seed can achieve a much better communication efficiency.

[0035] Furthermore, different variants of PRNGs (apart from uniform) can be suited forgenerating the random projections. There is no reason to limit this approach to a specific random number generator. Which randomness is most suitable can be application-specific and depend on the dataset and model used, although the proposed method may always be a preferred example.

[0036] In a further implementation of the method according to the second aspect, theaggregating the plurality of the scalars of the approximated gradient projections to obtain the model update scalar may be achieved by an aggregation rule.

[0037] The aggregation rule may include but not be limited to the trimmed mean. Thischoice of function can be made more general, in the sense that any kind of robust aggregation function can be used to replace the trimmed mean with no conceptual loss. The aggregation rules can be used to ensure robustness against client nodes who adversarially deviate from the described protocol.

[0038] In a further implementation of the method according to the second aspect, thepreset condition may include convergence of the model and / or a desired performance metric.

[0039] The iteration may continue until the model achieves good accuracy. For example,steps repeat until the model converges and / or meets the desired performance metrics.

[0040] According to a third aspect of the present disclosure, a client node is provided. Theclient node is configured to perform the method of any implementation according to the first aspect.

[0041] According to a fourth aspect of the present disclosure, a central node is provided.The central node is configured to perform the method of any implementation according to the second aspect.

[0042] According to a fifth aspect of the present disclosure, a distributed machine learningsystem includes: a plurality of client nodes, wherein the plurality of client nodes is configured to perform the method of any implementation according to the first aspect; and a central node, wherein the central node is configured to perform the method of any implementation according to the second aspect.

[0043] In a further implementation of the system according to the third aspect,communication between the plurality of client nodes and the central node may be secured by end-to-end encryption.

[0044] The end-to-end (public key) encryption can be applied between each client and thefederator to ensure privacy of all communication against an eavesdropper.

[0045] According to a sixth aspect of the present disclosure, a computer-readable storagemedium storing program code is provided. The program code includes instructions that when executed by a processor carry out the method of any implementation according to the first aspect or the second aspect.

[0046] Additional features will be set forth in part in the description which follows, and inpart will become apparent to those skilled in the art upon examination of the following and the accompanying drawings or may be learned by production or operation of the examples. The features of the present disclosure may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities, and combinations set forth in the detailed examples discussed below.BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present disclosure is further described in terms of exemplary embodiments.These exemplary embodiments are described in detail with reference to the drawings. The drawings are not drawn to scale. These embodiments are non-limiting exemplary embodiments, in which like reference numerals represent similar structures throughout the several views of the drawings, and wherein: FIG. 1 is a schematic diagram of an exemplary Neural Network Model according to some embodiments of the present disclosure; FIG. 2 is a schematic diagram of an exemplary federated learning process according to some embodiments of the present disclosure; FIG. 3 is a flow chart illustration a process for training the model according to some embodiments of the present disclosure; FIG. 4 is a schematic diagram of an exemplary iteration of communication-efficient and byzantine-resilient federated learning using the proposed procedure with 3 clients according to some embodiments of the present disclosure;FIG. 5a is a graph illustrating impact of sample size ^^ over the communication cost accordingto some embodiments of the present disclosure;FIG. 5b is a graph illustrating impact of sample size ^^ over the number of steps according tosome embodiments of the present disclosure; FIG. 5c is a graph illustrating effect of Byzantine client behaviors according to some embodiments of the present disclosure; FIG.6 is a graph illustrating effect of Byzantine clients on the convergence speed of CYBER-0 according to some embodiments of the present disclosure; FIG.7 shows a graph contrasting Byzantine and Non-Byzantine Scenarios Across Diverse Data Distributions with RoBERTa-large on TREC and SNLI according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0048] The following description is presented to enable any person skilled in the art tomake and use the present disclosure and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown but is to be accorded the widest scope consistent with the claims.

[0049] The terminology used herein is for the purpose of describing particular exampleembodiments only and is not intended to be limiting. As used herein, the singular forms “a”, “an”, and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprise”, “comprises” and / or “comprising”, “include”, “includes”, and / or “including” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0050] These and other features, and characteristics of the present disclosure, as well asthe methods of operation and functions of the related elements of structure and the combination of portions and economies of manufacture, may become more apparent upon consideration of the following description with reference to the accompanying drawings, all of which form a part of this disclosure. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended to limit the scope of the present disclosure. It is understood that the drawings are not to scale.

[0051] The flowcharts used in the present disclosure illustrate operations that systemsimplement according to some embodiments of the present disclosure. It is to be expressly understood, the operations of the flowchart may be implemented not in order. Conversely, the operations may be implemented in inverted order, or simultaneously. Moreover, one or more other operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0052] In order to make the purpose, technical solution, and advantages of the presentdisclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure.

[0053] FIG. 1 is a schematic diagram of an exemplary Neural Network Model according tosome embodiments of the present disclosure. Neural networks are a cornerstone of artificial intelligence, inspired by the biological neural networks that constitute animal brains. Neural networks are computational models designed to recognize patterns in data. They are capable of learning and making decisions or predictions based on input data.

[0054] The architecture of a neural network refers to its structure, specifically how neurons(also known as nodes) are organized. A typical neural network comprises an input layer, one or more hidden layers, and an output layer. The input layer can receive the raw input data, such as x1 and x2 in FIG.1. Each neuron in the input layer can represent a feature of the input data. The hidden layers are layers between the input and output layers. For example, there are three hidden layers in FIG.1. They can perform computations and transformations on theinputs received from the previous layer. Each hidden layer can receive a weighted sum ^^ of theinput data, and transform ^^ to generate its output by an activation function ^^ in FIG.1. Thecomplexity and capacity of a neural network to learn can be largely determined by the number and size of its hidden layers. The output layer can provide the output of the neural network,such as the Predication ^^′ in FIG.1. The output can represent a classification label, acontinuous value, or any other form of prediction.

[0055] Weights and biases can be the parameters of a neural network. They can play acritical role in the network’s ability to make accurate predictions. Weights are values that scale the input data. They can determine the importance of each input in the computation performed by the neurons. Biases are values that are added to the weighted input before applying the activation function. Biases can help the model make better fits to the data by adjusting the output along the value axis. Both the weights and the biases can be combined in a model vector ^^ describing the entire neural network state.

[0056] Training a neural network can involve adjusting its weights and biases based on thedifferences between the actual output and the desired output, such as the Predictions ^^′ andTrue Values ^^. This process can use a dataset of known inputs and outputs to teach the networkhow to make predictions. Training can occur over multiple iterations, or epochs, through a process called gradient descent. The goal is to minimize the error, or loss, which measures the difference between the predicted output and the actual output. By minimizing the loss, the neutral network can learn to make more accurate predictions.

[0057] Backpropagation is a key mechanism in training neural networks. It can involve thecalculation of the gradient of the loss function with respect to each weight in the network, which can inform how the model should adjust its weights to minimize the loss. The process can be summarized in the following steps: 1. Forward Propagation: The input data is passed through the network, layer by layer, until theoutput layer is reached. At each neuron, the weighted sum ^^ of the inputs is calculated andpassed through an activation function ^^.2. Loss Calculation: The output of the network (e.g., the Predictions ^^′) is compared to thedesired output (e.g., the True Values ^^), and the difference is quantified using a loss function,such as the mean squared error (MSE) for example: L = 1N∑Ni=1 (yi − y^i)2 , where ^^ is the loss, ^^is the number of samples, ^^^^ is the actual value, and ^^^^^ is the predicted value.3. Backward Propagation: The gradient of the loss function with respect to each weight can be computed to determine how the weights need to be adjusted.can be done by applying thechain rule of calculus, which can be expressed as: ∂^^ ∂^^ ∂^^^ ∂^^∂^^=∂^^^⋅∂^^⋅∂^^, where ^^ is the weighted sumof inputs to a neuron, and ^^ represents the weights. The partial derivative of the loss w.r.t. amodel parameter is referred to as a gradient ^^. All gradients of the model parameters can bestacked to obtain the gradient vector gi, similar to the model vector ^^. Both vectors have thesame dimension. 4. Weight Update: Finally, the weights can be updated in the opposite direction of the gradientto minimize the loss. The update rule is given by: ^^new = ^^old − η ⋅∂^^ ∂^^, where ^^ is the learningrate, a small positive scalar determining the size of the step taken in the direction opposite to the gradient.

[0058] By iteratively applying these steps, the neural network can learn to adjust itsweights and biases to minimize the error between its predictions and the actual outputs, effectively learning from the training data. Thus, neural networks are powerful tools for pattern recognition and prediction. Their effectiveness depends on their architecture, the adjustment of weights and biases, and the processes of training and backpropagation. By learning from data, neural networks can make complex decisions, making them invaluable in a wide range of applications.

[0059] FIG.2 is a schematic diagram of an exemplary federated learning process accordingto some embodiments of the present disclosure. Federated Learning is a distributed machine learning approach that enables a model to be trained across multiple devices or servers, each holding their own local data, and a central server, called the federator. Rather than transferring the data itself, the devices (called clients) collaboratively update and improve a central model by exchanging model parameters. Federated Learning can address various concerns related to privacy, security, and data centralization.

[0060] Federated Learning can involve a distributed system where multiple client nodescollaborate to train a shared model under the coordination of a central server or federator node. The general process includes the following steps:1. The federator node can initialize the shared model ^^(0) and distribute it to the client nodes(the superscript denotes the iteration index). Since the scheme repeats at each iteration, the iteration index may be dropped when later describing procedure. Also note that individualmodel parameters may later be subscripted by ^^, i.e., ^^^^.2. ^^ can be the number of clients in the system. Each client node ^^ can train the model locallyon its data, thus obtaining a gradient on its data, thus obtaining a gradient g ,current model with its local information. The local model)^^=−^^ . Here, wi is the vector that encapsulates all the parameters ^^^^ of the modelcomputed by client node ^^.3. The client nodes can send the model updates, specifically in terms of gradients of the model’s parameters (weights and biases), back to the federator node. These gradients represent the direction and magnitude of changes needed to improve the model based on the local data. Note there is no difference whether clients send the updated model or the gradients, since thefederator can always obtain ^^(^^^^+^^). In the present disclosure, the case where clients send thegradients ^^(^^^^)is considered.4. The node can aggregate the updates of all clients to improve the shared model. The most basic form of aggregation is simply averaging the updates of each parameter (weight orbias) across the different values sent by the clients, i.e., ^^(^^+1) = ^^(^^) −(^^) ^^ ^^^^the sum is taken of all clients’ gradients.5. The updated model can be then sent back to the client nodes, and the process repeats until the model converges and / or meets the desired performance metrics.

[0061] FIG. 2 is an exemplary iteration of federated learning with 3 clients. The threeclients shown in FIG.2 do not constitute a limitation on the number of clients.

[0062] The federator node, often also referred to as the central server, can orchestrate thelearning process. It initiates the training by sending the current model to the client nodes and aggregates the updates received from them. The federator node is responsible for ensuring the integrity and efficiency of the learning process, often implementing mechanisms to optimize communication and protect privacy.

[0063] Client nodes are the decentralized entities that hold their own local data. InFederated Learning, these nodes can perform computations on their data and share model updates, in the form of gradients, rather than the data itself. This approach can significantly enhance privacy and security, as sensitive information does not leave the node.

[0064] The aggregation function is a critical component of Federated Learning. It cancombine the updates received from the client nodes to improve the shared model. A common aggregation function is the Federated Averaging algorithm, which computes the weighted average of the updates. The aggregation function plays a vital role in ensuring the model effectively learns from the distributed data while addressing challenges such as non-IID (independently and identically distributed) data across nodes. IID data refers to the setting in which the data held by the clients follows the same distribution. In contrast, non-IID data means that the data held by the clients follows different distributions.

[0065] In Federated Learning, communication between the federator node and clientnodes typically involves the exchange of model parameters or gradients. The communication format can be efficient and secure, often employing compression techniques and encryption to minimize bandwidth usage and protect the transmitted data.

[0066] Federated Learning (FL) faces several key challenges, such as heterogeneity acrossclients’ data maintaining the privacy of the clients’ data, security against Byzantine clients and communication efficiency.

[0067] Federated Learning inherently enhances privacy by design, as the raw data remainson the client nodes. However, privacy concerns still exist, such as the potential for model updates to reveal information about the underlying data. Techniques like differential privacyare often employed to mitigate such risks.

[0068] Security challenges in Federated Learning include protecting the model andupdates from adversarial attacks. Security against Byzantine clients, i.e., clients deliberately corrupting their computation to disrupt the process, is paramount. If undetected, only one Byzantine client inserting errors is enough to prevent the learning algorithm from converging. Secure aggregation protocols and robust authentication mechanisms are critical to safeguarding the federated learning process. Trimmed mean can be used to solve the security issue. That operates as follows: the federator observes the gradient of all client nodes. For each coordinate of the model, the federator sorts the gradient values received from all the clients.For a fixed fraction β of malicious clients to be tolerated (known to the federator), the federatordiscards the largest and smallest βm values of the gradients. The remaining values can be usedto compute the average gradient.

[0069] Given the potentially large number of client nodes, memory and communicationefficiency are paramount. In FL settings, clients can run backpropagation on their local data and communicate a d-dimensional vector (usually d is in the order of 106–109) to the federator, making communication and computation two crucial bottlenecks, especially in settings where the clients are edge devices with limited computation and communication capabilities and energy constraints. Strategies to reduce the size of model updates and the frequency of communication are essential for scaling Federated Learning to many nodes with limited bandwidth. The following two approaches can reduce the communication cost in federated learning, which target to compress the gradients without using any, or only a few, relevant information about the full gradients. 1. Zero-order: the clients can compute an approximation of the true gradient (the one computed through backpropagation) by random perturbations of the model, explained in more detail below. The random perturbations are sampled by the clients at each iteration and used to compute an approximation of the true gradient, update the local model, and send the updated model to the federator. 2. Gradient projection: Opposed to computing approximations of the gradient, one can compute the true gradient and project it onto random directions as proposed for distributed learning in a more restrictive setting. Through the use of a shared seed, this approach can be made communication efficient in a federated learning setting.

[0070] Federated Learning is a powerful approach to machine learning that addresses keychallenges related to privacy, security, and data centralization. By distributing the training process across multiple nodes and aggregating their updates, specifically in terms of gradients, federated learning enables the development of sophisticated models while safeguarding sensitive information. 1. Introduction of CYBER-0

[0071] The present disclosure introduces CYBER-0, a novel communication-efficient andByzantine-resilient algorithm within the context of zero-order optimization. Zero-order optimization, i.e., optimization techniques that require no gradient computation, has seen significant traction in recent years. Zero-order optimization alleviates both computation and communication costs by allowing clients to send approximations of their gradients using random linear projections as (i) it does not require backpropagation; and (ii) it allows significant compression of the communicated vector.

[0072] While achieves communication efficiency through reduced frequencies of modelexchanges, significant communication cost savings can be achieved by a novel bi-directionalshared seed concept. As a result, clients and the federator can only send k real numbers, for aparameter k > 0, instead of high-dimensional vectors. This can be coupled with an effectiverobust aggregation technique specifically tailored to the zero-order optimization context.

[0073] The effectiveness of CYBER-0 can be validated through extensive experiments. ForMNIST, the same accuracy can be observed with a hundred-fold saving in communication. In fine-tuning Large Language Models (LLMs), CYBER-0 can achieve high accuracy in the presence of Byzantine clients, together with a million-fold saving in terms of communication compared to uncompressed transmission. The superiority of CYBER-0 in fine-tuning LLMs is evident in its ability to handle problem domains with inherently low intrinsic dimensionality, making zero-order optimization (and thus CYBER-0) a suitable optimization technique. The fast convergence of CYBER-0 aligns with those findings. Furthermore, it was infeasible to compare to state-of-the-art robust FL schemes that require backpropagation as the simulations in accordance with the prior art overloaded the memory of the GPUs of type Nvidia GeForce RTX 4090; highlighting the memory efficiency of CYBER-0. In addition, it can be theoretically proved that CYBER-0 converges under the assumption of strong convexity.

[0074] The present disclosure is the first work that simultaneously addresses security andcommunication / memory efficiency in FL through zero-order methods; offering a unique advantage, particularly in environments where first-order gradient information is not available or not computationally feasible due to memory-constrained client devices. 2. Problem Setting

[0075] There are notation and conventions used in the present disclosure.

[0076] Notation. Vectors are represented by boldface letters, e.g., z and sets are denotedby calligraphic letters, e.g., . The L2 norm of a vector is denoted by . The inner productof vectors z and is represented interchangeably by ^z, ^ and , chosen to enhance theclarity of exposition. The i-the coordinate of a vector is denoted by (i). The naturallogarithm is

[0077] and ^^d can as the set of unit-norm vectors in ℝd, d^^ =. Given a convex set ⊆ ℝd, the Euclidean projection of a vector ∈ onis . The operation of uniformly sampling a vector z from ^^d canbe denoted as z ∼ ^^d. Similarly, sampling independently and uniformly a set of vectors Z = {z1,z2, . . ., zn} is denoted as either Z ∼ ^^d or z1, z2, .. ., zn ∼ ^^d.

[0078] Federated Learning. The model can comprise a network comprising of a federator(also referred as a central node) and m clients (also referred as client nodes), denoted by theindices {1, 2, . . . , m}. Each client i has a set of data samples ⊆ Γ, where Γ isthe data space. Let ≜ be the global datai∈[m] i set. The by f ( ; γ), isdefined with respect to the parameter vector ∈ ⊆ and a data vector γ ∈ Γ.

[0079] The federator can exchange messages with clients over multiple rounds with thegoal of minimizing the statistical loss. At training step t ≥ 0, thefederator can send a model vector t to the clients. Each client i can then compute an updatemessage based on the federator’s model vector t and its local dataset i and send the resultback to the federator. Finally, the federator can update the model to t+1 as a function of tand { : i ∈ [m]}. At the end of training step T, the federator can output its learned model T.

[0080] Adversarial Model. Byzantine clients behave as in Definition 2.1. A fraction αm(unknown to the federator), 0 ≤ α < 1 / 2, of the clients are Byzantine. Byzantine clients’ indicesare denoted by .

[0081] Definition 2.1 (Attack Model). A Byzantine client b ∈ has complete knowledge ofthe vectors transmitted by all other clients to the federator at each training step. Given thisknowledge, it may send arbitrary vectors to the federator, denoted as ∗, aiming to disrupt theoptimization procedure. 3. Robust Zero-Order Federated Learning

[0082] The present disclosure introduces CYBER-0, a Byzantine-resilient federated zero-order optimization. CYBER-0 builds on the principles of MeZO, a memory-efficient centralized algorithm. In addition to providing Byzantine resilience, CYBER-0 can significantly improve the communication efficiency compared to FedZO, the FL counterpart of MeZO. CYBER-0 is given in Algorithm 1 and is explained next. Algorithm 1 CYBER-0Input: 0 ∈ is the initial model parameter vector, β is the trimmed mean factor, η is thelearning rate, µ is the perturbation step, k is the number of samples per estimate and T is thetotal number of learning steps.for t = 0 to T doFederator Samples Z t ={ ^^^^ ^^ ^^^^ , ^^^^ , . . ., ^^^^ } ∼ ^^dDistributes t = t to each clientfor i = 0 to m in parallel doclient ifor each ∈ Zt doCompute and send to federator, cf. Definition of Zero-Order estimate inthe following context end for end for Federator end

[0083] The successful combination of components of CYBER-0 yields subtle but significantcomplexities. Extending centralized zero-order algorithms to Byzantine-resilient FL requiresensuring that the choice of the vectors z1, · · · , zk does not compromise resiliency and stillallows convergence. In addition, proving that the robustness of a trimmed mean operation, meant to operate on the coordinates of gradient vectors, extends to the zero-order estimator requires novel theoretical tools. Further challenges include practical validation of the proposed algorithm and investigating zero-order fine-tuning in federated large language models. The following provides a formal explanation of the main concepts and the algorithm.

[0084] Definition 3.1 (Zero-Order estimate). For a given µ ≥ 0, a vector z ∈ ^^d and a lossfunction f (w, γ), the single-sample zero-order estimate of the gradient ∇ f (w, γ) is defined as

[0085] As considering a fixed µ, g( , z, γ) instead of g( , z, γ, µ) is written.

[0086] To simplify notation, for each client i ∈ [ ] and each j ∈ [| i|], the client partialestimate is defined as gi,j ( , z) ≜ g( , z, γi,j ) and the(, z). The norms ( , z) ≜ ∥gi( , z)∥2 and ( , z, γ, µ) ≜ g( , z γ, µ) are defined.

[0087] Definition 3.2 (Trimmed Mean). Given 0 ≤ β < 1 / 2 and a multiset X =, trimmed mean operation is defined,where Xβ is obtained by removing the largest and smallest ⌊β ⌋ elements from X.

[0088] Computation Procedure. At training step t, the federator shares the model t andk vectors ~ ^^d with the clients. Client i computes gi ( t, z ^^^^ ) and sends ( t, z ^^^^ ), r ∈[k], to the federator. Byzantine clients send arbitrary vectors denoted by *. The robustaggregation ĝβ( t, z ^^^^ ) as

[0089] Next, the tthe gradient estimate ĝ = andupdates the model as t + 1 for a givenTo reduce thecommunication cost, the federator can only broadcast and each clientreconstruct t + 1 individually.3.1 Properties of CYBER-0

[0090] The algorithm can exhibit Byzantine resilience and communication and memoryefficiency. Those are notable properties contributing to its effectiveness in federated learning.

[0091] Byzantine Resilience. A key component of the robust aggregation is that it alignswith the compression mechanism used by the clients. The transmitted values can be perceivedas a linear k-dimensional compressed representation of the gradient. This format is inherentlyamenable to scalar robustness procedures, such as the trimmed mean.

[0092] Communication Efficiency. CYBER-0 can compress d-dimensional vector into k realvalues, offering low communication costs. At first glance, CYBER-0 appears to rely ontransmitting the d-dimensional vectors at each training step. However, thisoverhead is efficiently mitigated through a simple yet effective strategy: utilizing a shared common seed among clients and the federator. The federator can disseminate this seed, whichis then used by the clients to sample the perturbation directions in a coordinatedmanner. Local updates can further reduce the communication cost. For clarity of exposition, the present disclosure provides CYBER-0 with local updates in Algorithm 4 in the section below on experimental details, together with corresponding numerical experiments.

[0093] Memory Efficiency. By using Zero-order approximation, which inherently does notnecessitate backpropagation, CYBER-0 can save significantly on memory (by up to a factor of 12) compared to traditional training methods relying on backpropagation. Furthermore, CYBER-0 can adopt in-place perturbations on model parameters, a technique also used in MeZO, to further reduce memory usage.

[0094] Under a different setting, µ = 0, CYBER -0’s gradient estimates are computed byprojecting the true gradient along different directions. This can remove the memory efficiency property in exchange for better computational efficiency, since instead of calculating 2kfunction evaluations, only a single gradient and k projection calculations are needed.

[0095] CYBER-0 is the first application of Byzantine resilience in zero-order compressedinformation scenarios. In contrast, for any case other than k = 1, coordinate-wise methods donot allow the same compression mechanism to occur in the federator-to-client communication. 3.2 Training procedures

[0096] FIG.3 is a flow chart illustration a process for training the model according to someembodiments of the present disclosure. The core of the present disclosure is described by a training procedure capable of reducing communication between clients and federator, and vice-versa, by sending only a limited number of scalars per round. There are two variants of communication-efficient methods, enabled through the use of a shared seed between thefederator and the clients. One variant (later referred to as µ = 0) uses backpropagation, butprojects the calculated gradient vectors gi onto random vectors generated by a common seed.The result is a scalar, which is the only necessary to be communicated. To achieve betterperformances, this can be repeated for multiple random projections (later referred to as k),hence more than one scalar to be transmitted. While the gradient projection approach requires to calculate the true gradient through backpropagation, it is more computationally efficient if k is big. However, backpropagation requires more memory resources compared to theapproximation approach (referred to as µ > 0) where no backpropagation is required. Thisapproach can also be pursued if computing the gradients is impossible, e.g., in black-boxoptimization or when the objective is not differentiable. Here, the k scalars can be obtainedthrough model perturbations in a random direction instead of the projection onto the truegradient gi. The byzantine-resilient aggregation of the clients’ results can be then conductedon the k scalars of each client. The procedure is described in detail in the following.Input Parameters

[0097] These parameters are defined by the federator and used by all clients. They are asfollows:w(0) represents the initial set of parameters of a generic neural network. In other words, w(0)the initial values of the weights and biases in all of the layers of the network. These values aremathematically represented a single, d-dimensional, vector (the neural network has, therefore,a total of d parameters in total, when accounting for all layers).k represents the number of scalars to be communicated by each client on each round. This number can be chosen as needed and therefore counts as an input of the procedure. µ is the approximation parameter. In practice, it can represent how close of an approximation chosen when deriving the to-be-transmitted scalars. In particular, there are two differentvariants of the procedure: one for µ = 0 (exact, with backpropagation) and the other for µ > 0(approximation, without backpropagation). The two different variants have different properties when regarding the memory vs computation efficiency trade-off.β represents the robustness factor. In particular, β stands for how many of the sent scalars willbe removed in the trimmed mean procedure responsible for bringing robustness to the present solution. η is the learning rate. s is a random seed (in simpler words, a random piece of text, such as WU1nUA7eM6cX4, or a random number such as 29333444982231150511349542129150) that will be used to synchronize the pseudo-random number generation (PRNG) across clients and the federator. This random seed allows the clients and federators to generate the same set of random values without needing to transmit the information to each other. Initial Phase

[0098] This phase can be executed only once in a training procedure. In this phase, thereare the following steps: 1. The federator can send the seed s to all clients. 2. The federator can send the initial model parameters w(0) (and possibly the architecture ofthe neural network, e.g., how many layers and the types of each).

[0099] There is no direct communication efficiency taking place here. The values aretransmitted in full. However, as this is a step to be taken only once throughout the entire process, it will impact no further in costs. To further save communication costs during the initialization phase, it is possible to generate the same random initializations at the federator and the clients’ by using a common seed. In that way, the same random weights can be generated and only the architecture of the model has to be communicated. Training Phase

[0100] In this phase, the federator can utilize the values sent by the clients to update theparameters (weights and biases) of the so-called central model. The federator can also transmit back the new central model to the clients so that this step can be repeated.

[0101] In particular, this step can be repeated a number of times, until the neural networkmodel achieves good accuracy, which refers to as “converges”. For the description of this phase,it is assumed that the current repetition number is t.

[0102] This phase can be divided into three parts:1. Client Local Training 2. Federator Robust Aggregation 3. Client Synchronization Client Local Training

[0103] On the local training of each client - fixing client number i - there may be thecalculation of an approximate gradient. This calculation, as mentioned in the description of theinput parameter µ, can be done in two variations. These two variations are discussed in detailin the following context.

[0104] 1. For each sample number r, from 1 to k:(a) The client can construct a new seed s′ by concatenating s with t and with r. For example, ifs = 1234, t = 10 and r = 2, the new seed s′ = 1234-10-2. Each client can do the sameindependently.(b) The client sets s′ as the seed in its own PRNG module, then iterates over each parameter oftheir current model w(t) (each weight and bias value in the current neural network), and doesthe following (the iteration identifier can be dropped for ease of exposition): i. If µ = 0 (with backpropagation):A. The client uses backpropagation to calculate the model gradient gi,B. The client randomly samples d values from its PRNG, using a Gaussiandistribution with mean 0 and standard deviation 1, constructing a vector zr.C. The client ^^the standard inner product between gi and zr, which is ^^^^ .D. The client sends ^^^^^^ to the federator.ii. If µ > 0 (without backpropagation):A. The client iterates over the model parameters wj (that is, each weight and biasof the neural network). B. Then the client samples a different value zj from its PRNG, using a Gaussiandistribution with mean 0 and standard deviation 1, for each model parameter wj andupdates the parameter from wj to wj + µzj (adds the random value).C. The client calculates the error value over its input data, which is denoted by e+D. The client now modifies each weight and bias to be wj − µzj (this can be done bysubtracting twice zj from the incremented value).E. The client calculates the error value over its input data, which is denoted by e− F. The client calculates the approximation ^^^^^^ = ^^(^^+−^^−)2µ and sends it to the federator. G. The client resets its local model to the original weights and biases wj (this can bedone by incrementing each value by zj).By the end of this step, each client has communicated k scalar values to the federator, namely^^^^^^ from r = 1 to k.Federator Robust Aggregation

[0105] The federator can receive the different scalars from all the clients and proceeds toaggregate them. It goes as follows:1. For each sample number r, from 1 to k:(a) The federator aggregates all values corresponding to sample r coming from each client, {^^^^^^ }from i = 1 to m (the total number of clients) using a trimmed mean with parameter β. Thismeans that a fraction of β values will be discarded from the top and bottom of the list. Theresult of the trimmed mean is denoted as gr(b) The federator constructs a new seed s′ by concatenating s with t and with r.(c) The federator sets s′ as the seed in its own PRNG module.(d) The federator iterates over the model parameters wj (that is, each weight and bias of theneural network).(e) Then the federator samples a different value zj from its PRNG, using a Gaussian distributionwith mean 0 and standard deviation 1, for each model parameter wj.(f) The federator modifies each if its model parameters (each weight and bias) wj by replacingit for wη j−^^ ^^^^^^^^. This is the step that modifies the federator model in order to optimize it, also called the descent step.(g) The federator can transmit gr to each client.By the end of this step, the federator has modified its local neural network model using the aggregate data from each client. Also, the federator can communicate such aggregate data to each client, amounting to k scalars per client.Client Synchronization

[0106] What remains is for each client to synchronize its outdated local neural networkmodel with the one held by the federator. For that, the clients can use the received k scalarsfrom the federator, gr. It goes as follows:1. For each sample number r, from 1 to k:(a) The client constructs a new seed s′ by concatenating s with t and with r.(b) The client sets s′ as the seed in its own PRNG module.(c) The client iterates over the model parameters wj (that is, each weight and bias of the neuralnetwork).(d) Then the client samples the same values zj from its PRNG as the federator (ensured throughthe common seed), using a Gaussian distribution with mean 0 and standard deviation 1, foreach model parameter wj.(e) The client can modify each if its model parameters (each weight and bias) wj by replacingit for wη j−^^ ^^^^^^^^. By the end of the steps, both client and federator models are synchronized, they have again the same (updated) weights and biases and the training phase can once more be conducted. FIG. 3 highlights the iterative nature of the procedure and the steps conducted per iteration. FIG.4 shows in more detail a specific iteration of the algorithm, highlighting the difference to a standard federated learning iteration as illustrated previously in FIG.2.

[0107] It is novel to combine the different aspects: zero-order optimization, the sharedseed concept, and the robust aggregation over the communicated scalars in a federated or distributed scenario.

[0108] The most related work proposes to use zero-order approximations for federatedlearning. Therein, communication efficiency is achieved through local iterations and partial device participation. While those tools could additionally straightforwardly be incorporated into the present disclosure, a much better communication efficiency can be achieved through the use of a shared seed, adapted from. Additionally accounting for byzantine clients in combination with zero-order methods (or gradient projection) in the federated or distributed learning setting is novel. There does not exist any work to be named here. Accounting for byzantine clients without communication and memory efficiency through zero-order methods was done.

[0109] In the procedure description, the aggregation function to be used is the trimmedmean - as this is the basis of the results. However, this choice of function can be made more general, in the sense that any kind of robust aggregation function can be used to replace the trimmed mean with no conceptual loss.

[0110] Furthermore, for ease of read, the procedure for the training of neural networks isdescribed, but it can be also utilized for the training of any machine learning model that relieson gradient descent, this includes Linear Regression, Logistic Regression ,Support Vector Machines (SVMs), Decision Trees and Ensemble Methods (like Gradient Boosting), Generalized Linear Models (GLMs), Deep Learning Models (beyond standard neural networks, like CNNs and RNNs).

[0111] On top of the explained scheme, one can add end-to-end (public key) encryptionbetween each client and the federator to ensure privacy of the communication scalars against an eavesdropper.

[0112] With a slight modification, this scheme can be adapted to account for partial clientparticipation. It had been shown that in some cases, the learning algorithm does not suffer from the occasional participation of clients instead of each client computing a partial gradient at each iteration. While this approach often does not hurt the performance of the model, it can bring significant communication savings on top of the efficiency provided by the present scheme. To ensure this, the federator has to keep track of all gradients a client needs to update its local model to the global state after it did not contribute for a certain number of global iterations.

[0113] More specifically, the federator has to send the averaged gradients for each iterationand each projection together with the information of which of those belong to which iteration. This information is sufficient to bring all clients up to speed in every iteration, even if some of them did not participate in any amount of preceding iterations.

[0114] Different variants of PRNGs (apart from uniform) can be suited for generating therandom projections. There is no reason to limit this approach to a specific random number generator. Which randomness is most suitable is application-specific and depends on the dataset and model used, although the proposed method may always be a good starting point.

[0115] Although the federated scenario is described, the described procedure can be usedin any kind of distributed scenario, regardless of the ownership of data and devices being centralized or decentralized, as long as the training topography is the same - a central node and a set of worker nodes. 4. Experiments and Proof of Concept

[0116] FIG. 5a to 5c illustrates CYBER -0 for logistic regression on MNIST under non-IIDdata distribution. FIG. 5a is a graph illustrating impact of sample size ^^ over thecommunication cost according to some embodiments of the present disclosure. FIG. 5b is agraph illustrating impact of sample size ^^ over the number of steps according to someembodiments of the present disclosure. FIG.5c is a graph illustrating effect of Byzantine client behaviors according to some embodiments of the present disclosure. FIG. 6 is a graph illustrating the effect of Byzantine clients on the convergence speed of CYBER -0 when fine- tuning the RoBERTa large model on the SST-2 dataset. Specifically, FIG. 5a and 5b show the convergence for varying k in the absence of Byzantine clients compared to federated averaging(FedAvg). FIG. 5c shows different attacks for k = 64 and α = β = 0.25.Experiments

[0117] A series of experiments showing the performance of CYBER -0 across variousscenarios is presented. First, a Logistic Regression model on MNIST is employed to investigate the parameters’ influence on the convergence of CYBER -0. This setting serves as a foundational test with insights into the baseline performance and parameter sensitivities of CYBER -0 in a controlled environment. For a more comprehensive understanding of CYBER -0’s performance reaching towards advanced applications in Natural Language Processing (NLP), the examination is extended to federated fine-tuning of large language models (LLMs).

[0118] While Algorithm 1 completely specifies the behavior of CYBER -0, the sectionDetails on Experiments A.1 presents an extended algorithm utilized for experiments, highlighting the memory and communication optimizations in implementation. The basic functionality of the algorithm remains unchanged. 4.1 Experimental Set-up

[0119] The key conditions for experiments are client data distribution and simulatingByzantine behaviors, explained next. General simulation parameters and hyperparameters are shown in the section Details on Experiments A.2. Tables and figures displaying standard deviation measures (denoted by the ± sign) represent the average outcomes of three independent simulation runs, initialized by different random seeds.

[0120] Data Distribution. Two distinct data distribution scenarios are investigated:independent and identically distributed (IID) data and non-IID data. In the IID scenario, data labels are uniformly distributed across all participating clients, ensuring an equal representation of each label in the local dataset. Conversely, in the non-IID scenario, a unique label set is assigned to each client, thereby creating a skewed label distribution and introducing additional complexity in the learning process.

[0121] Byzantine Behavior. A worst-case scenario is considered wherein Byzantine clientsare fully aware of the communication protocol and the transmissions of other clients. Byzantine clients can collude and act in a deliberately adversarial manner. The presentdisclosure draws inspiration from the attack model, i.e., focusing on maximizing the localgradient deviation at each training step. This is achieved by strategic manipulation of the information sent to the federator. In this attack, the Byzantine clients compute the true gradient estimate obtained from the honest clients. If that estimate is positive, they all send avalue equal to the ⌊βm⌋-th smallest honest gradient value. Otherwise, they all send a valueequal to the ⌊βm⌋-th largest honest gradient value. This attack is called Full Knowledge anddescribed in Algorithm 3.

[0122] For the MNIST experiments, this choice is compared to other Byzantine behaviorsto display its effectiveness. In particular, it is compared to three model poisoning strategies:Always Small, Always Large, and Random Choice, in which Byzantine clients all send eitherthe ⌊βm⌋-th smallest, largest or randomly pick one of them, respectively, for each perturbationdirection. And a data positioning strategy, Label Flipping, in which the Byzantine devicesswitch each MNIST label from ℓ to 9 − ℓ.4.2 CYBER-0 with Logistic Regression on MNIST

[0123] Impact of Sample Size k. The training loss trajectories for diverse settings of thesample size parameter k are illustrated in FIG.5a over the cost of communication, and in FIG. 5b over the number of steps. The conventional federated averaging (FedAvg) is included as a benchmark.

[0124] A notable trend is observed: as the value of k increases, the convergence rateprogressively aligns with that of standard SGD. This aligns with following theoreticalexpectations, as a larger sample size k yields a sample mean that more closely approximatesthe true gradient of the loss function.

[0125] Effect of Byzantine Client Behavior. FIG. 5c presents a comparative analysis of thetraining loss dynamics under different Byzantine client behaviors. The results show that the Full-Knowledge strategy presents the highest damage to the training process by causing the most substantial delay in convergence. This phenomenon underscores the potency of informed adversarial behaviors in disrupting the learning process. Table 1: Comparison with state-of-the-art: test accuracies of CYBER-0 (k = 64) in comparison with the trimmed mean and Krum algorithms under non-IID data distributions. Theexperiment uses β = α, the Full-Knowledge attack for CYBER-0, and the model poisoningattacks for trimmed mean and Krum. Table 2: Comprehensive comparison of CYBER-0 across varied configurations and datasets: This table presents a detailed analysis of CYBER-0’s performance under different experimental setups. Non-Byzantine behaviorfor CYBER-0 with no Byzantine clientsnor robust aggregation. Byzantine behavior uses β = α = 0.25. All results are averaged withstandard deviations shown after

[0126] Comparison with State-of-the-art. The performance of CYBER-0 is compared withthe trimmed mean and Krum. For this comparison, The Full-Knowledge attack on CYBER-0 is applied, while applying the model poisoning attacks from for the state-of-the-art approaches, as those are the most effective attacks against the respective algorithms. The results are shown in Table 1.

[0127] CYBER-0 can provide better Byzantine resilience for all α values while achieving aroughly 100-fold communication reduction. It can be assumed that the projection over the random directions leaves fewer degrees if freedom for the Byzantine clients to change the aggregated gradient, hence providing good Byzantine resilience and allowing for communication efficiency. 4.3 Fine-Tuning Language Models with CYBER-0

[0128] The RoBERTa-large model can be utilized for three distinct NLP tasks: sentimentanalysis, natural language inference (NLI) and topic classification. For sentiment analysis, the SST-2 dataset is employed. For NLI, the SNLI dataset is employed. For topic classification, the TREC dataset is used. A prompt-based fine-tuning approach in a few-shot learning framework is adopted. Fine-tuning LLMs is well-established in the literature. The details are omitted here for brevity.

[0129] A set of 512 data points distributed among the clients is used according to thespecified data distribution pattern. These experiments intend to show the applicability of CYBER-0 in more complex and real-world scenarios, particularly in the increasingly relevant field of NLP.

[0130] In the classical theory of zero-order optimization, fine-tuning LLMs is deemed tobe of prohibitively slow convergence due to the role exercised by the model dimension d.Nevertheless, certain problem domains exhibit an inherently low intrinsic dimensionality. Thefast convergence of CYBER-0, cf. first and second columns of Table 2.

[0131] Robustness of CYBER-0. The ability of CYBER-0 to mitigate the effect of Byzantineclients, using the strong Full-Knowledge attack, can be seen in the right-most column of Table 2. In IID settings, CYBER-0 exhibits a small drop in accuracy. However, for non-IID settings, while still converging, CYBER-0 exhibits a drop in accuracy in the presence of Byzantine clients. This behavior aligns with the literature on non-IID robust FL. The main reason is that the non- IID data distribution is reflected in the clients’ message updates. Making the distinction between malicious gradients and outliers more challenging. In addition to affecting the final accuracy, Byzantine clients also decrease the convergence speed of the algorithm. This effect on the SST-2 experiment is illustrated in FIG. 6, which complements the data presented in Table 2. Similar figures for the SNLI and TREC experiments are given in the section Details on Experiments A.3.

[0132] Communication optimizations and scalability aspects of CYBER-0 is furtherdemonstrated by using RoBERTa-large on the SST-2 dataset. The number of samples per training step and the total number of clients is varied, cf. Table 3. The section Details on Experiments A.4 provides an extra set of experiments regarding local epochs. Table 3: Varying the number of samples (k) and the number of clients (m) in SST-2 using RoBERTa-large in the presence of Byzantine clients with non-IID data distribution.

[0133] Number of Samples. As observed in Table 3, and consistent with findings fromSection 4.2 CYBER-0 with Logistic Regression on MNIST, for fixed values of m = 8 and β =0.25, an increase in the number of samples k correlates with accelerated convergence in terms of steps. CYBER-0 presents here a tradeoff between computational workload andcommunication efficiency. A larger value of k necessitates more forward passes per trainingstep. However, the transmission of these passes in batches potentially enhances communication efficiency by reducing the need for frequent synchronization rounds.

[0134] Number of Clients. The impact of scaling the number of clients is explored whilemaintaining the same ratio of Byzantine to non-Byzantine clients. As observed in Table 3, forfixed values of k = 1 and β = 0.25, increasing the number of clients does not significantly affectthe final test accuracy. This outcome aligns with the expectation that similar data distribution among non-Byzantine clients would result in consistent learning patterns, regardless of the network size. Proof of Concept

[0135] The algorithm is implemented on different data sets and models in a federatedlearning scenario both on IID (homogeneous) and non-IID (heterogeneous) data partitioningacross the users. To show the impact of the number of projections k (and hence the number ofscalars to be communicated), the procedure is tested on the well-known MNIST dataset containing images of handwritten digits from 0 to 9. The task for the model is to classify thedigits, for which a simple logistic regression model is used. FIG.5 highlights the impact of k onthe accuracy of the classification task, reflecting the percentage of test image correctly classifiedafter training the model. In FIG. 5b, it can be seen that with k = 16 comparable test accuraciesare reached compared to transmitting the entire gradients at each iteration (cf. FedAvg). As can be seen from FIG. 5a, this accounts to roughly a 1000-fold saving in communication, despite reaching the same final performance of the model.

[0136] In the following, it is verified that the present efficient and secure algorithm issuitable for recent and complex real-world machine learning tasks to the extent of reachingcompetitive accuracies while providing significant advantages. The RoBERTalarge model is utilized for three distinct natural language processing (NLP) tasks: sentiment analysis, natural language inference (NLI) and topic classification. For sentiment analysis, the SST-2 dataset is employed. For NLI, the SNLI dataset is employed. For topic classification, the TREC dataset is used. Table 2 shows the results of the present procedure for both byzantine and non-byzantine behavior for IID and non-IID data. The baseline for experiments is the case where data is not distributed, hence there are no challenges associated with byzantine behavior or communication constraints. The results are depicted in the leftmost column. The remaining columns are generated by running the present procedure in a federated learning environment. The non-byzantine setting with IID data corresponds to the easiest training task, yielding comparable performance compared to the non-distributed setting. The fact that the present approach sometimes outperforms the baseline can be traced-back to statistical fluctuations and the fact that employed gradient compression technique has some inherent regularization properties, that can improve the final generalization capabilities of the model and hence the observed accuracy. It is shown that even in a non-IID setting (without Byzantine clients), the present proposed algorithm reaches state-of-the-art results, despite the heterogeneity of the clients’ data. The two rightmost columns depict the results of training runs that can cope with the behavior of byzantine clients. Without countermeasure, the model would diverge from the optimal solution, hence not be able to learn anything useful from the dataset. With the proposed countermeasure, competitive accuracies can be achieved for both IID and non-IID data in most cases. Even for very challenging tasks (such as SNLI), it is observed that for non- IID data and the worst possible corrupt behavior of clients, the present procedure still ensures that the model can extract useful information from the data, resulting in 60% model accuracy.

[0137] All the results depicted for byzantine clients are worst-case results, since theyrequire full knowledge of all adversarial clients, i.e., the knowledge of all other clients’ computations and full knowledge of the countermeasure. This is a very unlikely scenario in practice, here meant to challenge the present algorithm in the worst possible way. Hence, the results are a very pessimistic underestimate of the present procedures’ performance for realistic scenarios. Nevertheless, it can be concluded that the present approach reaches comparable performance compared to state-of-the-art while providing significant improvements in terms of communication cost, memory efficiency, and ensuring resiliency against byzantine adversaries. Having in mind that those challenges are one of the most challenging ones in current federated learning systems, the proposed algorithm is a very promising candidate to further drive the progress in (and the usefulness of) large-scale decentralized machine learning by alleviating major concerns.

[0138] While it is demonstrated that the present procedure has significant advantageswhile achieving state-of-the-art and competitive performances in challenging settings, it remains to compare the present algorithm against well-established countermeasures forbyzantine behavior. Although those methods do not provide efficiency in terms of communication or memory usage, their capabilities of detecting corrupt clients can still be compared. In Table 2, the results for training a model on MNIST for different fractions α of byzantine clients are depicted. For each countermeasure evaluated, the worst-possible attack known, specifically tailored to the relative countermeasure, is utilized. Compared to the state- of-the-art countermeasures KRUM and Trimmed-Mean, the present algorithm provides significantly better robustness. Additionally considering that the present algorithm provides a multi-fold saving in communication costs and memory efficiency while training, as well as the option to be used without performing backpropagation, this is an impressive indicator for the effectiveness of the present disclosure. 5. Theoretical Analysis

[0139] To complement the present extensive numerical results, a theoretical convergenceguarantee for CYBER-0 for convex loss functions and IID data distribution is provided. The results under those assumptions quantify the interplay between convergence guarantee andthe choice of µ, k, and d in well-behaved settings and pave the way to an extended analysis fornon-convex losses and non-IID data distribution.

[0140] A probabilistic bound on the distance between the robust gradient estimationyielded by the present method and the ideal expected gradient obtained in a Byzantine-free context is established. With such a bound established, a convergence analysis is carried for an SGD algorithm that operates under bounded-error conditions, using the zero-order gradient estimate. 5.1 Preliminaries

[0141] To establish theoretical results, an assumption is made on the data distribution anddefine the population loss and the zero-order population estimate.

[0142] Assumption 5.1 (IID Data Distribution). Each client i has a set of n data samples {γi,1, . . . , γ i,n}sampled from a common data distribution D.

[0143] Definition 5.2 (Population Loss). The population loss F( ) is expressed as theexpected value of the loss over , i.e., F( ) = γ∼D[f( ; γ)].

[0144] Associated with the population loss, there is the following optimization problem

[0145] Assumption 5.3 (Local Minimum). The model is a local minimum of F.

[0146] Definition 5.4 (Zero-Order Population Estimate). Let F be the population loss forthe optimization defined by:and its norm by ( , z) = ∥ḡ( , z)∥2.5.2 Robustness Error Bound

[0147] The derivation of the aforementioned bound proceeds by adding three assumptionson the functions g, f and F and the parameter space .

[0148] Assumption 5.5 (Smoothness). Consider any µ ≥ 0, γ ∈ Γ, w ∈ ℝd, and z ∈ ^^d.Assuming that g(·, z,γ) exhibits Lw,µ -Lipschitz continuity, g( , ·, γ) exhibits Lz,µ -Lipschitzcontinuity and that f(·, γ) is L-Additionally, it is assumed that F(·) demonstrates LF -smoothness. For simplicity of notation, µ = Lw,µ + Lz,µ . is denoted.

[0149] Assumption 5.6 (Sub-Exponentiality). For all µ ≥ 0, z ∈ ^^d, and w ∈ , g( , z, γ)isdistributed as a -sub-exponential random variable, conditioned upon γ being sampled from.

[0150] Assumption 5.7 (Restriction on ). is both convex and compact with apredefined diameter .

[0151] With these assumptions in place, it is now possible to establish a bound on thediscrepancy between the robust estimate and the expected estimate among benign clients.

[0152] Theorem 5.8 (Robustness Error Bound). Let µ ≥ 0, z ∈ ^^d, ^ > 0. Then for any zr ∈5.3 Convergence analysis

[0153] To study the convergence properties of the proposed algorithm, it is explored howthe algorithm behaves in scenarios with different values of the parameter µ under theassumption of strong convexity.

[0154] Assumption 5.9 (Strong-Convexity). F( ) is λ-strongly convex.

[0155] Under this premise, the convergence characteristics of Algorithm 1 for µ = 0 areanalyzed.== ^ = ^^+^^−1 = 1^^ 2where T is the parameter vector at the T-th step and ∆ is the same as defined in Theorem5.8.

[0157] To achieve an order-optimal error rate of , where the β term representsthe introduced error by the Byzantine behavior, thetraining steps T should be atleast .

[0158] with zero-order theory, the analysis reveals a linear dependence of theconvergence rate on the model dimension d, not observed in first-order methods. Thisdependence is encapsulated in the parameter τ. As the number of sampled perturbationdirections k increases, the present algorithm approximates the standard first-order rate ofconvergence.

[0159] The following examines the case where µ > 0.

[0160] Definition 5.11 (Smoothed Version of F). Let F be a population loss, for any µ > 0and ∈ ℝd. The smoothed version of F, Fµ : ℝd→ ℝ is.

[0161] Assumption 5.12 (Local Minimum of Fµ). ^^∗µ= arg min Fµ( ) is local minimumof Fµ.For any , F( ) and Fµ ( ) cannot differ more than ^^2

[0162] µ2 , implying that the solutionsF( *) and F( ∗µ) can be made arbitrarily close by the choice of µ.^^^^in α≤where T is the parameter vector at the T-th step of Algorithm 1 and ∆ is as defined inTheorem 5.8.

[0164] Similar to the results for µ = 0, with µ > 0, the same order-optimal error rate isobserved under appropriate conditions for T. Notably, for k = 1, the convergence rate scaleslinearly with the dimension d. However, unlike the case with µ = 0, increasing k does notentirely mitigate this linear dependency, leaving a residual term proportional to √^^.6 Related Work

[0165] Zero-order Optimization. In recent years, zero-order optimization has significantlyevolved, broadening its applicability across various domains. This technique has been particularly instrumental in areas such as black-box optimization and reinforcement learning as gradient computations are not required.

[0166] A novel and notable application has emerged with the fine-tuning of LLMs,showcasing the versatility of zero-order optimization as a memory-efficient technique to allow the fine-tuning of billion-parameter models, while only using a fraction of the memory required by the first-order counterparts.

[0167] Communication Efficiency. Communication efficiency through gradientcompression, can be mainly divided into two categories: quantization-based methods, and sparsification-based methods. None of those, however, are tailored for zero-order estimates, in the sense that they act on the d-dimensional gradients. The present disclosure relies on compression on the gradient estimate by transmitting only the difference in perturbation losses, while taking advantage of an agreed randomness.

[0168] Byzantine Resilience. To mitigate adversarial conditions in the learning process,recent research has seen a surge in the development of Byzantine resilient algorithms. The present disclosure is closely related to applying one-dimensional coordinate-wise statistical robustness techniques to the gradient information transmitted by the clients. In contrast, countermeasure is applied for each perturbation, posing new theoretical challenges and bridging zero-order estimation and statistical robustness.

[0169] Poisoning attacks. From the wide range of attacks applied to Byzantine-resilientalgorithms, data poisoning and model manipulation are specially mentioned. Both attacks enable Byzantine clients to effectively reduce the speed of convergence, increase the error rate, or completely disrupt the optimization result. The attack model is related to that the Byzantine clients use the full information about the other clients’ responses to maximize the deviation from the benign estimate in each optimization steps.

[0170] The present disclosure introduces CYBER-0, a novel federated zero-orderoptimization scheme designed to withstand Byzantine behaviors. Its effectiveness even under challenging conditions is demonstrated, which include a coordinated full-knowledge attack on non-IID data distributions. The theoretical analysis underlines the robustness of CYBER-0, illustrating the limited ability of Byzantine clients to significantly influence the learning process. CYBER-0 can be further enhanced by exploring advanced compression techniques, such as quantization, to optimize the transmission of values within CYBER-0.

[0171] The importance of integrating privacy-enhancing measures further opens up manypotential directions including the incorporation of differential privacy, homomorphic encryption, and secure multi-party computation. The inherent efficiency of CYBER-0 yields promising compatibility with privacy preserving techniques, potentially opening up new frontiers in secure, private and efficient federated learning.

[0172] It should be noted that the above description is merely provided for the purposes ofillustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skill in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.

[0173] Having thus described the basic concepts, it may be rather apparent to those skilledin the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Various alterations, improvements, and modifications may occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested by this disclosure, and are within the spirit and scope of the exemplary embodiments of this disclosure.

[0174] Moreover, certain terminology has been used to describe embodiments of thepresent disclosure. For example, the terms “one embodiment,” “an embodiment,” and / or “some embodiments” mean that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the present disclosure.

[0175] Further, it will be appreciated by one skilled in the art, aspects of the presentdisclosure may be illustrated and described herein in any of a number of patentable classes or context including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented entirely hardware, entirely software (including firmware, resident software, micro-code, etc.) or combining software and hardware implementation that may all generally be referred to herein as a “unit,” “module,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer readable program code embodied thereon.

[0176] Furthermore, the recited order of processing elements or sequences, or the use ofnumbers, letters, or other designations therefore, is not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose, and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover modifications and equivalent arrangements that are within the spirit and scope of the disclosed embodiments. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software only solution, e.g., an installation on an existing server or mobile device.

[0177] Similarly, it should be appreciated that in the foregoing description of embodimentsof the present disclosure, various features are sometimes grouped together in a singleembodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various embodiments. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, claimed subject matter may lie in less than all features of a single foregoing disclosed embodiment. A. Details on Experiments A.1 Experimental Algorithm

[0178] Here, the algorithm utilized for the experiments presented in Section 4Experiments and Proof of Concept is elaborated in detail. Algorithm 2 illustrates the optimization steps that enable CYBER-0 to attain both communication and memory efficiency. Notably, the key differences from Algorithm 1 include:

[0179] Perturbation Direction Sampling. Contrary to the original method ofsampling the perturbation directions z from the unit sphere ^^d and scaling the estimate by afactor of d, the practical approach involves sampling each coordinate of z independently from a standard Gaussian distribution. This modification, while seemingly minor, has significant practical implications. On one hand, this modification retains the theoretical guarantees – the expected norm squared is identical to that of sampling over the unit sphere, and the normalized variance of the norm also decreases with increasing d. At the same time, from a practical point of view, it facilitates iterative sampling for each gradient coordinate, thereby considerably reducing the memory requirement in contrast to allocating the entire vector.

[0180] Shared Randomness. Since zero-order optimization requires clients and thefederator to agree upon the perturbation directions, one approach would be that, each client receives the transmission of perturbation directions from the federator. This would entail a significant downlink communication overhead. Instead, the optimized method relies on pseudorandom generation. A single seed value is transmitted from the federator to the clients at the start of the training loop. This enables every client to independently reconstruct all perturbation directions, dismissing the need for direct transmission from the server. The principal advantage of this strategy lies in its significant reduction of communication costs per training step—from communicating a d-dimensional vector to merely a single value per sample.

[0181] In-Place Operations. The standard procedure involves allocating memory forintermediate values such as perturbation vectors and perturbed models. In the present approach, these operations are executed in place. This method not only conserves memory but also streamlines the computational process. By performing and subsequently reversing these operations in place, a low memory footprint is maintained throughout the training phase.

[0182] Algorithm 2 CYBER-0- Experimental SetupInput:0is the initial model parameter vector, β is the trimmed mean factor, η is the learning rate, µ is the perturbation step, k is the number of samples per estimate and T is the total number of learning steps. ▶ Federator Distributes a random seed s to each client Distributes0i =0to each client for t = 0 to T do for i = 1 to m in parallel do ▷ client i for r = 1 to k doend Sendsend for / / Federator robust aggregation ▶ Federator for r = 1 to k do / / clients synchronization for i = 1 to in parallel do ▷ client i for or r = 1 to k doend for end for end for function ZOGRAD(F, , B, µ, s′)Return^^+^^−2µend functionfunction PERTURBPARAMS( , µ, s′)Set RNG seed as s′ for each(i)z ~(i) (i)+ µz / / Memory efficient end for end function Here the Algorithm summarizing the Full-Knowledge attack is provided.

[0183] Algorithm 3 Full-Knowledge Byzantine BehaviorifSend the ⌊β ⌋-th smallest value of : i ∈ [ ] \ }else Send the ⌊β ⌋-th largest value of : i ∈ [ ] \ }end if end for A.2 Simulation Parameters and Hyperparameters A.2.1 Logistic Regression on MNIST

[0184] The simulation parameters and hyperparameters for all CYBER-0 experiments inSection 4.2 CYBER-0 with Logistic Regression on MNIST are found in Table 4. Exceptionally, in the experiments of Table 1, m = 40 is used. All results in Section 4.2 CYBER-0 with Logistic Regression on MNIST are averaged across three random seeds.Table 4: Simulation 4.2 CYBER-0 with Logistic Regression on MNIST All of simulations can be run in a single GPU setting using an Nvidia RTX 4090 GPU. A.2.2 Prompt-Based Fine-Tuning

[0185] The simulation parameters and hyperparameters for all CYBER-0 experiments inSection 4.3 Fine-Tuning Language Models with CYBER-0 are found in Table 5. For Table 1 and 2, results are averaged across three different random seeds. Table 5: Simulation Parameters and Hyperparameters for Section 4.3 Fine-Tuning Language Models with CYBER-0 A.3 Further Loss Curves for TREC and SNLI

[0186] In this section, two additional loss curves for the TREC and SNLI experiments arepresented. Both curves are shown in FIG.7. For both datasets, a convergence reduction for the Non-Byzantine setting is observed. For SNLI, the reduction appears greater, which aligns with the results in Table 2. A.4 Local Epochs

[0187] In the present experimental framework, the investigation can be extended toevaluate the performance of CYBER-0 under the context of local epochs. This involved adapting practical algorithm (Algorithm 2) to include local epochs, incorporating memory- efficient operations, as detailed in Algorithm 4.

[0188] The findings, as summarized in Table 6, reveal an intriguing parallel to the impactof the variable k. Specifically, modifying the number of local epochs appears to exert a comparable influence. A notable insight from this experiment is the relative stability of thefinal test accuracy, despite variations in batching these training epochs. However, it’s important to acknowledge that the efficacy of such a technique can be highly dependent on the specific problem at hand, suggesting a need for cautious interpretation and application in different contexts.

[0189] FIG. 7 shows Contrasting Byzantine and Non-Byzantine Scenarios Across DiverseData Distributions with RoBERTa-large on TREC and SNLI: This figure compares the performance of CYBER-0 using a RoBERTa-large model on both TREC and SNLI datasets. Non-Byzantine behavior stands for CYBER-0 with no Byzantine clients nor robust aggregation. Table 6: Assessing the Impact of Varying Local Epochs in SST-2 Using RoBERTa-large: This table provides a comparative analysis of how different settings for local epochs affect performance on the SST-2 dataset using the RoBERTa-large model on a Byzantine and non- IID setting.

[0190] Algorithm 4 CYBER-0- Experimental Setup with Local EpochsInput:0is the initial model parameter vector, β is the trimmed mean factor, η is the learning rate, µ is the perturbation step, k is the number of samples per estimate, E is the number of local epochs and T is the total number of learning steps. ▶ Federator Distributes a random seed s to each client Distributesi=0to each client Sets0for t = 1 to T do for i = 1 to in parallel do ▷ client i for e = 1 to E do / / Sample k grad estimates forend for / / Apply local epoch learning step for r = 1 to k do s′ (s, t, r, e)end for / / Reset model to start of local epochs (alternatively, can store initial model, with extra 610 memory cost) for e = 1 to E do for r = 1 to k doend for end for end for / / Federator robust aggregation ▶ Federator for e = 1 to E doend for end / / clients synchronization for i = 1 to in parallel do ▷ client i for e = 1 to E do for r = 1 to k do s′end for end for end for end for function ZOGRAD(F, , B, µ, s′) / / Reset model state Return^^+^^−endfunction PERTURBPARAMS( , µ, s′)Set RNG seed as s′ for eachz efficient endend function

Claims

CLAIMS 1. A computer-implemented method for training a model using a central node and a plurality of client nodes, comprising, at one or more of the plurality of client nodes: receiving a set of model parameters and an initial seed from the central node; determining an update direction based on the initial seed; computing a scalar reflecting a sign and a size of an approximation of a projection of a gradient of a loss function on the update direction, using the set of model parameters; sending the scalar of the approximated gradient projection to the central node; receiving a model update scalar from the central node; and updating the set of model parameters in the update direction based on the model update scalar.

2. The method according to claim 1, wherein determining the update direction based on the initial seed comprises: constructing a current seed for each iteration during training the model based on the initial seed, preferably wherein the current seed is the same in each iteration during training the model; designating the current seed for a pseudo-random number generator, PRNG; sampling a PRNG result from the PRNG; and generating the update direction based on the PRNG result.

3. The method of claim 1 or 2, wherein computing the scalar of the approximation of the projection of the gradient of the loss function on the update direction comprises: determining the gradient using backpropagation; and projecting the gradient on the update direction to obtain the scalar of the approximation of the gradient projection.

4. The method of claim 1 or 2, wherein computing the scalar of the approximation of the projection of the gradient of the loss function on the update direction comprises: evaluating the loss function using the set of model parameters perturbed into the update direction, preferably wherein evaluating the loss function using the set of model parameters perturbed into the update direction comprises: perturbing the set of model parameters; updating the set of model parameters to a set of perturbed model parameters;calculating a first perturbed value and a second perturbed value based on the set of perturbed model parameters; and determining the approximated gradient of the loss function in the update direction based on the first perturbed value and the second perturbed value.

5. The method of any of claims 1 to 4, wherein the model comprises at least one of Linear Regression, Logistic Regression, Support Vector Machine, Decision Tree, Ensemble Methods, Generalized Linear Model, Neural Networks, Deep Learning Model, and fine-tuning Large Language Models.

6. A computer-implemented method for orchestrating training a model using a central node and a plurality of client nodes, comprising, at the central node: sending a set of model parameters and an initial seed to the plurality of client nodes; and iteratively executing, until a preset condition is satisfied: receiving a plurality of scalars of approximated gradient projections from the plurality of client nodes; aggregating them to obtain a model update scalar; and sending the model update scalar to the plurality of client nodes.

7. The method of claim 6, further comprising computing an updated model at the central node at every iteration or at the end.

8. The method of claim 6 or 7, further comprising determining the update direction based on the initial seed, wherein determining the update direction based on the initial seed comprises: constructing a current seed for each iteration during training the model based on the initial seed, preferably wherein the current seed is the same in each iteration during training the model; designating the current seed for a pseudo-random number generator, PRNG; sampling a PRNG result from the PRNG; and generating the update direction based on the PRNG result.

9. The method according to any of claims 6 to 8, wherein the aggregating the plurality of the scalars of the approximated gradient projections to obtain the model update scalar is achieved by an aggregation rule.

10. The method according to any of claims 6 to 9, wherein the preset condition comprises one or more of convergence of the model and a desired performance metric.

11. A client node configured to perform the method of any of claims 1 to 5.

12. A central node configured to perform the method of any of claims 6 to 10.

13. A distributed machine learning system, comprising: a plurality of client nodes according to claim 11; and a central node according to claim 12.

14. The distributed machine learning system of claim 13, wherein communication between the plurality of client nodes and the central node is secured by end-to-end encryption.

15. A computer-readable storage medium storing program code, the program code comprising instructions that when executed by a processor carry out the method of any of claims 1 to 10.

Images