Devices, system and methods for distributed machine learning

Abstract

This disclosure relates to distributed machine learning, such as federated learning. This disclosure proposes a client device outputting a set of learned local weights after training a local machine learning model using a local dataset. Each learned local weight is of a Boolean data type. The client device is configured to send the set of learned local weights to a server. After receiving multiple sets of learned local weights from multiple client devices, the server is configured to aggregate corresponding local weights and determine a set of global weights based thereon. Then, the server is configured to send the global weights to one or more client devices. Each global weight is of a Boolean data type. By exchanging the one-bit binary local weight and the one-bit binary global weight between each client device and the server, the communication overhead for distributed machine learning can be reduced.

Description

[0001] DEVICES, SYSTEM AND METHODS FOR DISTRIBUTED MACHINE LEARNING

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to devices, methods and a system for distributed neural networks in the field of computer technology. For instance, the disclosure proposes devices, methods and a system for distributed machine learning.

[0004] BACKGROUND

[0005] Distributed machine learning, such as federated learning, is a popular operation mode of deep learning that avoids the collection of user data on a central server so as to protect data privacy. Distributed machine learning typically includes a central computing server and a certain number of edge computing devices (also referred to as clients or client devices). Each of the edge computing devices performs the training of a neural network model with its own local data. The central server coordinates the operation of the whole system, specifies information with precise conditions that each edge computing device will report to it, updates the neural network model based on the information reported from the edge computing devices, and sends the updated model to the edge computing devices.

[0006] The inherent design of distributed machine learning ensures privacy as each client participates in training on the device and shares only model updates with the central server. Another advantage of distributed machine learning is that it distributes the computational load across networked clients.

[0007] Most approaches to distributed machine learning (e.g., federated learning) rely on synchronous operations. At each global iteration, a copy of the current model is sent from the central server to a selected subset of edge devices. The edge devices then update their model parameters using their private data and send the model updates back to the server. The server aggregates these updates to create a new shared model. This process is repeated until the shared model meets a desired criterion. The edge devices selected in each round must wait for the slowest one before starting the next round of computation. As this waiting time can be important in practice, the field of distributed machine learning (e.g., federated learning) has now been extended to the framework of asynchronous operations. In this case, the central server and edge devices typically operate with inconsistent versions of the shared model, and synchronization in lockstep is not required, even between participants in the same round.

[0008] SUMMARY

[0009] The present disclosure is based on the following further considerations.

[0010] Distributed machine learning (e.g., federated learning) ensures privacy as each client participates in training on the device and shares only model updates with the central server. However, this feature creates a major bottleneck: communicating updates from local edge devices to the central server. The drastic increase both in the number of parameters and of clients has made this problem even more acute. The training of neural networks in a distributed environment involves a high bandwidth consumption for exchanging large-dimensional tensors between the edge devices and the central server. Even in lightweight architectures dedicated to mobile devices, parameters can easily exceed the size of tens of megabytes (MB). Hence, sending a large amount of data in each communication round may result in an unacceptable delay and seriously damage the system performance.

[0011] A solution to tackle this issue is communication compression, which consists of applying lossy compression to the parameters or gradients to be transmitted. That is, distributed machine learning systems may involve training of latent (i.e. full-precision) weights (on each client device), compression of the parameters (or gradients) to be transmitted (on each client device), and averaging the compressed value of the parameters (on the server).

[0012] In view of the above, this disclosure aims to improve distributed machine learning. An objective is to reduce communication overhead for distributed machine learning.

[0013] This and other objectives are achieved by the solutions of this disclosure as described in the independent claims. Advantageous implementations are further defined in the dependent claims.

[0014] A first aspect of the present disclosure provides a client device for machine learning. The client device is configured to obtain a local dataset for training a local machine learning model; train the local machine learning model based at least in part on the local dataset to obtain a set of learned local weights of the local machine learning model; and provide the set of learned local weights to a server. Each learned local weight is of a Boolean data type.

[0015] It is noted that the local machine learning model on each client device may be any type of machine learning model comprising local weights of a Boolean data type. The weight of Boolean data type may also be referred to as a binary weight.

[0016] In this way, each weight can be represented using one-bit information. The communication overhead between each client device and the server can be significantly reduced. The training process of federated learning requires reduced bandwidth (e.g., 1 -bit per weight per coordinate).

[0017] A further effect is that the training process requires much less memory and computational power because full-precision computations are avoided.

[0018] In an implementation form of the first aspect, the client device may be further configured to obtain a set of global weights from the server, and update the local machine learning model based on the set of global weights. The set of global weights is determined by taking account of the set of learned local weights. Each global weight is of a Boolean data type.

[0019] In this way, each global weight can be represented using one-bit information. The model parameter update from the server to each client device requires reduced bandwidth (e.g., in comparison to full precision model training and update).

[0020] In a further implementation form of the first aspect, for training the local machine learning model based at least in part on the local dataset, the client device may be configured to optimize a set of local weights of the local machine learning model to obtain the set of learned local weights based at least in part on the local dataset such that at least a subset of the local weights are flipped. Each local weight is of a Boolean data type.

[0021] It is noted that in each training round, each local weight obtained after training may be referred to as a trained local weight. Each trained local weight may be either flipped or not flipped.

[0022] In a further implementation form of the first aspect, before training the local machine learning model, the client device may be configured to obtain a set of initial weights, and initialize the local machine learning model based on the set of initial weights. Each initial weight is of a Boolean data type.

[0023] Optionally, the set of initial weights may be provided through the server.

[0024] Alternatively, for obtaining the set of initial weights, each client device may be configured to receive a common random seed from the server. Each client device performs random initialization of each local machine learning model based on the random seed. For instance, the set of initial weights may be determined based on the random seed. As the client devices receive the same random seed, the local machine learning models of the client devices are initialized with the same set of initial weights. In this way, the transmission overhead between the server and each client device can be reduced.

[0025] In a further implementation form of the first aspect, the local machine learning model may comprise a Boolean neural network.

[0026] The Boolean neural network comprises a plurality of layers (e.g., an input layer, one or more hidden layers, and an output layer). Each layer comprises a plurality of neurons. Each neuron corresponds to a Boolean function (also referred to as a logical operation, e.g., NOT, AND, OR, XOR, NAND, NOR, XNOR, etc.) associated with at least one binary weight. Each neuron is adapted to receive at least one binary input, apply the at least one binary weight to the at least one binary input, and apply the at least one weighted binary input to the Boolean function to obtain an activation value.

[0027] A second aspect of the present disclosure provides a server for machine learning. The server is configured to obtain, from each of a plurality of client devices, a set of learned local weights of a local machine learning mode; determine a set of global weights based on the multiple sets of learned local weights; and provide the set of global weights to one or more client devices of the plurality of client devices.

[0028] Each learned local weight and each global weight are of a Boolean data type. In this way, the communication overhead between the server and each client device can be significantly reduced. The synchronization process of federated learning requires reduced bandwidth.

[0029] In an implementation form of the second aspect, for determining the set of global weights, the server may be configured to determine whether each global weight is flipped or not based on the multiple sets of learned local weights using a majority rule.

[0030] In a further implementation form of the second aspect, the majority rule may be associated with an adjustable threshold.

[0031] In a further implementation form of the second aspect, the server may be further configured to: aggregate a plurality of corresponding learned local weights of the multiple sets of learned local weights, to obtain an aggregation of the corresponding learned local weights; assign a corresponding global weight with a first Boolean value in response to determining that a value of the aggregation of the corresponding learned local weights exceeds the adjustable threshold; and assign the corresponding global weight with a second Boolean value in response to determining that the value of the aggregation of the corresponding learned local weights is below a negative value of the adjustable threshold.

[0032] It is noted that in response to determining that the value of the aggregation of the corresponding learned local weights is between the adjustable threshold and the negative value of the adjustable threshold, the server is adapted to do nothing for the corresponding global weight. That is, the corresponding global weight is kept unchanged. In another word, the corresponding global weight is assigned with a previous value (e.g., determined in a previous round).

[0033] In a further implementation form of the second aspect, the server may be configured to provide a set of initial weights to each of the plurality of client devices for initializing the local machine learning model of each client device. Each initial weight is of a Boolean data type.

[0034] Alternatively, the server may be configured to send a random seed to each of the plurality of client devices. The random seed is for initializing the local machine learning model of each client device. The local machine learning model is initialized with a set of initial weights of a Boolean data type.

[0035] A third aspect of the present disclosure provides a system for machine learning. The system comprises a plurality of client devices each according to the first aspect or any implementation form thereof, and a server according to the second aspect.

[0036] A fourth aspect of the present disclosure provides a method for machine learning. The method comprises the following steps: obtaining, by a client device, a local dataset fortraining a local machine learning model;

[0037] - training, by the client device, the local machine learning model based at least in part on the local dataset to obtain a set of learned local weights of the local machine learning model, in which each learned local weight is of a Boolean data type; and providing, by the client device, the set of learned local weights to a server.

[0038] In an implementation form of the fourth aspect, the method may further comprise the following steps: obtaining, by the client device, a set of global weights from the server, and

[0039] - updating, by the client device, the local machine learning model based on the set of global weights.

[0040] The set of global weights is determined by taking account of the set of learned local weights. Each global weight is of a Boolean data type.

[0041] In a further implementation form of the fourth aspect, the step of training the local machine learning model based at least in part on the local dataset may comprise optimizing, by the client device, a set of local weights of the local machine learning model to obtain the set of learned local weights based at least in part on the local dataset such that at least a subset of the local weights are flipped. Each local weight is of a Boolean data type.

[0042] In a further implementation form of the fourth aspect, before training the local machine learning model, the method may further comprise: obtaining, by the client device, a set of initial weights, and initializing, by the client device, the local machine learning model based on the set of initial weights. Each initial weight is of a Boolean data type.

[0043] In a further implementation form of the fourth aspect, the local machine learning model may comprise a Boolean neural network.

[0044] A fifth aspect of the present disclosure provides a method for machine learning. The method comprises the following steps: obtaining, by a server from each of a plurality of client devices, a set of learned local weights of a local machine learning mode; determining, by the server, a set of global weights based on the multiple sets of learned local weights; and providing, by the server, the set of global weights to each of the plurality of client devices.

[0045] Each local weight and each global weight are of a Boolean data type

[0046] In an implementation form of the fifth aspect, the step of determining the set of global weights may comprise determining, by the server, whether each global weight is flipped or not based on the multiple sets of learned local weights using a majority rule.

[0047] In a further implementation form of the fifth aspect, the majority rule may be associated with an adjustable threshold.

[0048] In a further implementation form of the fifth aspect, the method may further comprise: aggregating, by the server, a plurality of corresponding learned local weights of the multiple sets of learned local weights, to obtain an aggregation of the corresponding learned local weights; assigning, by the server, a corresponding global weight with a first Boolean value in response to determining that a value of the aggregation of the corresponding learned local weights exceeds the adjustable threshold; and assigning, by the server, the corresponding global weight with a second Boolean value in response to determining that the value of the aggregation of the corresponding learned local weights is below a negative value of the adjustable threshold. In a further implementation form of the fifth aspect, the method may further comprise providing, by the server, a set of initial weights to each of the plurality of client devices for initializing the local machine learning model of each client device. Each initial weight is of a Boolean data type.

[0049] Alternatively, the method may further comprise sending, by the server, a random seed to each of the plurality of client devices. The random seed is for initializing the local machine learning model of each client device. The local machine learning model is initialized with a set of initial weights of a Boolean data type.

[0050] A sixth aspect of the present disclosure provides a computer program product. The computer program product comprises a program code for performing the method according to the fourth aspect or any implementation form thereof, when executed on a computer.

[0051] A seventh aspect of the present disclosure provides a computer program product. The computer program product comprises a program code for performing the method according to the fifth aspect or any implementation form thereof, when executed on a computer.

[0052] An eighth aspect of the present disclosure provides a non-transitory storage medium storing executable program code which, when executed by a processor, causes the method according to the fourth aspect or any implementation form thereof, or according to the fifth aspect or any implementation form thereof to be performed.

[0053] A ninth aspect of the present disclosure provides a chipset comprising a memory and a processor, which are configured to store and execute program code to perform the method according to the fourth aspect or any implementation form thereof, or according to the fifth aspect or any implementation form thereof.

[0054] It has to be noted that all devices, elements, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of the present disclosure, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof.

[0055] BRIEF DESCRIPTION OF DRAWINGS

[0056] The above-described aspects and implementation forms will be explained in the following description in relation to the enclosed drawings, in which:

[0057] FIG. 1 shows a client device and a server according to the present disclosure;

[0058] FIG. 2 shows a client device and a server according to the present disclosure;

[0059] FIG. 3 shows an example of a system comprising multiple client devices and a server;

[0060] FIG. 4 shows a diagram of a method of the present disclosure; and

[0061] FIG. 5 shows a diagram of another method of the present disclosure.

[0062] DETAILED DESCRIPTION OF EMBODIMENTS

[0063] Communication involved in distributed learning (e.g., federated learning) typically comprises at least two phases: an upload phase and a download phase. In the upload phase, trained weights on each client device are transmitted to a server. In the download phase, updated weights (for inference) are provided by the server to each client device.

[0064] FIG. 1 shows a client device 110 (or 130) and a server 120 according to the present disclosure. A federated learning system 100 is also depicted in FIG. 1. The system 100 comprises a plurality of client devices 110, 130 each according to the present disclosure and the server 120. The number of client devices shown in FIG. 1 is for illustration purposes only. In this disclosure, features introduced with respect to one client device may be similarly applied to any other client device, unless specified otherwise. FIG. 1 directs to the upload phase of the federated learning. The client device 110 (or 130) is configured to obtain a local dataset fortraining a local machine learning model 112 (or 132) and train the local machine learning model 112 based at least in part on the local dataset to obtain a set oflearned local weights 111 (or 131) of the local machine learning model 112. Then, the client device 110 is configured to provide the set of learned local weights to a server 120. Each learned local weight is of a Boolean data type.

[0065] It is noted that each local weight (whether learned or not learned) of the local machine learning model 112 is of a Boolean data type. During the training process of the local machine learning model, the client device 110 is configured to flip or not flip the value of each local weight. In this way, each weight can be represented using only one bit. The client device does not need to perform any other operations (e.g quantization or compression). By outputting the learned binary weight directly, the communication overhead of the distributed machine learning can be reduced.

[0066] For instance, each client device 110 may be adapted to use a Boolean neural network in the field of Boolean learning as the local machine learning model.

[0067] The Boolean neural network comprises a plurality of layers (e.g., an input layer, one or more hidden layers, and an output layer), and each layer comprises a plurality of neurons (like any other conventional neural network). For the Boolean neural network, each neuron corresponds to a Boolean function (also referred to as a logical operation, e.g., NOT, AND, OR, XOR, NAND, NOR, XNOR, etc.) associated with at least one binary weight. Each neuron is adapted to receive at least one binary input, apply the at least one binary weight to the at least one binary input, and apply the at least one weighted binary input to the Boolean function to obtain an activation value.

[0068] The Boolean neural network facilitates the computation of training / inferencing based on binary weights and approximate b ackpropagation rules based on the local dataset. This disclosure proposes to apply optimization rules as defined by Boolean learning directly at each client device. The optimization rules are used to determine whether each local binary weight is flipped or not during the training process. Then, the learned local binary weights 111 are forwarded to the server 120. The server 120 is configured to receive, from each client device 110, 130, a set of learned local binary weights 111, 131. Then, the server is configured to determine a set of global binary weights based on the multiple sets of learned local binary weights.

[0069] Optionally, for determining the set of global binary weights, the server 120 may be configured to aggregate the corresponding local binary weights of each set received from each client device and apply a majority rule to determine a corresponding global binary weight. For this purpose, the server 120 may comprise an aggregation unit 122 to determine the set of global binary weights.

[0070] Consider each local binary weight as a voting input, and the corresponding global binary weight as a voting result. The principle of the majority rule is that a voting result can be positive or negative. For the voting result to be positive, a majority of the voting input shall be positive; for the voting result to be negative, a majority of the voting input shall be negative. If there is neither a majority of the positive voting input nor a majority of the negative voting input, then it is not confident enough to conclude the voting. In this case, the corresponding global binary weight is kept unchanged (i.e., stick to the previous binary value). In this disclosure, an adjustable threshold may be used to control to what degree (what percentage) is a majority.

[0071] Optionally, before the training process, the client device 110 may be configured to initialize the local machine learning model. That is, the parameters of the local machine model are randomly assigned with initial values for training purposes. The initial values may be directly provided by the server 120. Alternatively, the server 120 may be configured to send a common random seed to each client device 110, 130, such that each client device 110, 130 initialise the local machine learning model using the common random seed. In this way, communication overhead can be further reduced. For instance, the common random seed may be used by each client device to generate a set of common randomized parameters of the local machine model. The initialization of the local machine learning model shall be performed only once. Once the client device 110 receives a set of global weights from the server, a further training may be performed based on the set of received global weights.

[0072] FIG. 2 shows a client device 110 (or 130) and a server 120 according to the present disclosure. FIG. 2 is built based on FIG. 1 and directs to the download phase of federated learning. After obtaining the set of global binary weights 221, the server 120 is configured to send the set of global binary weights 221 to at least a subset of the plurality of client devices 110, 130. The set of global binary weights 221 may be used for inferencing.

[0073] Optionally, the server 120 may be configured to send the set of global binary weights to each and every client device, which may be referred to as synchronized federated learning. Alternatively, the server 120 may be configured to send the set of global binary weights to a subset of the multiple client devices that have participated in the upload phase, which may be referred to as asynchronous federated learning. Additionally and / or alternatively, the server 120 may be configured to send the set of global weights to a client device that has not participated in the upload phase. Overall, the server 120 may be configured to send the set of global binary weights to any client device that needs the set of global weights for inferencing.

[0074] Optionally, the operation described in FIG. 1 and FIG. 2 may be iterated multiple times until a performance requirement (e.g., precision, or convergence) is fulfilled.

[0075] FIG. 3 shows an example of a system comprising multiple client devices and a server. In FIGs. 1-3, corresponding elements may share the same features and function likewise. FIG. 3 depicts a federated learning system comprising n client devices (n > 1). The learned local binary weights are transmitted by the n client devices to the server. In FIG. 3, a corresponding local binary weight is depicted as w 1, w_2, w_n. Each of w 1, w_2, w_n represents a corresponding parameter (e.g., a same parameter associated with a same neuron of a same layer) of a common machine learning model shared across the clients (i.e., local machine learning model). Since each client device trains the local machine learning model based on a local dataset of each own, the values of w 1, w_2, w_n may be different (or may be the same).

[0076] For instance, each client device may be configured to send the set of learned local binary weights as a Boolean array. The corresponding local binary weights of w 1, w_2, w_n may share the same location / index in each Boolean array sent by each client device.

[0077] The server is configured to aggregate (e.g., sum) all the corresponding weights and determine a server Boolean weight (i.e., global binary weight) according to the aggregation result. The aggregation may be performed as p= wJ, where p is the aggregation result. For determining the server Boolean weight, a majority (or voting) rule may be used, such as follows: w = positive (e.g., 1) ifp > nT w = negative (e.g., -1) ifp < -nT, w is not updated, if -nT<=p<=nT where w is one of the global binary weights.

[0078] It is noted that the value of { 1, -1 } above for the global binary weight is merely given for illustration purposes only. In the present disclosure, any data of a Boolean data type may be assigned with a first value or a second value, as long as the first value and the second value fulfill the Boolean data type. For instance, each local weight (trained or untrained) and / or each global weight may have a value between {TRUE, FLASE}, or a value between { 1, 0}, or a value between { 1, -1 }. It is noted that when each local weight has a value between {TRUE, FLASE} or { 1, 0}, each local weight may be mapped to { 1, -1 } when applying the majority rule (e.g., when aggregated on the server).

[0079] A threshold value T may be set according to various requirements. For instance, if a strict consensus needs to be applied between clients, T may be set close to the value of one (e.g., T = 0.8 or higher). An intermediate setting (e.g., T = 0.5) may allow client devices to report different binary weights and still allow the server to flip weight based on the majority rule.

[0080] FIG. 4 shows a diagram of a method 400 of the present disclosure. The method 400 is performed by a client device for (distributed) machine learning (e.g., federated learning). The method 400 comprises the following steps: step 401 : obtaining, by the client device, a local dataset for training a local machine learning model; step 402: training, by the client device, the local machine learning model based at least in part on the local dataset to obtain a set of learned local weights of the local machine learning model, wherein each learned local weight is of a Boolean datatype; and step 403: providing, by the client device, the set of learned local weights to a server. The steps of the method 400 may share the same functions and details from the perspective of FIG. 1-3 described above. Therefore, the corresponding method implementations are not described again at this point.

[0081] FIG. 5 shows a diagram of a method 500 of the present disclosure. The method 700 is performed by a server for (distributed) machine learning (e.g., federated learning).. The method 500 comprises the following steps: step 501 : obtaining, by the server from each of a plurality of client devices, a set of learned local weights of a local machine learning mode; step 502: determining, by the server, a set of global weights based on the multiple sets of learned local weights; and step 503: providing, by the server, the set of global weights to each of the plurality of client devices, in which each local weight and each global weight are of a Boolean data type.

[0082] The steps of the method 500 may share the same functions and details from the perspective of FIG. 1-3 described above. Therefore, the corresponding method implementations are not described again at this point.

[0083] The present disclosure may be applied to scenarios where distributed machine learning (e.g., federated learning) is employed. A typical system may comprise client devices (or edge devices) such as smart phones, wearable devices, and autonomous driving cars that collect user’ s specific data for a specific deep learning task. The edge devices have suitable computing capabilities required for deep learning. For instance, each edge device may comprise one or more artificial intelligent (Al) chips such as graphics processing unit (GPU), tensor processing unit (TPU), neural processing unit (NPU), and a combination thereof. The server may be a computing device that has much higher computing capability than that of the edge devices. An example of the server may be a base station (or an application server) for a mobile cellular communication system (e.g., a 4G, 5G, 6G network, and beyong). In such a scenario, the two-way communication between the edge devices and the server may be performed by means of the underlying cellular communication system. According to this disclosure, the communication overhead between the edge devices and the server can be significantly reduced. Moreover, the training phase on each edge device does not require too much computing capability (vs. full- precision training). Therefore, the hardware requirements for the edge device can be simplified. The power consumption of the edge device can also be reduced.

[0084] It is noted that the client device 110 / 130 and the server 120 in the present disclosure may each comprise processing circuitry (not shown) configured to perform, conduct or initiate the various operations of the client device 110 / 130 and the server 120 described herein, respectively. The processing circuitry may comprise hardware and software. The hardware may comprise analog circuitry or digital circuitry, or a combination thereof. The digital circuitry may comprise components such as application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), digital signal processors (DSPs), or multi-purpose processors. The processing circuitry may comprise one or more processors (or chipsets) and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code which, when executed by the one or more processors, causes the client device 110 / 130 and the server 120 to perform, conduct or initiate the operations or methods described herein, respectively.

[0085] In the present disclosure, the client device 110 / 130 may be an electronic device capable of computing, such as a computer, a tablet, a vehicle, an loT device, and the like. The server 120 may comprise one or more electronic devices capable of computing, such as a computer, a server blade, a computer cluster, a virtual machine based on computer architectures, a tablet, an Al chip, and the like.

[0086] Moreover, a type of machine learning model in the present disclosure is not limited. For example, the neural network in the present disclosure may be a feedforward neural network, a convolutional neural network, a recurrent neural network, a deep neural network, or the like. Moreover, neurons of the neural network may be fully connected, or partially connected. In principle, the present disclosure may be applied to any type of machine learning model that is for Boolean learning.

[0087] The present disclosure has been described in conjunction with various embodiments as examples as well as implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed matter, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

CLAIMS1. A client device (110) for machine learning, the client device (110) being configured to: obtain a local dataset for training a local machine learning model (112); train the local machine learning model (112) based at least in part on the local dataset to obtain a set of learned local weights (111) of the local machine learning model, wherein each learned local weight is of a Boolean data type; and provide the set of learned local weights (111) to a server (120).

2. The client device (110) according to claim 1, wherein the client device (110) is further configured to: obtain a set of global weights from the server (120), wherein the set of global weights is determined by taking account of the set of learned local weights (111), and each global weight is of a Boolean data type; and update the local machine learning model (112) based on the set of global weights.

3. The client device (110) according to claim 1 or 2, wherein for training the local machine learning model (112) based at least in part on the local dataset, the client device (110) is configured to: optimize a set of local weights of the local machine learning model to obtain the set of learned local weights (111) based at least in part on the local dataset such that at least a subset of the local weights are flipped, wherein each local weight is of a Boolean data type.

4. The client device (110) according to any one of claims 1 to 3, wherein before training the local machine learning model (112), the client device (110) is configured to: obtain a set of initial weights; and initialize the local machine learning model (112) based on the set of initial weights, wherein each initial weight is of a Boolean data type.

5. The client device (110) according to any one of claims 1 to 4, wherein the local machine learning model (112) comprises a Boolean neural network.

6. A server (120) for machine learning, the server (120) being configured to: obtain, from each of a plurality of client devices (110, 130), a set of learned local weights (111, 113) of a local machine learning mode; determine a set of global weights (221) based on the multiple sets of learned local weights (111, 113); and provide the set of global weights (221) to one or more client devices of the plurality of client devices (110, 130), wherein each local weight and each global weight are of a Boolean data type.

7. The server (120) according to claim 6, wherein for determining the set of global weights (221), the server (120) is configured to: determine whether each global weight is flipped or not based on the multiple sets of learned local weights using a majority rule.

8. The server (120) according to claim 7, wherein the majority rule is associated with an adjustable threshold.

9. The server (120) according to claim 8, wherein the server (120) is configured to: aggregate a plurality of corresponding learned local weights of the multiple sets of learned local weights, to obtain an aggregation of the corresponding learned local weights; assign a corresponding global weight with a first Boolean value in response to determining that a value of the aggregation of the corresponding learned local weights exceeds the adjustable threshold; and assign the corresponding global weight with a second Boolean value in response to determining that the value of the aggregation of the corresponding learned local weights is below a negative value of the adjustable threshold.

10. The server (120) according to any one of claims 6 to 9, wherein the server (120) is configured to provide a set of initial weights to each of the plurality of client devices (110, 130) for initializing the local machine learning model of each client device, wherein each initial weight is of a Boolean data type.

11. A system comprises one or more client devices (110, 130) each according to any one of claims 1 to 5, and a server (120) according to any one of claims 6 to 10.

12. A method (400) for machine learning, the method comprising: obtaining (401), by a client device, a local dataset for training a local machine learning model; training (402), by the client device, the local machine learning model based at least in part on the local dataset to obtain a set of learned local weights of the local machine learning model, wherein each learned local weight is of a Boolean data type; and providing (403), by the client device, the set of learned local weights to a server.

13. A method (500) for machine learning, the method comprising: obtaining (501), by a server from each of a plurality of client devices, a set of learned local weights of a local machine learning mode; determining (502), by the server, a set of global weights based on the multiple sets of learned local weights; and providing (503), by the server, the set of global weights to each of the plurality of client devices, wherein each local weight and each global weight are of a Boolean data type.

14. A computer program comprising instructions which, when the program is executed by a second computer, cause the second computer to perform the method according to claims 12

Images