Decentralized federated learning using random walks on communication graph

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for training machine learning models. One example method generally includes receiving, at a device, optimization parameters, parameters of a machine learning model, and optimization state values to be updated based on a local data set. Parameters of the machine learning model and the optimization state values of the optimization parameters are updated based on the local data set. A peer device is selected to refine the machine learning model based on a graphical data object that includes connections between the device and a plurality of peer devices including the peer device. The updated parameter and the updated optimization state value are transmitted to the selected peer device for refinement by the selected peer device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Greek patent application serial number 20220100404, filed on May 17, 2022, entitled “Decentralized Federated Learning Using a Random Walk Over a Communication Graph,” which application is assigned to the assignee of this application, and the entire contents of which are hereby incorporated by reference. Background Art

[0003] Aspects of the present disclosure relate to machine learning.

[0004] Federated learning generally refers to various techniques that allow training of machine learning models to be distributed across multiple client devices, which beneficially allows a wide variety of data to be used to train the machine learning models. For example, using federated learning to train machine learning models for facial recognition can allow these machine learning models to be trained from a wide range of datasets that include different sets of facial features, different amounts of contrast between foreground data of interest (e.g., a person's face) and background data, etc.

[0005] In some examples, federated learning can be used to learn embeddings across multiple client devices. However, sharing the embeddings of a model may not be appropriate because the embeddings of the model may contain client-specific information. For example, the embeddings may expose data from which sensitive data used in the training process can be reconstructed. Therefore, for machine learning models trained for security-sensitive applications or privacy-sensitive applications (such as biometric authentication or medical applications), sharing the embeddings of the model may expose data that can be used to undermine the biometric authentication application or cause privacy loss of other sensitive data.

[0006] Therefore, there is a need for improved techniques for training machine learning models using federated learning techniques. Summary of the invention

[0007] Certain aspects provide a method for training a machine learning model. The method generally includes receiving at a device an optimization parameter, a parameter of the machine learning model, and an optimization state value to be updated based on a local data set. The parameters of the machine learning model and the optimization state value of the optimization parameter are updated based on the local data set. A peer device is selected to refine the machine learning model based on a graph data object, the graph data object including connections between the device and a plurality of peer devices including the peer device. The updated parameters and the updated optimization state value are transmitted to the selected peer device for refinement by the selected peer device.

[0008] Other aspects provide: a processing system configured to perform the aforementioned methods and those described herein; a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the aforementioned methods and those described herein; a computer program product embodied on a computer-readable storage medium, the computer program product comprising code for performing the aforementioned methods and those further described herein; and a processing system comprising components for performing the aforementioned methods and those further described herein.

[0009] The following description and associated drawings set forth in detail certain illustrative features of the one or more aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings depict certain aspects of the disclosure and therefore are not to be considered limiting of the scope of the disclosure.

[0011] Figure 1 Depicted is an example environment in which a machine learning model is trained using a decentralized federated learning scheme and a random walk traversal of a communication graph representing connections between client devices in the environment, in accordance with aspects of the present disclosure.

[0012] Figure 2 Depicted is example pseudocode for training a machine learning model using a decentralized federated learning scheme and a random walk traversal of a communication graph representing connections between client devices in a computing environment in accordance with aspects of the present disclosure.

[0013] Figure 3 Depicted are examples of a parallelized hyperparameter search and update process across participating devices in a federated learning scheme in accordance with aspects of the present disclosure.

[0014] Figure 4 Depicted are example operations that may be performed by a client device for training a machine learning model in accordance with aspects of the present disclosure.

[0015] Figure 5 Depicted are example implementations of processing systems in which machine learning models may be trained in accordance with aspects of the present disclosure.

[0016] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one aspect may be beneficially incorporated in other aspects without further recitation. DETAILED DESCRIPTION

[0017] Aspects of the present disclosure provide apparatus, methods, processing systems, and computer-readable media for training a machine learning model using a decentralized federated learning scheme and a communication graph illustrating connections between participating devices in the federated learning scheme.

[0018] In a system that uses federated learning to train a machine learning model, the machine learning model is typically defined based on model updates (e.g., changes to weights or other model parameters) generated by each of a plurality of participating client devices. In general, each of these client devices can train the model using data stored locally on the client device. By doing so, a wide variety of data can be used to train the machine learning model, which can reduce the likelihood that the resulting global machine learning model will underfit the data (e.g., resulting in a model that neither fits the training data nor generalizes to new data) or overfit the data (e.g., resulting in a model that fits the training data too closely so that new data is inaccurately generalized).

[0019] In many cases, training a machine learning model using federated learning can be controlled by a central server that coordinates the training process. When coordinating the training process, the central server can select a group of client devices to participate in updating the machine learning model, provide information about the current version of the machine learning model to the group of client devices, and receive and aggregate updates to the machine learning model from each client device in the group of client devices. However, when using federated learning to train a global machine learning model, the shared information (such as embeddings) generated by each of the participating client devices may compromise the security and privacy of the data used to train the machine learning model on the client device. For example, because the embeddings generated by the participating client devices are tightly coupled to the data used to generate the embeddings, sharing the embeddings between different client devices or to a server that coordinates the training of the machine learning model may expose sensitive data (e.g., so that the underlying data can be reconstructed or at least substantially reconstructed based on the embedded representation of the underlying data). Therefore, sharing the embeddings generated by the client device with other devices in a federated learning environment may cause security risks (e.g., for biometric data used to train a machine learning model deployed in a biometric authentication system) or may expose private data to unauthorized parties.

[0020] In order to mitigate privacy and security issues that may arise from sharing updates to machine learning models with a central server, a decentralized process can be used to train machine learning models using federated learning. In one example, a participating client device in a federated learning scheme can broadcast updates to the machine learning model to other participating client devices. However, broadcasting updates to participating client devices in a federated learning scheme may increase the communication overhead involved in the federated learning process. In another example, a participating client device can randomly select a peer device to update the model without adapting various optimization parameters for updating the model. However, updating the model in this manner may result in a model with poor inference performance because the resulting model may overfit or underfit the training data set and / or may otherwise inaccurately generate inferences for various inputs.

[0021] Various aspects of the present disclosure provide techniques for training and updating machine learning models using decentralized federated learning techniques, which improve security and privacy when sharing embedded data generated by client devices, while also producing models with good inference performance, compared to conventional federated learning techniques. To this end, a client device training a machine learning model uses a communication graph to randomly select the next peer client device to update the machine learning model (e.g., using a random walk procedure through the communication graph). Various parameters and other information about the machine learning model (such as optimizer state data) may be provided to the selected client device to adapt the rate at which the machine learning model changes during each iteration of the federated learning process. By using decentralized federated learning techniques to update machine learning models, various aspects of the present disclosure can participate in the federated learning process without exposing information about the basic data set used to train the machine learning model to potentially untrusted parties (e.g., a central server). Therefore, the security and privacy of the data in the basic data set used to train the machine learning model can be protected, which can improve the security and privacy of the data used when training the machine learning model using federated learning techniques relative to federated learning methods coordinated by a third party (e.g., a central server). Furthermore, because each participating client device may receive optimizer state data that the client device may use to control the rate at which the machine learning model changes, aspects of the present disclosure may allow the use of decentralized federated learning techniques to train machine learning models with sufficient inference performance.

[0022] Example decentralized federated learning architecture

[0023] Figure 1An example environment 100 is depicted in which a machine learning model is trained using a decentralized federated learning technique by multiple client devices. As illustrated, the environment 100 includes multiple client devices 102, 104, 106, 108, 110, and 112. Although for simplicity of illustration, Figure 1 Six client devices are illustrated in FIG. 1 , but it should be understood that environment 100 in which a machine learning model is trained using a decentralized federated learning technique may include any number of client devices.

[0024] The plurality of client devices 102-112 may be designated as a client device pool S, where each client device s∈S accesses N s Local dataset of data points To train a machine learning model using federated learning, the set of model parameters w can be learned according to the following equation:

[0025]

[0026] Where p(s) represents the probability distribution And f(w,ξ) represents the loss function calculated for the parameter w on the data sample ξ. The model parameter set w * Typically corresponds to a dataset based on a pool S associated with client devices 102-112 The optimized w of uniform random sampling of the global data distribution on the aggregated data. As discussed, in a typical federated learning scheme, a central server or other coordinator can learn a set of model parameters w by exchanging information about model updates between client devices participating in the federated learning scheme and the central server, which risks leaking private or sensitive information to unknown parties.

[0027] To mitigate the risk of sharing private or sensitive information in a federated learning scheme, the training of the machine learning model can be distributed among a group of client devices without using a central server or other coordinator to manage the training of the machine learning model. To this end, the client devices 102-112 in the environment 100 can be organized into a communication graph The pool S of client devices 102-112 is connected by a set of communication links E. In general, the communication diagram It can be a connection graph such that each client device in the communication graph is connected to at least one other client device in the communication graph. The communication graph can be a non-bipartite graph, which allows client devices to be indirectly connected to other client devices. For example, as illustrated in the figure, client device 102 can be directly connected to client devices 104, 106, 110 and 112, and can be indirectly connected to client device 108 (for example, it can be connected to client device 108 through any one of client devices 106 or 110). Finally, the communication graph can be a symmetric graph, in which the connection between the connected client devices is bidirectional (for example, for a given pair of client devices A and B, client device A can transmit information about the machine learning model to client device B, and client device B can transmit information about the machine learning model to client device A).

[0028] In this graph, each client device s has a set of neighbors defined by the connections available to the client device s. For example, environment 100 can be illustrated as having the following connected diagram. Client device 102 is illustrated as having direct communication links with client devices 104, 106, 110, and 112 (e.g., client device 102 has neighboring client devices 104, 106, 110, and 112). Client device 104 is illustrated as having direct communication links with client devices 102 and 106. Client device 106 is illustrated as having direct communication links with client devices 102, 104, and 108. Client device 108 is illustrated as having direct communication links with client devices 106 and 110. Client device 110 is illustrated as having direct communication links with client devices 108, 112, and 102. Finally, client device 112 is illustrated as having direct communication links with client devices 110 and 102.

[0029] In general, communication diagrams The client devices in the pool S in can be various devices, such as smart phones, Internet of Things (IoT) devices, or other computing devices that can communicate with each other and can participate in the federated learning process. The connection between these devices can be based on physical networks, virtual networks, routing time, trust between the owners of these devices, social network connections between the owners of these devices, etc.

[0030] In order to train a machine learning model in environment 100 and learn a set of model parameters w, the machine learning model can be trained based on a random walk through a communication graph representing environment 100. In the random walk procedure, a client device can randomly select a neighboring client device as the next device to train the machine learning model (e.g., updating parameter w based on local data at the selected client device). The parameters of the machine learning model and other optimization settings and state variables (as discussed in further detail below) can be provided to the selected client device, and the selected client device can update the parameters of the machine learning model based on the optimization settings, state variables, and local data at the selected client device. After updating the machine learning model, the selected client device can determine whether to randomly select a neighboring client device to further update the model or to perform another update locally using the local data at the selected client device.

[0031] For example, as illustrated in the figure, an initial loop of training (or updating) a machine learning model using a federated learning scheme may be performed at a client device 102, which has a neighbor set When client device 102 determines that another client device in environment 100 should update the machine learning model based on the local data at the other client device, client device 102 may therefore randomly select one or more of client devices 104, 106, 110, or 112 to train (or update) the machine learning model. As illustrated, at stage 120, client device 102 randomly selects client device 106 as the next client device to train (or update) the machine learning model. Similarly, after client device 106 trains (or updates) the machine learning model based on the local data at client device 106, client device 106 selects one or more of client devices 104, 106, 110, or 112 from the neighbor set. At stage 122, as illustrated, client device 106 selects client device 108 as the next client device for training (or updating) the machine learning model. Finally, at stage 124, client device 108 selects a neighboring client device from the set of neighboring client devices. In some aspects, the client device 110 is selected as the next client device for training (or updating) the machine learning model. In some aspects, the client device may monitor for confirmation or other information indicating that the selected next client device has received information about the machine learning model and has begun training (or updating) the machine learning model. If the client device does not receive a confirmation message within a threshold amount of time, the client device may resend the information about the machine learning model or reselect a neighboring client device to train (or update) the model. In some aspects, information about client device performance can be shared among client devices 102-112 in the environment 100. In general, when multiple client devices determine that a given client device is nonperformant (e.g., unresponsive, slow to respond, etc.), information about the given client device may be published in the environment 100 so that the given device is not considered a viable candidate for selection as the next client device for training (or updating) the machine learning model

[0032] To optimize the model parameter w, the communication graph representing the client devices 102-112 in the environment 100 The random walk procedure on the client devices 102-112 may be defined such that the random walk procedure has a desired stable distribution (e.g., such that ). By defining the communication diagram The random walk procedure on The global loss function can be optimized asymptotically so that the global loss function asymptotically approaches the minimum value.

[0033] According to the communication diagram To define the adjacency matrix A∈{0,1} representing the connections of the N client devices 102-112 N×N If node i is connected to node j, then A ij =A ji =1; otherwise, A ij =A ji = 0. The transfer matrix P can be defined as N client devices in the environment 100 (and therefore, the communication graph In this transfer matrix P, the transaction from node j to node i can be expressed by the following equation: where p(i|j) represents the probability distribution of selecting node i given the previous selection of node j for updating the machine learning model, and Returns the number of neighbors of a given node (e.g., communication graph Therefore, the transfer matrix P can be an inverse diagonal matrix D, where D includes the degree of each node in the diagonal of the matrix.

[0034] In some aspects, a method for selecting a neighboring client device (in a communication diagram) to which an update is to be sent The strategy of the node (denoted as node in FIG) can be based on a uniform random selection of neighboring client devices, which is given by the equation P = AD -1 Representation. The transfer from one client device to another client device can rely on local information (e.g., information about other client devices connected to a particular client device). However, in order to implement a stable distribution on the client device so that the global loss function of the machine learning model is progressively optimized, a Metropolis-Hastings adjustment of the probability distribution can be used to select a specific neighboring client device as the next client device for training (or updating) the model. The Metropolis-Hastings adjustment of the probability distribution can be defined according to the following equation:

[0035]

[0036] Using a Metropolis-Hastings adjustment of the transition probability distribution may ensure that the stable distribution remains equal to p(s) and may involve some data sharing between the client devices 102-112 in the environment 100, which may be performed when initiating the federated learning process.

[0037] In some aspects, in order to determine when to terminate the federated learning process to update the machine learning model, the client device may evaluate the inference performance of the machine learning model using a validation data set that includes test data and ground-truth inferences associated with the test data. If the inference performance of the machine learning model for the validation data set meets or exceeds a target set of metrics, such as the percentage of accurate or inaccurate inferences generated for the validation data set, the client device may determine that the model is sufficiently trained and computing resources do not need to be spent on further training (or updating) of the machine learning model. Therefore, the client device may propagate the parameters of the machine learning model through the communication graph so that each client device 102-112 in the environment 100 has the latest version of the machine learning model. Each of the receiving client devices may verify the performance of the machine learning model based on the propagated parameters, and based on determining that the performance of the machine learning model using the propagated parameters meets or exceeds a threshold performance level, the receiving client device may determine that further training is not required.

[0038] To further assist in training the machine learning model and generating a resulting model with sufficient inference performance, various adaptive optimization techniques may be used in the decentralized federated learning scheme discussed herein. In general, the adaptive optimization technique defines two moments for each parameter in the machine learning model that correct for randomness in the gradient direction that may be experienced during the training of the machine learning model. The first moment may be an exponential moving average of the gradient, and the second moment may be an exponential moving average of the squared gradient. These moments may be used to adaptively increase or decrease the effective learning rate in response to changes in the gradient at each iteration of training the machine learning model in the environment 100 (e.g., when the model is trained by different client devices 102-112 in the environment 100).

[0039] Because the training of the machine learning model in environment 100 is not controlled by a central server or other central coordinator, when a first client device commands a second client device to train (or update) a machine learning model based on local data at the second client device, information about the moment can be shared with the parameters of the machine learning model. However, because this information can take up a lot of space (e.g., can have the same bit size as the underlying data associated with each parameter), the communication overhead involved in transferring information about the machine learning model from one client device to another client device is increased. Therefore, various techniques can be used to compress the moment data. For example, the first moment can be set to 0, which can reduce the amount of data sent between client devices and avoid introducing deviations in the update direction that may occur due to compressing the first moment. The second moment can be compressed by quantizing the second moment to one of a plurality of quantization values. Various techniques can be used to compress the second moment, including scalar quantization, factorization, relative entropy decoding, or other techniques that can appropriately quantize the second moment to one of a plurality of quantization values.

[0040] Figure 2 Depicted is example pseudo-code 200 for training a machine learning model using a decentralized federated learning scheme and a random walk traversal of a communication graph representing connections between client devices in a computing environment, in accordance with aspects of the present disclosure.

[0041] As illustrated in pseudo code 200, at block 210, client device i receives a model from a neighboring client device in the computing environment. It represents the state of the model at time t (e.g., after having been trained by neighboring client devices, as the initial state of the device, etc.).

[0042] At block 220, client device i The accuracy of the validation dataset at client device i is evaluated to determine whether to continue training the model. If the accuracy of the validation data set at client device i meets or exceeds the threshold accuracy level, the client device may determine that further training is not warranted and proceed to block 240. However, if If the accuracy of the validation data set at client device i does not meet the threshold accuracy level, client device i may proceed to block 230 .

[0043] At block 230, the client device i performs a Updates to the generated model to generate updated models The updated model can be generated according to the following expression:

[0044]

[0045] Where η represents the learning rate or the The rate at which updates are made, and Represents a model that can be optimized to train an update Then, we can get the loss function of the neighboring client devices of client device i from The neighboring client device j is selected as described above, and the updated model may be distributed to selected neighboring client devices j for evaluation and potentially for updating based on local data at the selected neighboring client devices j.

[0046] However, if at block 220, the client device i determines that the model If the inference accuracy of meets or exceeds the threshold accuracy level, then at block 240, the client device i Distribute to neighboring client devices

[0047] In some aspects, client devices may use various techniques to adaptively increase or decrease hyperparameters, such as learning rates, in response to changes in gradients across iterations of training machine learning at different client devices participating in a distributed federated learning scheme. For example, neighboring client devices may use an exponential moving average of gradients (also referred to as first moment data) and an exponential moving average of squared gradients (also referred to as second moment data) to control updates to a machine learning model. To communicate information while reducing the overhead involved in communicating updates between client devices participating in a federated learning scheme, the first moment data may be discarded and the second moment data may be updated with the updated model. are communicated together to adjacent client devices.

[0048] At the neighboring client devices, given the above adjacency matrix A and the transfer matrix P considering the connection characteristics of the client devices reflected in the adjacency matrix A, the marginal distribution of the random walk on the client devices participating in the federated learning scheme can be expressed by the following equation:

[0049] π t =P t π 0

[0050] where π 0 represents the initial distribution, and π t represents the stable distribution of random walk. Assume that the matrix Q exists such that QPQ -1 is a product with eigenvalue 1 = λ 1 >λ 2 ≥…≥λ N >-1, the Euclidean space of the model mapping data can be expressed according to the following expression:

[0051]

[0052] where |f| ∞ ≤G and λ=max(λ 2 ,|λ N |). In addition, it can be assumed that each total loss function f s is L-smooth (e.g., is a continuously differentiable function), the gradient is upper bounded by a constant G, and for any dimension, the variance of the gradient at client device s for all client devices S is upper bounded by And the upper limit of the global gradient variance is and

[0053] Therefore, the convergence using the first moment data and the second moment data satisfies the following expression:

[0054]

[0055] where w is w 1 With w T The iterations are randomly selected between , and T represents the number of iterations to update the machine learning model.

[0056] Therefore, the asymptotic bounds achieved by training machine learning models using the decentralized federated learning scheme and random walk traversals discussed in this paper can be similar to the asymptotic bounds achieved using adaptive optimization in a centralized setting, where an error term from the decentralization is added, which can decrease linearly as the number of iterations T increases.

[0057] In some aspects, in order to further reduce the communication overhead when training a machine learning model using a decentralized federated learning scheme and a random walk traversal of a communication graph, the second moment data may be quantized (e.g., in the logarithmic domain). Quantization may be performed so that a single bit is used to degrade whether the second moment is a zero value or a non-zero value, and the remaining b-1 bits (for the b-bit quantization of the second moment data) are used for the quantized value of the second moment data, so that the minimum and maximum values ​​of the logarithm of the second moment data are accurately represented, thereby making the data not clipped. In this case, the updates to the machine learning model generated by the participating client devices may satisfy the following expression:

[0058]

[0059] where w is w 1 With w T The iterations are randomly selected between , and T represents the number of iterations to update the machine learning model.

[0060] For a number of updates K that a client device performs before distributing the model to neighboring client devices, it can be assumed that for any given point There is a constant ζ that limits the upper limit of the distance between the gradient on any given client s and the global gradient (e.g., the L2 distance between vectors) according to the following expression 2 :

[0061]

[0062] Assume that the learning rate parameter η and error ∈ of the machine learning model are chosen such that:

[0063]

[0064] The update of the machine learning model based on the quantized second moment data can satisfy the following expression:

[0065]

[0066] Therefore, by using the second moment data and quantizing such data when training the machine learning model, for a number of updates K performed by the client device, it can be shown that the total variance σ 2 In exchange for local gradient variance and bias ζ to train the machine learning model. Therefore, when the data used by each client device is an independent and identically distributed random variable, the bias term is a small term, and multiple updates K are performed at each client device before distributing the machine learning model to neighboring client devices, aspects of the present disclosure may allow the use of distributed federated learning techniques to train accurate models.

[0067] Example Parallelized Hyperparameter Search

[0068] Because a machine learning model can be trained by many devices in a distributed environment without relying on the training process to be completed by other devices in the distributed environment, and because no single device can be a bottleneck for training the machine learning model, aspects of the present disclosure can allow for parallel identification of hyperparameters of the machine learning model.

[0069] Figure 3 An example 300 of a parallelized hyperparameter search and update process for participating client devices in a federated learning scheme according to aspects of the present disclosure is illustrated. As illustrated, a plurality of client devices 320A-320C (which may represent Figure 1 The client devices 102-112 shown in FIG. 1 and FIG. 2 train the machine learning model using different hyperparameters and report information about the hyperparameters and the performance of the machine learning model to the central server 310. The central server 310 generally does not need to play a role in controlling or coordinating the training of the machine learning model by the client devices 320A-320C, so that further training of the machine learning model (as described above with respect to FIG. 1 ) is not necessary. Figure 1 discussed) can be performed in a decentralized manner and thus protect the participating client devices 320A-320C (as well as Figure 3 The privacy and security of training data at other client devices that also participate in the federated learning scheme (not illustrated in FIG).

[0070] When performing the parallelized hyperparameter search and update process, other client devices (not shown) can communicate with each other via the communication graph with different optimization parameters and machine learning model hyperparameters. It should be noted that when reaching these client devices 320A-320C, the communication diagram There may be no dependencies between the different paths through which these client devices are reached, and the client devices in the environment do not need to be synchronized. Therefore, poorly performing (or slow) client devices can be bypassed. In addition, the performance of many versions of the machine learning model can be evaluated within the environment 100 with little or no additional computational cost.

[0071] In general, each client device 320A-320C can train (or update) a machine learning model using a set of optimization parameters and hyperparameters. These parameters or hyperparameters may include, for example, model architecture information (e.g., the number of layers in a neural network, the type of layers in a neural network, the number of hidden units in a neural network, etc.), learning rate, training batch size, etc. After training the machine learning model, each client device 320A-320C can verify the performance of its corresponding updated machine learning model based on a validation data set. As discussed, the performance of the machine learning model can be defined based on the inference accuracy of the machine learning model (e.g., the percentage of correct inferences generated for the validation data set, the percentage of incorrect inferences generated for the validation data set, etc.). Hyperparameters of the machine learning model and / or other parameters of the machine learning model (e.g., weights), and performance information associated with these hyperparameters and / or other parameters.

[0072] In general, the central server 310 can provide a virtual blackboard that allows client devices 320A-320C (and other client devices participating in a decentralized federated learning scheme or otherwise using a model generated using a decentralized federated learning scheme as discussed herein) to read and write performance data and model hyperparameters and / or other parameter data. If a client device determines that performance data written to the central server 310 by another client device is associated with better inference performance, the client device can retrieve the hyperparameters and / or other machine learning model parameters from the central server 310 and apply these parameters to a local version of the trained machine learning model.

[0073] For example, Figure 3 As illustrated, client devices 320A, 320B, and 320C publish or otherwise write information about hyperparameters and / or other machine learning model parameters to central server 310 for inspection by other devices. In some aspects, client devices 320A, 320B, and 320C may write this information after verifying that the performance of the machine learning model trained at each of these client devices meets or exceeds a threshold performance level (e.g., inference accuracy, etc.). Client devices 320A and 320B may examine the published performance data and determine that the performance of the machine learning model with the hyperparameters and / or other parameters generated by client device 320C exceeds the performance of the machine learning model trained by client devices 320A and 320B. Therefore, client devices 320A and 320B retrieve the parameters generated by client device 320C and implement these hyperparameters and / or other parameters for subsequent operations (e.g., training and / or inference).

[0074] Example method for training a machine learning model using decentralized federated learning techniques

[0075] Figure 4 Example operations 400 that may be performed for training a machine learning model in accordance with certain aspects of the present disclosure are illustrated. Operations 400 may be performed, for example, by a device (such as a mobile phone, tablet computer, laptop computer, or other computing device that may participate in a distributed federated learning scheme) that participates in a distributed federated learning scheme to train a machine learning model using local data.

[0076] As illustrated, operations 400 begin at box 410, where optimization parameters and parameters of a machine learning model are received at a device along with an optimization state value to be updated based on a local data set.

[0077] In some aspects, the optimization state value includes one or more state variables that control the amount by which the peer device adjusts the updated parameters of the machine learning model. These state variables may include, for example, an exponential moving average associated with each parameter in the machine learning model and an exponential moving average of the square of the gradient associated with each parameter in the machine learning model. In some aspects, one or more state variables may include a quantized value of at least one of the state variables. For example, as discussed, the exponential moving average of the gradient may be zeroed, and the exponential moving average of the square of the gradient may be quantized to one of a plurality of categories having a bit size that is smaller than the bit size of the base parameter for which the exponential moving average of the square of the gradient is calculated.

[0078] At box 420, operation 400 continues by updating the parameters of the machine learning model and the optimization state values ​​of the optimization parameters based on the local data set.

[0079] At block 430 , the operations 400 continue by selecting a peer device to refine the machine learning model based on a graph data object including connections between the device and a plurality of peer devices including the peer device.

[0080] In some aspects, the graph data object may be a connection graph including network connections between the device and a plurality of peer devices. In some aspects, the graph data object may include social connections between a user of the device and users associated with a plurality of peer devices. In some aspects, the graph data object may be generated based on routing times between the device and the plurality of peer devices, physical proximity between the device and the plurality of peer devices, or other information identifying relationships between the device and the plurality of peer devices.

[0081] In some aspects, selecting a peer device may include selecting a device from a plurality of peer devices based on a random selection of devices in the graph data object that have a connection to the device. In doing so, a random walk may be performed through the graph data object. As described above, in some aspects, selecting a peer device from a plurality of peer devices may be performed to select a device based on a stable distribution. To asymptotically optimize the global loss function.

[0082] In some aspects, selecting a peer device to refine the machine learning model can be performed based on validating the performance of the updated machine learning model based on the validation data set. If the performance of the updated machine learning model meets or exceeds a threshold performance metric, the device can determine that the model can be further trained by the peer device and can select the peer device for use in training the model. Otherwise, the device can perform additional training cycles on the machine learning model (e.g., using a different local data set) to further refine the machine learning model before selecting a peer device for further training the machine learning model.

[0083] At box 440, the operation 400 continues by transmitting the updated parameters and updated optimization state value of the machine learning model to the selected peer device for refinement by the selected peer device.

[0084] In some aspects, the device may verify the performance of the updated machine learning model based on a validation data set. Information associated with the performance of the updated machine learning model and parameters of the machine learning model may be published to a central server. In some aspects, the device may identify a second parameter different from the published parameter based on the performance information published on the central server, and generate a machine learning model with improved performance characteristics relative to the updated machine learning model. Based on identifying these parameters that generate a machine learning model with improved performance characteristics relative to the updated machine learning model, the device may update the local version of the machine learning model based on the second parameters. For example, the second parameters may include hyperparameters for training the machine learning model and / or other parameters (e.g., weights, etc.) that characterize the machine learning model.

[0085] In some aspects, the device may verify the performance of the updated machine learning model based on the validation data set. If it is determined that the performance of the updated machine learning model meets the threshold performance level, the device may distribute the updated machine learning model to one or more peer devices in the graph data object.

[0086] Example processing system for training machine learning models using decentralized federated learning techniques

[0087] Figure 5 Describes a method for using a decentralized federated learning technique (such as the one described herein, for example, with respect to Figure 4 An example processing system 500 for training a machine learning model as described above.

[0088] The processing system 500 includes a central processing unit (CPU) 502, which may be a multi-core CPU in some examples. Instructions executed at the CPU 502 may be loaded, for example, from a program memory associated with the CPU 502, or may be loaded from a memory partition (e.g., a memory partition of the memory 524).

[0089] The processing system 500 also includes additional processing components customized for specific functions, such as a graphics processing unit (GPU) 504 , a digital signal processor (DSP) 506 , a neural processing unit (NPU) 508 , a multimedia processing unit 510 , and a wireless connectivity component 512 .

[0090] An NPU, such as NPU 508, is typically a dedicated circuit configured to implement control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU is sometimes alternatively referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graphics processing unit.

[0091] NPUs, such as NPU 508, are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system on a chip (SoC), while in other examples, such NPUs may be part of a dedicated neural network accelerator.

[0092] The NPU can be optimized for training or inference, or in some cases configured to balance performance between the two. For NPUs that can perform both training and inference, the two tasks can generally still be performed independently.

[0093] NPUs designed to accelerate training are generally configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing data set (usually labeled or tagged), iterating on the data set, and then adjusting model parameters (such as weights and biases) to improve model performance. Generally speaking, optimization based on error predictions involves propagating back through the layers of the model and determining the gradient to reduce the prediction error.

[0094] NPUs designed to accelerate inference are generally configured to operate on complete models. Thus, such NPUs can be configured to input a new piece of data and quickly process the new piece through an already trained model to generate a model output (e.g., inference).

[0095] In some implementations, the NPU 508 is part of one or more of the CPU 502, GPU 504, and / or DSP 506. These may be located on a user equipment (UE) or another computing device in a wireless communication system.

[0096] In some examples, wireless connectivity component 512 may include, for example, subcomponents for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. Wireless connectivity component 512 is further connected to one or more antennas 514.

[0097] The processing system 500 may also include one or more sensor processing units 516 associated with any manner of sensors, one or more image signal processors (ISPs) 518 associated with any manner of image sensors, and / or a navigation component 520, which may include a satellite-based positioning system component (e.g., GPS or GLONASS) and an inertial positioning system component.

[0098] The processing system 500 may also include one or more input and / or output devices 522, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, speakers, microphones, and the like.

[0099] In some examples, one or more of the processors of processing system 500 may be based on the ARM or RISC-V instruction set.

[0100] The processing system 500 also includes a memory 524, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 524 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 500.

[0101] Specifically, in this example, memory 524 includes a model parameter receiving component 524A, a parameter updating component 524B, a peer device selecting component 524C, a parameter transmitting component 524D, and a machine learning model component 524E. The depicted components, as well as other components not depicted, may be configured to perform various aspects of the methods described herein.

[0102] In general, the processing system 500 and / or its components may be configured to perform the methods described herein, including: Figure 4 Operation 400.

[0103] Notably, in other aspects, elements of the processing system 500 may be omitted, such as where the processing system 500 is a server computer, etc. For example, in other aspects, the multimedia component 510, the wireless connectivity component 512, the sensor processing unit 516, the ISP 518, and / or the navigation component 520 may be omitted. Furthermore, elements of the processing system 500 may be distributed, such as training a model and using the model to generate inferences.

[0104] Sample Clauses

[0105] Specific implementation details of various aspects of the disclosure are described in the following numbered clauses.

[0106] Clause 1: A computer-implemented method, the method comprising: receiving at a device optimization parameters, parameters of a machine learning model, and an optimization state value to be updated based on a local data set; updating the parameters and the optimization state value of the machine learning model based on the local data set; selecting a peer device to refine the machine learning model based on a graph data object, the graph data object comprising connections between the device and a plurality of peer devices including the peer device; and transmitting the updated parameters and the updated optimization state value of the machine learning model to the selected peer device for refinement by the selected peer device.

[0107] Clause 2: A method according to clause 1, wherein the optimization state value includes one or more state variables, and the one or more state variables control the amount by which the selected peer device adjusts the updated parameters of the machine learning model.

[0108] Clause 3: A method according to Clause 2, wherein the one or more state variables include an exponential moving average of the gradient associated with each parameter in the machine learning model and an exponential moving average of the square of the gradient associated with each parameter in the machine learning model.

[0109] Clause 4: The method of clause 2 or 3, wherein the one or more state variables comprise a quantized value of at least one of the state variables.

[0110] Clause 5: The method of any of clauses 1 to 4, wherein selecting the peer device comprises selecting a device from the plurality of peer devices based on a random selection of devices in the graphic data object that have a connection to the device.

[0111] Clause 6: A method according to any one of clauses 1 to 5, the method further comprising: the device verifying the performance of the machine learning model using the updated parameters based on a validation data set, wherein the peer device is selected based on verifying that the performance of the machine learning model using the updated parameters satisfies a threshold performance metric.

[0112] Clause 7: According to the method described in any one of clauses 1 to 6, the method also includes: the device verifying the performance of the machine learning model using the updated parameters based on a verification data set; and publishing information associated with the performance of the machine learning model using the updated parameters and the parameters of the machine learning model to a central server.

[0113] Clause 8: The method according to Clause 7 further includes: identifying a second parameter different from the published parameter based on the performance information published on the central server, and generating a machine learning model having improved performance characteristics relative to the machine learning model using the updated parameter; and updating the machine learning model based on the second parameter.

[0114] Clause 9: The method of clause 8, wherein the second parameter comprises one or more hyperparameters for training the machine learning model.

[0115] Clause 10: According to the method described in any one of clauses 1 to 9, the method also includes: the device verifying the performance of the machine learning model using the updated parameters based on a verification data set; determining that the performance of the machine learning model using the updated parameters meets a threshold performance level; and based on determining that the performance level of the updated machine learning model meets the threshold performance level, distributing the updated parameters of the machine learning model to one or more peer devices in the graph data object.

[0116] Clause 11: A method as described in any of clauses 1 to 10, wherein the graphical data object comprising connections between the device and the plurality of peer devices comprises network connections between the device and the plurality of peer devices.

[0117] Clause 12: A method as described in any of clauses 1 to 11, wherein the graphical data object including connections between the device and the plurality of peer devices includes social connections between a user of the device and users associated with the plurality of peer devices.

[0118] Clause 13: An apparatus comprising: a memory having executable instructions stored thereon; and a processor configured to execute the executable instructions so that the apparatus performs the method according to any one of clauses 1 to 12.

[0119] Clause 14: An apparatus comprising: means for performing the method according to any one of clauses 1 to 12.

[0120] Clause 15: A non-transitory computer readable medium having stored thereon instructions which, when executed by a processor, perform the method of any one of clauses 1 to 12.

[0121] Clause 16: A computer program product embodied on a computer readable storage medium, the computer readable storage medium comprising: code for performing the method according to any one of clauses 1 to 12.

[0122] Additional considerations

[0123] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein do not limit the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, without departing from the scope of the present disclosure, the functions and arrangements of the elements discussed may be changed. Each example may omit, replace, or add various procedures or components as appropriate. For example, the described method may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, the features described for some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods practiced using other structures, functions, or structures and functions that are supplemented or substituted for the various aspects of the present disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of the claims.

[0124] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0125] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items (including single members). As an example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0126] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), ascertaining, etc. In addition, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. In addition, "determining" may include resolving, selecting, choosing, establishing, etc.

[0127] The method disclosed herein includes one or more steps or actions for implementing the method. The steps and / or actions of the method can be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claims. In addition, the various operations of the method described above can be performed by any appropriate component capable of performing the corresponding function. The component may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors. In general, in the case of operations illustrated in the accompanying drawings, those operations may have corresponding corresponding components with similar numbers plus functional components.

[0128] The following claims are not intended to be limited to the various aspects shown herein, but should be granted the full scope consistent with the language of the claims. Within the claims, unless specifically stated otherwise, references to singular elements are not intended to mean "one and only one", but "one or more". Unless otherwise specifically stated, the term "some" refers to one or more. No claim element should be interpreted according to the provisions of 35 U.S.C. § 112 (f), unless the element is explicitly recorded using the phrase "a component for...", or in the case of a method claim, the element is recorded using the phrase "a step for...". All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or will be known later to a person of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, regardless of whether such disclosure is explicitly recorded in the claims, anything disclosed herein is not intended to be disclosed to the public.

Claims

1. A computer-implemented method, the method comprising: receiving, at the device, optimization parameters, parameters of the machine learning model, and an optimization state value to be updated based on a local data set; updating the parameters of the machine learning model and the optimized state values ​​of the optimized parameters based on the local data set; selecting a peer device to refine the machine learning model based on a graph data object including connections between the device and a plurality of peer devices including the peer device; as well as The updated parameters and updated optimization state value of the machine learning model are transmitted to the selected peer device for refinement by the selected peer device.

2. The method of claim 1, wherein the optimization state value comprises one or more state variables, the one or more state variables controlling the amount by which the selected peer device adjusts the updated parameters of the machine learning model.

3. A method according to claim 2, wherein the one or more state variables include an exponential moving average of the gradient associated with each parameter in the machine learning model and an exponential moving average of the square of the gradient associated with each parameter in the machine learning model. The method of claim 2 , wherein the one or more state variables include a quantized value of at least one of the state variables.

5. The method of claim 1 , wherein selecting the peer device comprises: A device is selected from the plurality of peer devices based on a random selection of devices in the graphic data object that have a connection to the device.

6. The method according to claim 1, further comprising: The device verifies the performance of the machine learning model using the updated parameters based on a validation data set, wherein the peer device is selected based on verifying that the performance of the machine learning model using the updated parameters satisfies a threshold performance metric.

7. The method according to claim 1, further comprising: Validating, by the device, the performance of the machine learning model using the updated parameters based on a validation dataset; as well as Publish to a central server (1) information associated with the performance of the machine learning model using the updated parameters and (2) parameters of the machine learning model.

8. The method according to claim 7, further comprising: identifying, based on the performance information published on the central server, second parameters that are different from the published parameters, and generating a machine learning model having improved performance characteristics relative to the machine learning model using the updated parameters; as well as The machine learning model is updated based on the second parameter.

9. The method of claim 8, wherein the second parameter comprises one or more hyperparameters for training the machine learning model.

10. The method according to claim 1, further comprising: Validating, by the device, the performance of the machine learning model using the updated parameters based on a validation dataset; determining that the performance of the machine learning model using the updated parameters satisfies a threshold performance level; as well as Based on determining that a performance level of the machine learning model using the updated parameters satisfies a threshold performance level, the updated parameters of the machine learning model are distributed to one or more peer devices in the graph data object.

11. The method of claim 1, wherein the graphical data object comprising connections between the device and the plurality of peer devices comprises network connections between the device and the plurality of peer devices.

12. The method of claim 1, wherein the graphical data object comprising connections between the device and the plurality of peer devices comprises social connections between a user of the device and users associated with the plurality of peer devices.

13. A system, comprising: A memory, wherein executable instructions are stored in the memory; and at least one processor configured to execute the executable instructions to cause the system to: receiving at the system optimization parameters, parameters of the machine learning model, and an optimization state value to be updated based on a local data set; updating the parameters of the machine learning model and the optimized state values ​​of the optimized parameters based on the local data set; selecting a peer device to refine the machine learning model based on a graph data object including connections between the system and a plurality of peer devices including the peer device; as well as The updated parameters and updated optimization state value of the machine learning model are transmitted to the selected peer device for refinement by the selected peer device.

14. The system of claim 13, wherein the optimized state value comprises one or more state variables that control an amount by which the selected peer device adjusts the updated parameters of the machine learning model.

15. A system according to claim 14, wherein the one or more state variables include an exponential moving average of the gradient associated with each parameter in the machine learning model and an exponential moving average of the square of the gradient associated with each parameter in the machine learning model.

16. The system of claim 14, wherein the one or more state variables include a quantized value of at least one of the state variables.

17. The system of claim 13, wherein to select the peer device, the processor is configured to cause the system to select a peer device from the plurality of peer devices based on a random selection of devices in the graphic data object that have a connection to the system.

18. A system according to claim 13, wherein the processor is further configured to cause the system to verify the performance of the machine learning model using the updated parameters based on a validation data set, wherein the peer device is selected based on verifying that the performance of the machine learning model using the updated parameters satisfies a threshold performance metric.

19. The system of claim 13, wherein the processor is further configured to cause the system to: Validating the performance of the machine learning model using the updated parameters based on a validation dataset; and Publish to a central server (1) information associated with the performance of the machine learning model using the updated parameters and (2) parameters of the machine learning model.

20. The system of claim 19, wherein the processor is further configured to cause the system to: Based on the performance information published on the central server, identifying second parameters that are different from the published parameters, and generating a machine learning model having improved performance characteristics relative to the machine learning model using the updated parameters; and The machine learning model is updated based on the second parameter.

21. The system of claim 20, wherein the second parameter comprises one or more hyperparameters for training the machine learning model.

22. The system of claim 13, wherein the processor is further configured to cause the system to: Validating the performance of the machine learning model using the updated parameters based on a validation dataset; determining that the performance of the machine learning model using the updated parameters satisfies a threshold performance level; and Based on determining that a performance level of the machine learning model using the updated parameters satisfies a threshold performance level, the updated parameters of the machine learning model are distributed to one or more peer devices in the graph data object.

23. The system of claim 13, wherein the graphical data object comprising connections between the system and the plurality of peer devices comprises network connections between the system and the plurality of peer devices.

24. The system of claim 13, wherein the graphical data object comprising connections between the system and the plurality of peer devices comprises social connections between users of the system and users associated with the plurality of peer devices.

25. A system, comprising: means for receiving, at the device, optimization parameters, parameters of the machine learning model, and an optimization state value to be updated based on a local data set; means for updating the parameters of the machine learning model and the optimized state values ​​of the optimized parameters based on the local data set; means for selecting a peer device to refine the machine learning model based on a graph data object including connections between the system and a plurality of peer devices including the peer device; and A component for transmitting the updated parameters and the updated optimization state value of the machine learning model to the selected peer device for refinement by the selected peer device.

26. A computer readable medium having executable instructions stored thereon, the executable instructions, when executed by a processor, performing operations comprising: receiving, at the device, optimization parameters, parameters of the machine learning model, and an optimization state value to be updated based on a local data set; updating the parameters of the machine learning model and the optimized state values ​​of the optimized parameters based on the local data set; selecting a peer device to refine the machine learning model based on a graph data object including connections between the system and a plurality of peer devices including the peer device; as well as The updated parameters and updated optimization state value of the machine learning model are transmitted to the selected peer device for refinement by the selected peer device.

Images