Associative Mixture Models

Abstract

Aspects described herein comprise a method of processing data, the method including receiving a set of global parameters for a plurality of machine learning models; processing data stored locally on a processing device using the plurality of machine learning models according to the set of global parameters to generate machine learning model outputs; receiving, at the processing device, user feedback regarding the machine learning model outputs for the plurality of machine learning models; performing optimization of the plurality of machine learning models based on the machine learning outputs and the user feedback to generate locally updated machine learning model parameters; transmitting the locally updated machine learning model parameters to a remote processing device; and receiving the set of globally updated machine learning model parameters for the plurality of machine learning models.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to Greek Provisional Patent Application No. 20190100556, filed December 13, 2019, the entire contents of which are incorporated herein by reference.

[0002] Aspects of the present disclosure relate to machine learning models, and in particular to federated mixture models. [Background technology]

[0003] In machine learning, a trained model (e.g., an artificial neural network, tree, or other structure) may be created, where the trained model exhibits a generalized fit to a set of training data that is known a priori. Applying the trained model to new data produces inferences, which may be used to gain insights into the new data. Sometimes, applying the model to new data is referred to as "performing inference" on the new data.

[0004] Machine learning models are increasingly being employed in a wide variety of fields, including for use in classification, detection, and recognition tasks. For example, machine learning models are being used to perform complex tasks on electronic devices, such as automatically detecting features (e.g., faces) in images based on sensor data provided by one or more sensors on such devices.

[0005] Traditional machine learning is often performed centrally, such as when training data is collected in a central repository and processed together to train machine learning models. This simplifies some aspects of machine learning. For example, having a unified training dataset allows data to be processed according to the independent and identically distributed (IID) assumption for variables in the training dataset, which means that all training data instances (e.g., observations) taken from the training dataset come from the same generation process that has no memory of previously generated samples. This assumption therefore allows training data to be more easily split into training data subsets and validation data subsets, because both subsets are assumed to have the same distribution. Moreover, this assumption is the basis of standard maximum likelihood optimization objectives.

[0006] Modern electronic devices, especially decentralized portable electronic devices, Internet of Things (IoT) devices, always-on connected (AON) devices, and other "edge" devices, are increasingly capable of performing machine learning tasks. Therefore, it is attractive to utilize these devices as machine learning computational resources. However, in many contexts, it may be impossible or impractical to generate globally applicable machine learning models that use decentralized processing techniques. For example, physical limitations such as processing speed, network speed, battery life, as well as policy limitations such as privacy laws, security requirements, etc. may limit the ability to decentralize the training of machine learning models using a wide variety of computational resources.

[0007] Federated learning aims to solve some of the decentralized processing problems mentioned above by distributing machine learning related processing to devices at the "edge" (such as the portable electronic devices mentioned above). Unfortunately, decentralizing data processing explicitly violates the standard IID assumption that underlies the standard maximum likelihood optimization objective of various machine learning techniques. Thus, federated learning can degrade the performance of current machine learning techniques.

[0008] Therefore, there is a need for improved methods for performing federated learning without compromising the effectiveness of existing machine learning techniques. Summary of the Invention [Means for solving the problem]

[0009] In a first aspect, a method for processing data includes, in a processing device s, generating a set of global parameters w for each machine learning model k of a plurality of machine learning models K. k t and for each machine learning model k of the plurality of machine learning models K, receiving a machine learning model output y s,k At the processing device, the data stored locally on the processing device is processed using each machine learning model k to generate a set of global parameters w k t and, in the processing device, processing the machine learning model output y s,k receiving user feedback on the locally updated machine learning model parameters w at the processing device; s,k t+τ To generate the machine learning output y s,k and the machine learning model output y s,k and optimizing each machine learning model k based on user feedback related to the locally updated machine learning model parameters w s,k t+τ to a remote processing device; and for each machine learning model k of the plurality of machine learning models K, transmitting a set of globally updated machine learning model parameters wk t+τ from the remote processing device, and for each respective machine learning model k, globally updated machine learning model parameters w k t+τ The locally updated machine learning model parameters w s,k t+τ The method includes steps based at least in part on:

[0010] In a second aspect, a method for processing data includes, for each model k of a plurality of models K, at each remote processing device s of a plurality of remote processing devices S, generating an initial set w of model parameters for each machine learning model k. k t from the server to each remote processing device s; and transmitting an updated set of model parameters w for each machine learning model k from each remote processing device s to the server. s,k t+τ and receiving an updated set of global model parameters w k t+τ At the server, an updated set of model parameters w received from each remote processing device s of the plurality of remote processing devices S is generated to generate s,k t+τ and optimizing each machine learning model k based on an updated set of global model parameters w for each machine learning model k of the plurality of models K. k t+τ and transmitting the same to each remote processing device s of the plurality of remote processing devices S from the server.

[0011] Further aspects relate to apparatus configured to perform the methods described herein as well as non-transitory computer-readable media comprising computer-executable instructions that, when executed by a processor of the device, cause the device to perform the methods described herein.

[0012] The following description and the related drawings set forth in detail certain illustrative features of the one or more embodiments.

[0013] The accompanying drawings illustrate several aspects of the one or more embodiments and therefore should not be considered limiting of the scope of the disclosure. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 illustrates an exemplary machine learning model architecture. [Diagram 2] FIG. 1 illustrates an example of a federated mixed algorithm based on the above derived formula. [Diagram 3] FIG. 1 illustrates an example method for processing federated mixed model data on a device. [Figure 4] FIG. 1 illustrates an example method for processing federated mixed model data on a centralized device, such as a server device. [Diagram 5] FIG. 1 illustrates an example electronic device that may be configured to perform the methods described herein. [Figure 6] FIG. 1 illustrates an example multi-processor processing system that may be configured to perform the methods described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] For ease of understanding, the same reference numbers have been used, where possible, to designate identical elements common to the figures. It is contemplated that elements and features of one embodiment may be advantageously incorporated in other embodiments without further recitation.

[0016] Aspects of the present disclosure provide apparatus, methods, processing systems, and computer-readable media for improving federated machine learning performance based on using multiple model instances (or "experts") to perform maximum likelihood optimization and thus mitigate the impact of training data that does not conform to the independent and identically distributed (IID) assumption. Advantageously, the federated mixed model methods described herein can be performed synchronously or asynchronously across each federated device. Thus, such federated mixed model methods are particularly useful in utilizing low-power processing systems for federated learning, such as mobile, IoT, edge, and other processing devices that have processing, power, data connection, and / or memory size limitations.

[0017] A brief background on neural networks, deep neural networks, and deep learning Neural networks are structured as layers of interconnected nodes. In general, a node (or neuron) is where computations take place. For example, a node may combine input data with a set of weights (or coefficients) that amplify or attenuate the input data. Thus, the amplification or attenuation of an input signal may be viewed as an assignment of relative importance to various inputs with respect to the task the network is trying to learn. In general, each input-weight product is added (or accumulated), and then this sum is passed through the node's activation function to determine whether and how far the signal should proceed further through the network.

[0018] In its most basic implementation, a neural network may have an input layer, a hidden layer, and an output layer. "Deep" neural networks generally have two or more hidden layers.

[0019] Deep learning is a method of training deep neural networks. In general, deep learning is sometimes called a "universal approximator" because it maps inputs to the network to outputs from the network, and thus allows you to approximate an unknown function f(x)=y between input x and output y. In other words, deep learning finds the correct f to transform x to y.

[0020] More specifically, deep learning trains each layer of nodes based on a different set of features, i.e., the output from the previous layer. Thus, with each successive layer of a deep neural network, each feature may become more complex. Deep learning is therefore highly effective in performing complex tasks such as object recognition by extracting progressively higher level features from the input data and augmenting useful feature representations of the input data through multiple layers and levels of abstraction.

[0021] For example, when presented with visual data, the first layer of the deep neural network may be adapted to recognize relatively simple features, such as edges, in the input data. In another example, when presented with audio data, the first layer of the deep neural network may be adapted to recognize spectral power at specific frequencies in the input data. The second layer of the deep neural network may then be adapted to recognize combinations of features, such as simple shapes in the visual data or combinations of sounds for the audio data, based on the output of the first layer. Then, higher layers may be adapted to recognize complex shapes in the visual data or words in the audio data. Still higher layers may be adapted to recognize general visual objects or spoken words. Thus, deep learning architectures may work particularly well when applied to problems that have a natural hierarchical structure.

[0022] Machine learning model maximum likelihood optimization Machine learning models come in many forms, such as neural networks (e.g., deep neural networks and convolutional neural networks), regression (e.g., logistic or linear), decision trees (including random forests of trees), support vector machines, cascaded classifiers, etc. Neural networks are described throughout this specification as one exemplary application of the methods described herein, although these same methods may be applied to other types of machine learning models as well.

[0023] In machine learning, training a model may be viewed as an optimization process by taking a set of observations and performing maximum likelihood estimation such that a target probability is maximized. In statistics, maximum likelihood estimation is a method of estimating the parameters of a probability distribution by maximizing a likelihood function such that the probability of the observed data under an assumed statistical model is the highest. Thus, in the context of machine learning models, the following formula may be derived:

number

[0024] In the above formula,

number

number

number

[0025] Mixed models A mixture model is a probabilistic model for an observed data set to represent the existence of subpopulations within the entire population of data without the need to identify the subpopulations to which each individual observation belongs. Thus, a mixture model corresponds to a mixture distribution that represents the probability distribution of observations within the entire population of observations. Mixture models may be used to make statistical inferences about properties of subpopulations given only observations about the pooled populations, without subpopulation identification information.

[0026] Some methods for implementing mixture models include determining that the underlying subpopulation specific information corresponds to the individual observations (or weights for such subpopulations), in which case they can be considered as a kind of unsupervised learning or clustering procedure. For example, a Gaussian mixture is a function that includes several Gaussian distributions, each identified by k∈{1,...,K}, where K is the number of clusters in the dataset that share some common property, such as a statistical distribution, a centroid of the data points, etc. Each individual Gaussian distribution k in the mixture may include the following parameters: mean μ, which defines the center of the Gaussian distribution; covariance Σ, which defines the width of the Gaussian distribution (equivalent to the dimension of the ellipsoid in a multivariate scenario); and mixture probability π, which defines the size of the Gaussian function.

[0027] A set of parameters for each Gaussian distribution may be defined as θ={π, μ, Σ}. A maximization algorithm may then be applied to determine the optimal value of θ, such as the expectation maximization (EM) algorithm. For example, the optimal value may be calculated according to the following formula:

number

[0028] Notably, this is one exemplary formulation and other formulations are possible.

[0029] Federated Machine Learning Traditional machine learning utilizes a centralized data collection and processing architecture. In contrast, federated machine learning distributes the machine learning process to multiple devices, each with its own federated data set that may not be shareable as a centralized data set. Thus, federated machine learning enables various "edge" processing devices, such as smartphones, to collaboratively learn a shared machine learning model using training data on the individual edge processing devices, but without sharing individual device data. Instead, the edge processing devices only share model parameters, such as weights and biases, that are obtained from the processing device's own local model optimization procedure. Thus, there is no need to transfer data over a network to a centralized repository, reducing data transmission costs while improving data security and confidentiality.

[0030] Federated machine learning has become extremely popular, especially as the number of edge processing devices with available computational resources grows rapidly and the processing capabilities of such edge processing devices are enhanced. Although edge processing devices may not be as powerful as dedicated machine learning processing systems (e.g., mainframes, servers, supercomputers, etc.) individually, their sheer numbers can compensate for their relatively low processing power. Furthermore, edge devices such as smartphones are beginning to incorporate specialized processing chips, such as neural processors, which are dedicated to performing machine learning processing. Thus, in some instances, edge devices may be more powerful than standard computing devices due to their specialized machine learning hardware.

[0031] As described herein, model blending may be used to combine multiple models (or sub-models or experts) to generate a resulting model.

[0032] Example of a Federated Learning Architecture FIG. 1 illustrates an exemplary federated learning architecture 100 .

[0033] In this example, the mobile devices 102A-102C are examples of edge processing devices, each having a respective local data store 104A-104C and a respective local machine learning model instance 106A-106C. For example, the mobile device 102A includes an initial machine learning model instance 106A that the model device 102A may receive, for example, from a global machine learning model coordinator 108, which may be a software provider in some examples. Each of the mobile devices 102A-102C may use the respective machine learning model instance (106A-106C) for some useful task, such as processing the local data 104A-104C, and may further perform local training and optimization of its respective machine learning model instance (106A-106C).

[0034] For example, the mobile device 102A may use the machine learning model 106A of the mobile device 102A to perform face recognition on photos stored as data 104A on the mobile device 102A. Because such photos may be considered personal, the mobile device 102A may not want to or may be prevented from sharing its photo data with the global model coordinator 108. However, the mobile device 102A may want to or be allowed to share its local model updates, such as updates to model parameters (e.g., weights and biases), with the global model coordinator 108. Similarly, the mobile devices 102B and 102C may similarly use their local machine learning model instances 106B and 106C, respectively, and share local model updates with the global model coordinator 108 without sharing the underlying data (104B and 104C) used to generate the local model updates of the mobile devices 102B and 102C.

[0035] The global model coordinator 108 may use all of the local model updates to determine a global (or consensus) model update, which may then be distributed to the mobile devices 102A-102C. In this manner, federated machine learning may be performed using the mobile devices 102A-102C without centralizing training data and processing.

[0036] Thus, the federated learning architecture 100 enables decentralized deployment and training of machine learning models, which advantageously reduces latency, network utilization, and power consumption, while maintaining data confidentiality and security, and potentially increasing utilization of idle computational resources. Additionally, the federated learning architecture 100 advantageously enables local models (e.g., 106A-106C) to evolve differently on different devices, while simultaneously training a global model based on the evolution of the local models.

[0037] In particular, the local data stored on the mobile devices 102A-102C, respectively, and used by the machine learning models 106A-106C may be referred to as individual data shards (e.g., data 104A-104C) and / or federated data. Because these data shards are generated by different users on different devices and are not mixed, they cannot be assumed to be independent and identically distributed (IID) with respect to one another. This applies more generally to any kind of device-specific data that is not combined to train a machine learning model. Only by combining the individual data sets 104A-104C of the mobile devices 102A-102C, respectively, can a global data set be generated where the IID assumption is valid.

[0038] Machine Learning with Federated Mixture Models To overcome the non-IID property of federated data used for federated machine learning, such as data 104A-104C described with respect to FIG. 1, maximum likelihood optimization may be extended to a mixture of K different predictive models or "experts." Each expert is expected to model a region in the joint data space (e.g., a data space combining all federated data spaces). To do so, it may be assumed that observed data (e.g., data generated by mobile devices 102A-102C of FIG. 1) is created from a mixture of K individual predictive models. Thus, for example, model 106C on mobile device 102A may be considered as a single model comprising multiple K mixture model components (e.g., experts) in the context of federated mixture model learning. Advantageously, the federated mixture model serves as a single model for providing inputs to and receiving outputs from an application that uses the model.

[0039] In one example, the K experts may refer to K different neural network models. In some cases, the neural networks may have the same architecture, while in other cases the neural networks may be different. Let Z be the sum of all z s,i where every data point (y s,i ,x s,i ) has z. In that case, z s,i is the time for a particular data point (y s,i ,x s,i ) is selected from among the K experts (in this example, neural networks, for example).

[0040] Given k neural networks, which individual neural network k is the "best" at describing a data point, or how well each individual neural network k models a given data point (e.g., z s,iDifferent questions can be asked of the model, such as: (where the posterior distribution can be calculated at ) In the methods described herein, the goal is not necessarily to determine which expert (e.g., neural network) of a set of K experts is the "best." Rather, the goal is to train K experts (e.g., neural networks) such that each expert specializes in a different portion of the global dataset.

[0041] In a federated training context, we have data D={(x1, y1, ..., (x N ,y N ) may be split across S different shards (or sets), such that each shard s has a s data points. In addition, we have the data across all S shards (say, D=D1..U...UD S ) is taken from K clusters, and we can assume that the cluster parameter w is shared across all shards in each individual cluster.

[0042] Then the total probability of the model is:

number

[0043] The data to be aggregated may be assumed to be located at a certain position in order to calculate the correct gradient for the model. Thus, the data log-likelihood is maximized by calculating the gradient with respect to w according to

number

[0044] In a federated learning scenario, a global server (e.g., the global model coordinator 108 in FIG. 1) sends a copy of the current parameters w to each local worker (e.g., the mobile devices 102A-102C in FIG. 1). Each worker s sends a copy of the current parameters w to each of its N s Each local worker is tasked with computing (within the outer brackets in equation (5)) a fraction of the total gradients corresponding to that data point. Rather than just performing one gradient update per local worker, the local worker performs several gradient updates on its local copy of the parameters, thereby allowing local progress without relying on frequent, slow, and potentially costly data communication.

[0045] In some cases, averaging the updates by local workers based on iterative determination of the gradient by each local worker according to equation (5) is not optimal. This is because it is advantageous to use an adaptive learning rate optimization algorithm such as Adam (designed for training deep neural networks) to accelerate the learning progress on each local shard. Since each local worker maintains an individual Adam momentum, simply averaging the resulting updates does not properly take into account the influence of each shard on a particular expert k (in set K) relative to the other shards.

[0046] The technical solution to this technical model optimization problem is to further develop equation (5). For notational convenience, we use one mixture component w k We may only be interested in the slope for a "soft" count N according to the following formula: sk may be defined.

number

number

[0047] In equation (11), the local worker computes and applies the gradient inside the outer brackets for τ steps. s,k t After τ local updates, w s,k t+τ Then, each local worker receives the updated set of parameters w s,k t+τ to the global server, which then interprets these updated parameters by calculating an "effective gradient" as a change to the current global server parameters. For example,

number

[0048] FIG. 2 shows an example of a federated mixed algorithm based on the above derived formula.

[0049] It should be noted that the algorithm of Figure 2 is an example of a distributed synchronous training algorithm, and variations of this algorithm are possible, for example, the algorithm may be modified for an asynchronous training context.

[0050] Generating more informative priors Data points (y s,i ,x s,i ) from which expert k is chosen. s,i ), where the subscripts s and i respectively list the shards and the data points within the shards as described with respect to Equation (1). Intuitively, the expert k that is best suited to perform the classification (or regression) task for a particular machine learning model should be selected from all K experts. In one embodiment, the decision on how much weight should be given to the prediction of expert k is determined by, for example, the weighting of the prediction of the input x, rather than assigning equal probabilities to each expert k in the set K. s,i This can be done by looking at the

[0051] To determine p(z=k|x) based on the data point x, the mapping needs to be parameterized and learned. In one embodiment, this may be accomplished, for example, by interpreting p(z=k|x) as the responsibility of an (unsupervised) clustering problem according to

number

[0052] Therefore, each cluster has a φ k The parameter φ is a one-to-one correspondence between cluster k and expert k, and k′ represents the index for addition. k is used as part of the same algorithm formulation. k In Algorithm 1, the weights are jointly optimized using k Similarly to what was explained for parameter φ k is trained by performing local updates using local data and periodically sent to (eg, synchronized with) a global server (eg, global model coordinator 108 in FIG. 1).

[0053] Exemplary Method for Processing Federated Mixture Model Data on an Edge Device FIG. 3 illustrates an example method 300 for processing federated mixture model data on an edge device, such as, for example, the mobile devices 102A-102C of FIG.

[0054] The method 300 begins at step 302 by determining a set of global parameters w for each machine learning model k of a plurality of machine learning models K at an edge processing device s. k t Receive.

[0055] The method 300 then proceeds to step 304, where for each respective machine learning model k of the plurality of machine learning models K, a machine learning model output y s,kTo generate a global parameter set w, the edge processing device applies each machine learning model k to the data stored locally on the edge processing device. k t Process according to the above.

[0056] The method 300 then proceeds to step 306, where for each machine learning model k of the plurality of machine learning models K, at the edge processing device, s,k Receive user feedback regarding.

[0057] The method 300 then proceeds to step 308, where for each respective machine learning model k of the plurality of machine learning models K, a locally updated machine learning model parameter w s,k t+τ To generate the machine learning output y s,k and the machine learning model output y s,k , and optimize each machine learning model k based on user feedback related to y for model k. Note that in some embodiments, s,k Plus all other models * Optimize for all other model outputs y s,k * Depends on.

[0058] The method 300 then proceeds to step 310, where for each respective machine learning model k of the plurality of machine learning models K, a locally updated machine learning model parameter w s,k t+τ to the remote processing device.

[0059] The method 300 then proceeds to step 312, where for each machine learning model k of the plurality of machine learning models K, a set of globally updated machine learning model parameters w k t+τ from the remote processing device.

[0060] In some embodiments of the method 300, the globally updated machine learning model parameters w for each respective machine learning model k are k t+τ is the locally updated machine learning model parameter w s,k t+τ Based at least in part on.

[0061] Some embodiments of the method 300 include, at the edge processing device, locally updating the machine learning model parameters w s,k t+τ The method further comprises the step of performing a number of optimizations τ before transmitting τ to the remote processing device.

[0062] In some embodiments of the method 300, the globally updated machine learning model parameters w for each respective machine learning model k of the plurality of machine learning models K are k t+τ is based at least in part on locally updated machine learning model parameters of the second edge processing device.

[0063] In some embodiments of the method 300, the user feedback includes an indication of the correctness of the machine learning model output.

[0064] In some embodiments of the method 300, the data stored locally on the edge processing device is one of image data, audio data, or video data.

[0065] In some embodiments of the method 300, the edge processing device is one of a smartphone or an Internet of Things device.

[0066] Exemplary Method for Processing Federated Mixture Model Data on a Server Device FIG. 4 illustrates an example method 400 for processing federated mixed model data on a centralized device, such as a server device (eg, the global model coordinator 108 of FIG. 1).

[0067] The method 400 begins at step 402 by determining an initial set of model parameters w for each machine learning model k. k t is sent from the server to each remote processing device s.

[0068] The method 400 then proceeds to step 404, where an updated set of model parameters w for each machine learning model k is transmitted from each remote processing device s to the server. s,k t+τ Receive.

[0069] The method 400 then proceeds to step 406, where an updated set of global model parameters w k t+τ At the server, an updated set of model parameters w received from each remote processing device s of the plurality of remote processing devices S is generated to generate s,k t+τ We perform optimization of each machine learning model k based on

[0070] Note that in some embodiments, steps 402 to 406 may be performed iteratively for each respective model k of the multiple models K and for each respective remote processing device s of the multiple remote processing devices S.

[0071] The method 400 then proceeds to step 408, where an updated set of global model parameters w for each machine learning model k of the plurality of models K is calculated. k t+τ is transmitted from the server to each remote processing device s of a plurality of remote processing devices S.

[0072] In some embodiments of method 400, performing, at the server, the optimization of each machine learning model k, comprises:

number

[0073] Some embodiments of the method 400 may include, for each respective model k of the plurality of models K, determining a weighting parameter φ for each respective model k. k The corresponding density estimator p(x|φ k The weighting parameter φ may be used to combine k models (or sub-models) into a single model output based on the model inputs. In this manner, multiple models (e.g., K models) may be combined using the weighting parameter φ k It is possible to train and "mix" via

[0074] Some embodiments of the method 400 include:

number

[0075] In some embodiments of the method 400, the remote processing device is a smartphone.

[0076] In some embodiments of the method 400, the remote processing device is an Internet of Things device.

[0077] In some embodiments of method 400, each model k of the plurality of models K is a neural network model. In some embodiments of method 400, each model k of the plurality of models K comprises the same network structure. In some embodiments of method 400, one or more of the plurality of models K comprises a different network structure than other models in the plurality of models K.

[0078] Exemplary Processing System 5 illustrates an example electronic device 500. The electronic device 500 may be configured to perform the methods described herein, including the methods with respect to FIGS.

[0079] The electronic device 500 includes a central processing unit (CPU) 502, which in some embodiments may be a multi-core CPU. Instructions executed in the CPU 502 may be loaded, for example, from a program memory associated with the CPU 502 or from a memory block 524.

[0080] The electronic device 500 also includes additional processing blocks tailored to specific functions, such as a graphics processing unit (GPU) 504, a digital signal processor (DSP) 506, a neural processing unit (NPU) 508, a multimedia processing block 510, and a wireless connectivity block 512.

[0081] An NPU, such as 508, is generally a specialized circuit configured to implement all the necessary control and computational logic to execute machine learning algorithms, such as algorithms for processing artificial neural networks (ANN), deep neural networks (DNN), random forests (RF), etc. An NPU may alternatively be referred to as a tensor processing unit (TPU), neural network processor (NNP), intelligence processing unit (IPU), vision processing unit (VPU), or graph processing unit.

[0082] An NPU, such as 508, may be configured to accelerate the execution of common machine learning tasks, such as image classification, machine transformation, object detection, and various other predictive models. In some embodiments, multiple NPUs may be instantiated on a single chip, such as a system-on-chip (SoC), while in other embodiments, they may be part of a dedicated neural network accelerator.

[0083] An NPU may be optimized for training or inference, or in some cases may be configured to balance training and inference performance. In an NPU that can perform both training and inference, the two tasks may nevertheless generally be performed independently.

[0084] NPUs designed to accelerate training may generally be configured to accelerate the optimization of new models, where optimization is a computationally intensive operation that involves inputting an existing dataset (often labeled or tagged), iterating over the dataset, and then adjusting model parameters such as weights and biases to improve model performance. In general, optimization based on mispredictions involves backpropagating through each layer of the model and determining gradients to reduce prediction errors.

[0085] NPUs designed to accelerate inference are generally configured to operate on a complete model. Thus, such NPUs may be configured to input new data and rapidly process the data through a model that has already been trained to generate model outputs (e.g., inference information).

[0086] In one implementation, the NPU 508 is part of one or more of the CPU 502, the GPU 504, and / or the DSP 506.

[0087] In some embodiments, the wireless connectivity block 512 may include components for, for example, third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and wireless data transmission standards. The wireless connectivity processing block 512 is further connected to one or more antennas 514.

[0088] The electronic device 500 may include one or more sensor processors 516 associated with any type of sensor, one or more image signal processors (ISPs) 518 associated with any type of image sensor, and / or a navigation processor 520, which may include satellite-based positioning system components (e.g., GPS or GLONASS) as well as inertial positioning system components.

[0089] The electronic device 500 may include one or more input and / or output devices 522, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, a speaker, a microphone, etc.

[0090] In some embodiments, one or more of the processors of electronic device 500 may be a processor based on an ARM or RISC-V instruction set.

[0091] The electronic device 500 also includes a memory 524, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 524 includes computer-executable components, which may be executed by one or more of the aforementioned processors of the electronic device 500. In particular, in this embodiment, the memory 524 includes a sending component 524A, a receiving component 524B, a processing component 524C, a determining component 524D, an output component 524E, a training component 524F, an inference component 524G, and an optimization component 524H. The illustrated components and other components not illustrated may be configured to perform various aspects of the methods described herein.

[0092] In general, the electronic device 500 and / or its components may be configured to perform the methods described herein.

[0093] Notably, aspects of electronic device 500 may be omitted in other embodiments, such as when electronic device 500 is a server computer, etc. For example, multimedia component 510, wireless connectivity 512, sensors 516, ISP 518, and / or navigation component 520 may be omitted in other embodiments. Additionally, aspects of electronic device 500 may be distributed, such as in a cloud-based processing environment.

[0094] 6 illustrates an exemplary multi-processor processing system 600 that may be implemented using embodiments described herein. For example, the multi-processing system 600 may represent various processors of the electronic device 500 of FIG.

[0095] In this example, system 600 includes processors 601, 603, and 605, although in other examples, any number of individual processors may be used. Additionally, processors 601, 603, and 605, while similarly depicted, may represent various different types of processors in electronic devices, such as CPUs, GPUs, DSPs, NPUs, etc., as described herein.

[0096] Each of processors 601, 603, and 605 includes an instruction scheduler, various hardware subcomponents (e.g., Hardware X, Hardware Y, and Hardware Z), and local memory. In some embodiments, the local memory may be tightly coupled memory (TCM). Note that while the components of each of processors 601, 603, and 605 are shown as the same components in this example, in other examples, some or each of processors 601, 603, and 605 may have different hardware configurations, different hardware elements, and the like.

[0097] Each of the processors 601, 603, and 605 is also in data communication with a global memory, such as a DDR memory, or other type of volatile working memory. For example, the global memory 607 may represent the memory 524 of FIG.

[0098] In some implementations, in a multiprocessor processing system such as 600, one of the processors may act as a master processor. For example, in this example, processor 601 may be the master processor. The master processor may include a compiler that, when executed, can determine how a model, such as a neural network model, is processed by various components of the processing system 600. For example, a hardware parallel configuration may be implemented by mapping some of the model's processing to various hardware (e.g., hardware X, hardware Y, and hardware Z) in a given processor (e.g., processor 601) and mapping some of the model's processing to other processors (e.g., processors 603 and 605) and their associated hardware. For example, parallel blocks in the parallel block processing architecture described herein may be mapped to different portions of each of the various hardware in processors 601, 603, and 605.

[0099] Example clauses Clause 1: A method for processing data, the method comprising: receiving, at a processing device, a set of global parameters for each machine learning model of a plurality of machine learning models; for each machine learning model of the plurality of machine learning models, processing, at the processing device, data stored locally on the processing device with the respective machine learning model in accordance with the set of global parameters to generate a machine learning model output; receiving, at the processing device, user feedback related to the machine learning model output; performing, at the processing device, optimization of each machine learning model based on the machine learning model output and the user feedback associated with the machine learning model output to generate locally updated machine learning model parameters; transmitting the locally updated machine learning model parameters to a remote processing device; and receiving, for each machine learning model of the plurality of machine learning models from the remote processing device, the set of globally updated machine learning model parameters for each respective machine learning model based at least in part on the locally updated machine learning model parameters.

[0100] Clause 2: The method of clause 1, further comprising the step of performing, at the processing device, some optimization before transmitting the locally updated machine learning model parameters to the remote processing device.

[0101] Clause 3: The method of any one of clauses 1 and 2, wherein the set of globally updated machine learning model parameters for each machine learning model of the plurality of machine learning models is based at least in part on locally updated machine learning model parameters of the second processing device.

[0102] Clause 4: The method of any one of clauses 1 to 3, wherein the user feedback includes an indication of correctness of the machine learning model output.

[0103] Clause 5: The method of any one of clauses 1 to 4, wherein the data stored locally on the processing device is one of image data, audio data, or video data.

[0104] Clause 6: The method of any one of clauses 1 to 5, wherein the processing device is one of a smartphone or an Internet of Things device.

[0105] Clause 7: A method according to any one of clauses 1 to 6, wherein, on the processing device, the step of processing the data stored locally on the processing device with the machine learning model is performed at least in part by one or more neural processing units.

[0106] Clause 8: A method according to any one of clauses 1 to 7, wherein in a processing device, the step of performing optimization of the machine learning model is at least partially performed by one or more neural processing units.

[0107] Clause 9: A method for processing data, the method comprising: for each machine learning model of a plurality of machine learning models, at each remote processing device of a plurality of remote processing devices, sending an initial set of global model parameters for the each machine learning model from a server to the respective remote processing device; receiving at the server from the respective remote processing device an updated set of model parameters for the each machine learning model; performing at the server an optimization of the each machine learning model based on the updated set of model parameters received from each remote processing device of the plurality of remote processing devices to generate an updated set of global model parameters; and sending the updated set of global model parameters for each machine learning model of the plurality of machine learning models from the server to each remote processing device of the plurality of remote processing devices.

[0108] Clause 10: The method of clause 9, wherein, at the server, performing optimization of each machine learning model includes calculating an effective gradient for each model parameter of an initial set of global model parameters for each machine learning model.

[0109] Clause 11: The method of any one of clauses 9 and 10, further comprising, for each respective machine learning model of the plurality of machine learning models, determining a corresponding density estimator parameterized by weighting parameters for the respective machine learning model.

[0110] Clause 12. The method of clause 11, further comprising determining pre-mixture weights for each machine learning model.

[0111] Clause 13: The method of any one of clauses 9 to 12, wherein the plurality of remote processing devices includes smartphones.

[0112] Clause 14: The method of any one of clauses 9 to 13, wherein the plurality of remote processing devices include Internet of Things devices.

[0113] Clause 15: The method of any one of clauses 9 to 14, wherein each of the plurality of machine learning models is a neural network model.

[0114] Clause 16: The method of clause 15, wherein each of the plurality of machine learning models has the same network structure.

[0115] Clause 17: A processing system comprising a memory containing computer-executable instructions and one or more processors configured to execute the computer-executable instructions to cause the processing system to perform a method according to any one of clauses 1 to 16.

[0116] Clause 18: A processing system comprising means for carrying out the method according to any one of clauses 1 to 16.

[0117] Clause 19: A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method of any one of clauses 1 to 16.

[0118] Clause 20: A computer program product embodied on a computer readable storage medium comprising code for performing the method of any one of clauses 1 to 16.

[0119] Additional Considerations The foregoing description is provided to enable any person skilled in the art to practice the various embodiments described herein. The examples described below are not intended to limit the scope, applicability, or embodiments described in the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. For example, changes may be made in the function and arrangement of the elements described without departing from the scope of the disclosure. The various examples may omit, substitute, or add various procedures or components as appropriate. For example, the methods described may be performed in a different order than described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of aspects described herein. In addition, the scope of the disclosure is intended to cover such apparatus or methods practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the disclosure described herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0120] As used herein, the term "exemplary" means "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

[0121] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. As an example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination of multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other permutation of a, b, and c).

[0122] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, searching (e.g., looking up in a table, database, or another data structure), ascertaining, and the like. Also, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Also, "determining" may include resolving, selecting, choosing, establishing, and the like.

[0123] The methods disclosed herein comprise one or more steps or actions for achieving the method. The steps and / or actions of the methods may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Furthermore, various operations of the methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs), or processors. In general, where there are operations illustrated in the figures, those operations may have corresponding counterpart means-plus-function components numbered similarly.

[0124] The following claims are not intended to be limited to the embodiments set forth herein, but are to be accorded the full scope consistent with the language of the claims. Within the claims, reference to an element in the singular shall mean "one or more," and not "one and only," unless expressly stated otherwise. The term "several" refers to one or more. No element of a claim shall be construed under the provisions of 35 U.S.C. 112(f) unless the element is expressly recited using the phrase "means for," or, in the case of a method claim, unless the element is recited using the phrase "step for." All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or later become known to those skilled in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Furthermore, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is expressly recited in the claims. [Explanation of symbols]

[0125] 100 Federated Learning Architecture 102A~102C Mobile Devices 104A~104C Local Data Store 106A~106C Local Machine Learning Model Instances 108 Global Machine Learning Model Coordinator 500 Electronic Devices 502 Central Processing Unit (CPU) 504 Graphics Processing Unit (GPU) 506 Digital Signal Processor (DSP) 508 Neural Processing Unit (NPU) 510 Multimedia Processing Block 512 Wireless Connection Block 514 Antenna 516 Sensor Processor 518 Image Signal Processor (ISP) 520 Navigation Processor 522 Input and / or Output Devices 524 memory 524A Transmitting Components 524B Receiver Components 524C Processing Components 524D Judgment Component 524E Output Component 524F Training Components 524G Inference Components 524H Optimized Components 600 Multiprocessor Processing System 601, 603, 605 Processors 607 Global Memory

Claims

1. 1. A method for processing data, comprising the steps of: receiving, at a processing device, a set of global parameters for each machine learning model of a plurality of machine learning models; For each of the plurality of machine learning models, processing, at the processing device, data stored locally on the processing device with each machine learning model according to the set of global parameters to generate machine learning model outputs; receiving, at the processing device, user feedback regarding the machine learning model output; performing, at the processing device and prior to transmitting locally updated machine learning model parameters to a remote processing device, an optimization of the respective machine learning models based on the machine learning model outputs and the user feedback associated with the machine learning model outputs to generate locally updated machine learning model parameters; transmitting the locally updated machine learning model parameters to the remote processing device; receiving, from the remote processing device, a set of globally updated machine learning model parameters for each machine learning model of the plurality of machine learning models; the set of globally updated machine learning model parameters for each respective machine learning model is based at least in part on the locally updated machine learning model parameters; A method comprising:

2. 2. The method of claim 1 , wherein the set of globally updated machine learning model parameters for each respective machine learning model of the plurality of machine learning models is based at least in part on locally updated machine learning model parameters of a second processing device.

3. The method of claim 1 , wherein the user feedback includes an indication of correctness of the machine learning model output.

4. The method of claim 1, wherein the data stored locally on the processing device is one of image data, audio data, or video data.

5. The method of claim 1, wherein the processing device is one of a smartphone or an Internet of Things device.

6. The method of claim 1, wherein, on the processing device, the step of processing the data stored locally on the processing device using the machine learning model is performed at least in part by one or more neural processing units.

7. The method of claim 1, wherein in the processing device, the step of performing the optimization of the machine learning model is performed at least in part by one or more neural processing units.

8. 1. A processing device, comprising: a memory containing computer executable instructions; One or more processors for executing the computer-executable instructions, the processing device comprising: receiving a set of global parameters for each machine learning model of a plurality of machine learning models; For each of the plurality of machine learning models, processing the data stored locally on the processing device with each machine learning model according to the set of global parameters to generate a machine learning model output; receiving user feedback regarding the machine learning model output; performing an optimization of the respective machine learning models based on the machine learning model outputs and the user feedback associated with the machine learning model outputs to generate locally updated machine learning model parameters prior to transmitting the locally updated machine learning model parameters to a remote processing device; transmitting the locally updated machine learning model parameters to a remote processing device; receiving from the remote processing device a set of globally updated machine learning model parameters for each machine learning model of the plurality of machine learning models; receiving a set of globally updated machine learning model parameters for each respective machine learning model based at least in part on the locally updated machine learning model parameters; one or more processors configured to cause A processing device comprising:

9. 10. The processing device of claim 8, wherein the set of globally updated machine learning model parameters for each respective machine learning model of the plurality of machine learning models is based at least in part on locally updated machine learning model parameters of a second processing device.

10. 10. The processing device of claim 8, wherein the user feedback includes an indication of correctness of the machine learning model output.

11. The processing device of claim 8, wherein the processing device is one of a smartphone or an Internet of Things device.

12. The processing device of claim 8, wherein one of the one or more processors is a neural processing unit configured to process the data stored locally on the processing device using the machine learning model.

13. The processing device of claim 8, wherein one of the one or more processors is a neural processing unit configured to perform the optimization of the machine learning model.

14. 1. A method for processing data, comprising the steps of: For each of the multiple machine learning models, For each of the plurality of remote processing devices, transmitting, from a server to each of the remote processing devices, an initial set of global model parameters for each of the machine learning models; receiving at the server from each of the remote processing devices an updated set of model parameters for each of the machine learning models; determining, for each respective machine learning model of the plurality of machine learning models, a corresponding density estimator parameterized by weighting parameters for the respective machine learning model; the weighting parameters are determined based on inputs that are being used to update a set of parameters for the machine learning model; performing, at the server, an optimization of the respective machine learning models based on the updated set of model parameters received from each remote processing device of the plurality of remote processing devices and further based on the density estimator to generate an updated set of global model parameters; transmitting from the server to each remote processing device of the plurality of remote processing devices the updated set of global model parameters for each machine learning model of the plurality of machine learning models; A method comprising:

15. 15. The method of claim 14, wherein performing the optimization at the server for the respective machine learning model comprises calculating an effective gradient for each model parameter of the initial set of global model parameters for the respective machine learning model.

16. The method of claim 14 , further comprising determining pre-mixture weights for each of the machine learning models.

17. the plurality of remote processing devices include smartphones; the plurality of remote processing devices include Internet of Things devices; The method of claim 14.

18. The method of claim 14 , wherein each of the plurality of machine learning models is a neural network model.

19. The method described in claim 18, wherein each of the plurality of machine learning models has the same network structure.

20. 1. A processing device, comprising: a memory containing computer executable instructions; One or more processors for executing the computer-executable instructions, the processing device comprising: For each of the multiple machine learning models, For each of the plurality of remote processing devices, transmitting an initial set of global model parameters for each of the machine learning models to each of the remote processing devices; receiving an updated set of model parameters for the respective machine learning models from the respective remote processing devices; determining, for each respective machine learning model of the plurality of machine learning models, a corresponding density estimator parameterized by weighting parameters for the respective machine learning model; the weighting parameters are determined based on inputs that are being used to update a set of parameters for the machine learning model; and performing an optimization of the respective machine learning models based on the updated set of model parameters received from each remote processing device of the plurality of remote processing devices to generate an updated set of global model parameters and further based on the density estimator; transmitting the updated set of global model parameters for each machine learning model of the plurality of machine learning models to each remote processing device of the plurality of remote processing devices; one or more processors configured to cause A processing device comprising:

21. 21. The processing device of claim 20, wherein the one or more processors are further configured to cause the processing device to calculate an effective gradient for each model parameter of the initial set of global model parameters for the respective machine learning model to perform the optimization of the respective machine learning model.

22. 21. The processing device of claim 20, wherein the one or more processors are further configured to cause the processing device to determine, for each respective machine learning model of the plurality of machine learning models, pre-mixture weights for the respective machine learning model.

23. 21. The processing device of claim 20, wherein each of the plurality of machine learning models is a neural network model.

24. A processing device as described in claim 23, wherein each of the plurality of machine learning models has the same network structure.

Images