Decentralized learning of machine learning models by utilizing outdated updates received from lagger computing devices

Abstract

During a round of decentralized learning for updating a global machine learning (ML) model, a remote processor of a remote system may send a primary weight for a primary version of the global ML model to a population of computing devices and cause each of the computing devices to generate a corresponding update for the primary version of the global ML model. Further, the remote processor may cause a primary version of the global ML model to be updated based on corresponding updates received during decentralized learning of the round. However, the remote processor may receive other corresponding updates after the round of decentralized learning. Thus, various techniques described herein (e.g., FARe-DUST, MSG-based FeAST, and / or other techniques) enable other corresponding updates to be utilized in implementing a final version of a global ML model.

Description

Background Art

[0001] Due to various privacy considerations, decentralized learning of machine learning (ML) models is an increasingly popular ML technique for updating ML models. In a common implementation of decentralized learning, an on-device ML model is stored locally on a user's client device, and a global ML model, which is a cloud-based counterpart to the on-device ML model, is stored remotely at a remote system (e.g., a server or a cluster of servers). During a given round of decentralized learning, a client device using the on-device ML model may process instances of client data detected at the client device to generate predicted outputs, and may generate updates for the global ML model based on the instances of the processed client data. Further, the client device may send the update to the remote system. The remote system may update the global weights of the global ML model using the update received from the client device and additional updates generated at and received from additional client devices in a similar manner. The remote system may send the updated global ML model (or updated global weights of the updated global ML model) to the client device and the additional client devices. The client device and the additional client devices may then replace the respective on-device ML models with the updated global ML model (or replace the respective on-device weights of the respective on-device ML models with the updated global weights of the global ML model), thereby updating the respective on-device ML models.

[0002] However, the client devices and additional client devices participating in a given round of decentralized learning have different latencies associated with generating corresponding updates and / or sending corresponding updates to the remote system. For example, the client device and each of the additional client devices may only be able to dedicate a certain amount of computing resources to generating corresponding updates, so that corresponding updates are generated at different rates at the client device and each of the additional client devices. In addition, for example, the client device and each of the additional client devices may have different connection types and / or strengths, so that corresponding updates are sent to the remote system and from the client device and each of the additional client devices at different rates. Nevertheless, when decentralized learning utilizes a synchronous training algorithm, the remote system may have to wait for corresponding updates from one or more of the slowest client devices (also referred to as "straggler devices") among the client devices and additional client devices before updating the global ML model based on the corresponding updates. Therefore, because the remote system is forced to wait for corresponding updates from these stragglers (also referred to as "outdated updates"), the update of the global ML model is performed in a suboptimal manner.

[0003] A common technique for eliminating problems caused by these stragglers when decentralized learning utilizes synchronous training algorithms is to utilize only corresponding updates received from a small portion of client devices and / or additional client devices that provide corresponding updates at the fastest rate, and discard outdated updates received from any of these stragglers. However, the resulting global ML model updated based only on corresponding updates received from a small portion of client devices and / or additional client devices that provide corresponding updates at the fastest rate may be biased against the data domain associated with these stragglers and / or have other unintended consequences. Another common technique for eliminating problems caused by these stragglers in decentralized learning is to utilize asynchronous training algorithms. Although utilizing asynchronous training algorithms in decentralized learning does eliminate some problems caused by these stragglers, the update of the global ML model is still performed in a suboptimal manner due to the lack of a powerful representation of these outdated updates received from these stragglers by the remote system.

[0004] Therefore, there is a need in the art for techniques to eliminate the problems caused by these outdated updates received from these stragglers by not only allowing remote systems to advance updates to the global ML model without having to wait for these outdated updates from these stragglers, but also utilizing these outdated updates received from these stragglers in a more efficient manner to enhance the representation of these outdated updates received from these stragglers. Summary of the invention

[0005] Implementations described herein relate to various techniques for improving decentralized learning of a global machine learning (ML) model. For example, for a given round of decentralized learning for updating a global ML model, a remote processor of a remote system (e.g., a server or a cluster of servers) may send master weights for a master version of the global ML model to a population of computing devices; and cause each of the computing devices in the population to generate corresponding updates for the master version of the global ML model via utilizing the master version of the global ML model at each of the computing devices in the population. Further, the remote processor may asynchronously receive a first subset of the corresponding updates for the master version of the global ML model from one or more of the computing devices in the population; and cause the master version of the global ML model to be updated based on the first subset of the corresponding updates to generate updated master weights for the updated version of the global ML model. Notably, the first subset of corresponding updates may be received during a given round of decentralized learning for updating a given global ML model, and may include corresponding updates from less than all computing devices in the population. Thus, the remote processor can continue to generate updated primary weights for the updated version of the global ML model even if not all computing devices in the population have provided corresponding updates, and avoid waiting for corresponding updates from one or more other computing devices in the population (e.g., avoiding waiting for outdated updates from stragglers). Furthermore, the remote processor can cause one or more given additional rounds of decentralized learning for updating the global ML model to be implemented with corresponding additional populations of client devices, such that the primary version of the global ML model is continuously updated.

[0006] However, after the given round of decentralized learning for updating the global ML model (e.g., during one or more given additional rounds of decentralized learning for updating the global ML model), the remote processor may asynchronously receive from one or more other computing devices in the swarm a second subset of corresponding updates from one or more previous rounds of decentralized learning for updating the global ML model (e.g., asynchronously receive outdated updates from stragglers). Notably, when the second subset of corresponding updates is received from one or more other computing devices in the swarm, the major version of the global ML model has been updated during the given round of decentralized learning for updating the global ML model (and may be further updated during one or more given additional rounds of decentralized learning for updating the global ML model). Therefore, due to the difference between the primary weights (e.g., utilized by one or more of the other computing devices when generating the corresponding updates) and the updated (or further updated) primary weights, causing the updated (or further updated) major version of the global ML model to be updated based on the second subset of corresponding updates is suboptimal. Nevertheless, the techniques described herein can utilize the second subset of corresponding updates to influence future updates of the major version of the global ML model and / or the deployed final version of the global ML model.

[0007] In some implementations, the remote processor may implement a technique that causes a master version of the global ML model (e.g., originally sent to a population of computing devices during a given round of decentralized learning of the global ML model) to generate a corresponding historical version of the global ML model based on the first subset of corresponding updates and the second subset of corresponding updates, and causes the corresponding historical version of the global ML model to be used as a corresponding teacher model for one or more of the given additional rounds of decentralized learning for updating the global ML model via distillation. This technique may be referred to as "Federated Asynchronous Regularization with Distillation Using Stale Teachers" (FARe-DUST), and ensures that knowledge in the second subset of corresponding updates provided by one or more of the additional computing devices (e.g., stragglers) is distilled into the master version of the global ML model during one or more of the given additional rounds of decentralized learning for updating the global ML model without requiring the remote processor to wait for the second subset of corresponding updates provided by one or more of the additional computing devices during the given additional round of decentralized learning for updating the global ML model.

[0008] In additional or alternative implementations, the remote processor may implement a technique that causes a primary version of the global ML model (e.g., originally sent to the population of computing devices during a given round of decentralized learning of the global ML model) to generate a corresponding historical version of the global ML model based on the first subset of corresponding updates and the second subset of corresponding updates, but causes the corresponding historical version of the global ML model to be combined with the updated primary version of the global ML model to generate an auxiliary version of the global ML model. The remote processor may continue to cause the primary version of the global ML model to be updated, cause additional corresponding historical versions of the global ML model to be generated (e.g., based on additional asynchronous receipt of outdated updates from additional stragglers computing devices during one or more given additional rounds of decentralized learning for updating the global ML model), and generate additional auxiliary versions of the global ML model, such that the most recent auxiliary version of the global ML model can be used as the final version of the global ML model. This technique may be referred to as “Federated Asynchronous Straggler Training with Mismatched and Stale Gradients” (MSG-based FeAST), and ensures that knowledge in the corresponding updated second subset provided by one or more of the additional computing devices (e.g., stragglers) is aggregated into a unified auxiliary version of the global ML model.

[0009] As described in more detail herein, one or more of the given additional rounds of decentralized learning for updating the global ML model may be different based on whether the remote processor implements FARe-DUST or the MSG-based FeAST technique. In various implementations, a developer or other user associated with the remote system may provide the remote processor with an indication of whether to implement FARe-DUST, the MSG-based FeAST technique, or some combination of the two techniques. In various implementations, a developer or other user associated with the remote system may provide the remote processor with an indication of the type of group of computing devices to be utilized when implementing FARe-DUST, the MSG-based FeAST technique, or some combination of the two techniques. For example, the group of computing devices may include one or more client devices of the corresponding users, so that corresponding updates may be generated in a manner that utilizes a federated learning framework. Additionally or alternatively, the group of computing devices may include one or more remote servers other than the remote system.

[0010] As used herein, a "round of decentralized learning" may be initiated when a remote processor sends data to a population of computing devices for the purpose of updating a global ML model. The data sent to the population of computing devices for the purpose of updating the global ML model may include, for example, master weights for a master version of the global ML model (or updated master weights for an updated master version of the global ML model for one or more subsequent rounds of decentralized learning to update the global ML model), one or more corresponding historical versions of the global ML model, other data (e.g., audio data, visual data, textual data, and / or other data) that may be processed by computing devices in the population in generating corresponding updates, and / or any other data. Further, the round of decentralized data may conclude when the remote processor causes the master weights for the master version of the global ML model to be updated (or causes the updated master weights for the updated master version of the global ML model to be updated during one or more subsequent rounds of decentralized learning in subsequent rounds of decentralized learning to update the global ML model). Notably, the remote processor may cause the master weights for the master version of the global ML model to be updated (or cause the updated master weights for the updated master version of the global ML model to be updated during one or more subsequent rounds of decentralized learning for updating the global ML model) based on one or more criteria. The one or more criteria may include, for example, receiving a threshold number of corresponding updates from one or more of the computing devices in the population (e.g., so that any other corresponding updates received from one or more of the other computing devices in the population can be utilized in generating and / or updating the corresponding historical version of the global ML model), a threshold amount of time has passed since the round of decentralized learning was initiated (e.g., 5 minutes, 10 minutes, 15 minutes, 60 minutes, etc.), and / or other criteria.

[0011] As used herein, a "major version of a global ML model" may correspond to an instance of a global ML model that is stored in a remote storage of a remote system and that is continuously updated through multiple rounds of decentralized learning. Notably, a major version of a global ML model may refer to the major version of the global ML model itself, its major weights, or both. In various implementations, after each round of decentralized learning, additional instances of the major version of the global ML model may be stored in the remote storage of the remote system, such as an updated major version of the global ML model after a given round of decentralized learning, a further updated major version of the global ML model after a given additional round of decentralized learning, and so on for any further additional rounds of decentralized learning used to update the global ML model. Thus, multiple major versions of the global ML model may be stored in the remote storage of the remote system at any given time. In some versions of those implementations, each of the multiple major versions of the global ML model may be stored in the remote storage of the remote system and associated with an indication of the corresponding round of decentralized learning during which the corresponding version of the multiple major versions of the global ML model was updated. As described in more detail herein, indications of corresponding rounds of decentralized learning during which corresponding versions of multiple primary versions of the global ML model were updated enable generation of other versions of the global ML model (e.g., corresponding historical versions of the global ML model, corresponding auxiliary versions of the global ML model, etc.).

[0012] By using the techniques described herein, one or more technical advantages may be achieved. As a non-limiting example, by utilizing outdated updates received from stragglers computing devices to generate corresponding historical versions of the global ML model and / or corresponding auxiliary versions of the global ML model, the final version of the global ML model has more knowledge about the domains associated with these stragglers computing devices and / or other unintended consequences are mitigated. Therefore, the final version of the global ML model is more robust to the domains associated with these stragglers computing devices. As another non-limiting example, the consumption of computing resources by these stragglers computing devices is not wasted unnecessarily because the outdated updates generated by these stragglers computing devices are utilized when generating the final version of the global ML model, rather than being simply wasted. As another non-limiting example, by utilizing the FARe-DUST techniques and / or the MSG-based FeAST techniques described herein, the outdated updates generated by these stragglers computing devices can be utilized when generating the final version of the global ML model in a faster and more efficient manner than other known techniques.

[0013] The above description is provided as an overview of some implementations of the present disclosure. Further descriptions of those implementations and other implementations will be described in more detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Depicted are example process flows demonstrating various aspects of the disclosure according to various implementations.

[0015] Figure 2A An example technique (e.g., “Federated Asynchronous Regularization with Distillation using Stale Teachers” (FARe-DUST)) for improving decentralized learning of machine learning models using outdated updates received from stragglers computing devices is described.

[0016] Figure 2B Another example technique for improving decentralized learning of machine learning models using outdated updates received from stragglers computing devices (e.g., “Federated Asynchronous Straggler Training with Mismatched and Stale Gradients” (MSG-based FeAST)) is described.

[0017] Figure 3 Depicted are block diagrams showing various aspects of the disclosure according to various implementations.

[0018] Figure 4 Depicted is a flowchart illustrating an example method for causing a master version of a global machine learning model to be updated during a given round of decentralized learning of the global machine learning model but to receive outdated updates from stragglers computing devices after the given round of decentralized learning of the global machine learning model, according to various implementations.

[0019] Figure 5A Depicted is a flowchart illustrating an example method (e.g., “Federated Asynchronous Regularization with Distillation using Stale Teachers” (FARe-DUST)) for improving decentralized learning of machine learning models using stale updates received from stragglers computing devices according to various implementations.

[0020] Figure 5B A flowchart illustrating another example method (e.g., “Federated Asynchronous Straggler Training with Mismatched and Stale Gradients” (MSG-based FeAST)) for improving decentralized learning of machine learning models using stale updates received from stragglers computing devices according to various implementations is depicted.

[0021] Figure 6 Depicted are example architectures for computing devices in accordance with various implementations. DETAILED DESCRIPTION

[0022] Now turn to Figure 1 , depicting an example process flow that demonstrates various aspects of the present disclosure. Multiple computing devices 1201-120 Nand a remote system 160 (e.g., a remote server or a cluster of remote servers) at Figure 1 are exemplified in and each includes Figure 1 The corresponding boxes in the figure represent computing devices 1201-120 N Multiple computing devices 1201-1202 may be utilized during one or more rounds of decentralized learning for updating a global machine learning (ML) model (e.g., stored in a global ML model database 160B) as described herein. N and remote system 160. Multiple computing devices 1201-120 N The remote system 160 may include, for example, a client device of a corresponding user, other remote servers, or clusters of other remote servers, and / or other computing devices. For example, the remote system 160 may be configured to utilize multiple computing devices 1201-120 N One or more computing devices in the system facilitate decentralized learning of a global ML model. The global ML model may include any audio-based ML model, vision-based ML model, text-based ML model, and / or any other type of ML model. Some non-limiting examples of global ML models (e.g., Figure 3 ) are described herein, but it should be understood that those ML models are provided for example and are not intended to be limiting.

[0023] In various implementations, the decentralized learning engine 162 of the remote system 160 may identify a global ML model (e.g., in the global ML model database 160B) to be updated during a given round of decentralized learning for updating the global ML model. In some implementations, the decentralized learning engine 162 may identify the global ML model to be updated using decentralized learning based on an indication provided by a developer or other user associated with the remote system 160. In additional or alternative implementations, the decentralized learning engine 162 may randomly select a global ML model to be updated using decentralized learning from the global ML model database 160B without receiving any indication from a developer or other user associated with the remote system 160. Although Figure 1 The description is about the decentralized learning engine 162 identifying a single global ML model for decentralized learning, but it should be understood that this is for example and is not intended to be limiting. Instead, it should be understood that the decentralized learning engine 162 can identify multiple global ML models for decentralized learning, so that multiple rounds of decentralized learning of the corresponding multiple global ML models can be performed in parallel.

[0024] In various implementations, a computing device identification engine 164 of a remote system 160 may identify a population of computing devices to participate in a given round of decentralized learning for updating a global ML model. In some implementations, the computing device identification engine 164 may identify all available computing devices communicatively coupled to the remote system 160 (e.g., via one or more networks) for inclusion in the population of computing devices to participate in a given round of decentralized learning for updating a global ML model. In other implementations, the computing device identification engine 164 may identify a specific number of computing devices (e.g., 100 computing devices, 1,000 computing devices, 10,000 computing devices, and / or other numbers of computing devices) communicatively coupled to the remote system 160 for inclusion in the population of computing devices to participate in a given round of federated learning for updating a global ML model. In order to Figure 1 For example, it is assumed that at least computing device 1201 and computing device 120 N Identified for communication with (e.g., as provided by computing device 1201 and computing device 120 N 1201 and 1202 are included in the group of client devices. N For inclusion in a population of computing devices to participate in a given round of federated learning for updating a global ML model, the ML model distribution engine 172 may send at least the primary weight for the primary version of the global ML model (e.g., as indicated by 172A) to computing devices 1201 and 120 N , so that each of the computing devices in the population generates a corresponding update for the major version of the global ML model.

[0025] In various implementations, in response to receiving at least a primary weight for a primary version of a global ML model from remote system 160, computing device 1201 and computing device 120 N The master weights for the master version of the global ML model may be stored in a corresponding storage (e.g., ML model database 120B1 of computing device 1201, computing device 120 N ML model database 120B N , and so on for each of the other computing devices in the group). In some versions of those implementations, computing device 1201 and computing device 120 N Any previous weights for the global ML model may be replaced in the corresponding storage with the primary weights for the primary version of the global ML model. NEach of them can generate corresponding updates for the global ML model using the master weights for the master version of the global ML model during a given round of decentralized learning.

[0026] In various implementations, when generating corresponding updates for the global ML model during a given round of decentralized learning, the corresponding ML model engine (e.g., ML model engine 1221 of computing device 1201, computing device 120 N ML Model Engine 122 N , and so on for each of the other computing devices in the population) can be processed using the primary version of the ML model (e.g., from the data database 120A1 of computing device 1201, computing device 120 N Data Database 120A N , and for each other computing device in the group, and so on) corresponding data to generate one or more corresponding predicted outputs (e.g., predicted output 122A1 of computing device 1201, predicted output 122A2 of computing device 120 N The predicted output 122A N , and so on for each other of the other computing devices in the population). Notably, the output of the corresponding data and one or more corresponding predictions may depend on the type of global ML model being updated during a given round of decentralized learning.

[0027] For example, in an implementation where the global ML model is an audio-based ML model, the corresponding data processed to generate one or more corresponding predicted outputs may be corresponding audio data and / or features of the corresponding audio data. Further, one or more corresponding predicted outputs generated based on processing the corresponding audio data and / or features of the corresponding audio data may depend on the type of the audio-based ML model. For example, in an implementation where the audio-based ML model is a hot word detection model, one or more corresponding predicted outputs generated based on processing the corresponding audio data or features of the corresponding audio data may be a value (e.g., a binary value, a probability, a log-likelihood, or another value) indicating whether the audio data captures a specific word or phrase that invokes a corresponding automated assistant when detected. In addition, for example, in an implementation where the audio-based ML model is an automatic speech recognition (ASR) model, one or more corresponding predicted outputs generated based on processing the corresponding audio data or features of the corresponding audio data may be a distribution of values ​​(e.g., probabilities, log-likelihoods, or another value) over a word or phrase vocabulary, and the recognized text (e.g., predicted to correspond to a spoken utterance captured in the audio data) may be determined based on the distribution of values ​​over the word vocabulary.

[0028] As another example, in an implementation where the global ML model is a vision-based ML model, the corresponding data processed to generate one or more corresponding predicted outputs may be corresponding visual data and / or features of the corresponding visual data. Further, one or more corresponding predicted outputs generated based on processing the corresponding visual data and / or features of the corresponding visual data may depend on the type of the vision-based ML model. For example, in an implementation where the vision-based ML model is an object classification model, one or more corresponding predicted outputs generated based on processing the corresponding visual data and / or features of the corresponding visual data may be a distribution of values ​​(e.g., probabilities, log-likelihoods, or another value) over multiple objects, and the one or more given objects may be determined (e.g., predicted to be captured in one or more frames of visual data) based on the distribution of values ​​over multiple objects. Additionally, for example, in an implementation where the vision-based ML model is a facial recognition model, one or more corresponding predicted outputs generated based on processing the corresponding visual data and / or features of the corresponding visual data may be an embedding (or other low-level representation) that can be compared with a previously generated embedding (or other low-level representation) in a low-dimensional space that can be utilized to identify a user captured in one or more frames of visual data.

[0029] As yet another example, in an implementation where the global ML model is a text-based ML model, the corresponding data processed to generate one or more corresponding predicted outputs may be corresponding text data and / or features of the corresponding text data. Further, the one or more corresponding predicted outputs generated based on processing the corresponding text data and / or features of the corresponding text data may depend on the type of the text-based ML model. For example, in an implementation where the text-based ML model is a natural language understanding (NLU) model, the one or more corresponding predicted outputs generated based on processing the corresponding text data and / or features of the corresponding text data may be one or more annotations identifying the predicted intent included in the text data, one or more slot values ​​of one or more parameters associated with one or more of the intents, and / or other NLU data.

[0030] Therefore, it should be understood that the corresponding data processed by the corresponding ML model engine using the major version of the global ML model can be different based on the type of global ML model being updated during a given round of decentralized learning. Although the above examples are described with respect to using a specific corresponding ML model to process specific corresponding data, it should be understood that this is for example and is not intended to be limiting. Instead, it should be understood that any ML model that can be trained using decentralized learning is contemplated herein.

[0031] In various implementations, when generating corresponding updates for the global ML model during a given round of decentralized learning, a corresponding gradient engine (e.g., gradient engine 1241 of computing device 1201, computing device 120 N Gradient Engine 124 N , and so on for each of the other computing devices in the group) may be based on at least one or more corresponding predicted outputs (e.g., predicted output 122A1 of computing device 1201, predicted output 122A1 of computing device 120 N The predicted output 122A N , and so on for each of the other computing devices in the population) to generate a corresponding gradient (e.g., gradient 124A1 for computing device 1201, gradient 124A2 for computing device 120 N Gradient 124 N , and so on for each other computing device in the population. In these implementations, the corresponding gradient engine may optionally communicate with the corresponding learning engine (e.g., learning engine 1261 of computing device 1201, computing device 120 N Learning Engine 126 N , and so on for each other computing device in the other computing devices in the population). The corresponding learning engine can cause the corresponding gradient engine to utilize various learning techniques (e.g., supervised learning, semi-supervised learning, unsupervised learning, or any other learning techniques or combinations thereof) when generating the corresponding gradient.

[0032] For example, in an implementation in which there is an available supervisory signal, the corresponding learning engine may cause the corresponding gradient engine to utilize a supervised learning technique. Otherwise, the corresponding learning engine may cause the corresponding gradient engine to utilize a semi-supervised or unsupervised learning technique (e.g., a student-teacher technique, a masking technique, and / or another semi-supervised or unsupervised learning technique). For example, it is assumed that the global ML model is a hot word detection model that is utilized to process the corresponding audio data and / or the features of the corresponding audio data. It is further assumed that one or more corresponding predicted outputs generated at a given computing device (e.g., the predicted output 122A1 of the computing device 1201) indicate that the audio data does not capture a specific word or phrase to call the corresponding automated assistant. However, it is further assumed that the corresponding user of the given computing device subsequently calls the corresponding automated assistant by other means (e.g., via actuating a hardware button or a software button) immediately after generating the corresponding audio data. In this example, the subsequent call of the corresponding automated assistant can be used as a supervisory signal. Thus, in this example, the corresponding learning engine (e.g., learning engine 1261 of computing device 1201) can cause the corresponding gradient engine (e.g., gradient engine 1241 of computing device 1201) to compare one or more predicted outputs (e.g., predicted output 122A1 of computing device 1201) that (incorrectly) indicate that the corresponding automated assistant should not be invoked with one or more true-value outputs (correctly) indicating whether the corresponding automated assistant should be invoked, to generate a corresponding gradient (e.g., gradient 124A1 of computing device 1201).

[0033] In contrast, it is further assumed that the corresponding user of the given computing device has not subsequently invoked the corresponding automated assistant, such that an unsupervised signal is available. In some of these examples, in accordance with the semi-supervised student-teacher technique, a corresponding learning engine (e.g., learning engine 1261 of computing device 1201) may process the corresponding audio data using a teacher hotword detection model (e.g., stored in ML model database 120B1, global ML model database 160B, and / or other database accessible to client device 1201) to generate one or more teacher outputs, the one or more teacher outputs also indicating whether the corresponding audio data captures a particular word or phrase that, when detected, invokes the corresponding automated assistant. In these examples, the teacher hotword detection model may be another hotword detection model, or another distinct hotword detection model, that is utilized to generate one or more corresponding predicted outputs. Further, the corresponding gradient engine 1241 may compare one or more corresponding predicted outputs with one or more teacher outputs to generate a corresponding gradient.

[0034] In other examples, according to the semi-supervised masking technique, the corresponding learning engine (e.g., the learning engine 1261 of the computing device 1201) can mask the target portion of the corresponding audio data (e.g., a portion of the data that can include a specific word or phrase in the audio data), and can cause the corresponding ML model engine (e.g., the ML model engine 1221 of the computing device 1201) to use the hot word detection model to process other portions of the corresponding audio data when generating one or more corresponding predicted outputs (e.g., one or more predicted outputs 122A1). The one or more corresponding predicted outputs may still include a value indicating the following: based on processing other portions of the corresponding audio data (e.g., based on features of the other portions of the corresponding audio data, such as mel-bank features, mel-frequency cepstral coefficients (mel-frequency cepstralcoefficient) and / or other features of the corresponding audio data), whether the target portion of the audio data is predicted to include a specific word or phrase. In some of these instances, the corresponding learning engine (e.g., the learning engine 1261 of the computing device 1201) may also use the hot word detection model to process the unmasked version of the audio data to generate one or more benchmark outputs, the one or more benchmark outputs including a value of whether the corresponding audio data is predicted to include a specific word or phrase. Further, the corresponding gradient engine (e.g., the gradient engine 1241 of the computing device 1201) may compare one or more corresponding predicted outputs with one or more benchmark outputs generated by the learning engine 1261 to generate a corresponding gradient 124A1 (e.g., the gradient 124A1 of the computing device 1201).

[0035] Although the above examples of semi-supervised learning and unsupervised learning are described with respect to specific techniques, it should be understood that those techniques are provided as non-limiting examples of semi-supervised learning techniques or unsupervised learning techniques, and are not intended to be limiting. On the contrary, it should be understood that any other unsupervised learning techniques or semi-supervised learning techniques are contemplated herein. In addition, although the above examples are described with respect to hot word detection models, it should be understood that this is also for example, and is not intended to be limiting. On the contrary, it should be understood that the same or similar techniques can be used to generate corresponding gradients in the same or similar manner.

[0036] In various implementations, the corresponding gradients may be corresponding updates sent from each of the computing devices in the group and back to the remote system 160. In additional or alternative implementations, the corresponding ML model update engine (e.g., the ML model update engine 1281 of the computing device 1201, the computing device 120 N ML model update engine 128 N, and so on for each other computing device in the other computing devices in the population) can update the primary weights for the primary version of the global ML model at the computing device based on the corresponding gradients (e.g., using stochastic gradient descent or another technique), resulting in updated primary weights for the global ML model. In these implementations, the updated primary weights for the global ML model or the difference between the primary weights (e.g., before the update) and the updated primary weights (e.g., after the update) can correspond to the corresponding updates sent from each of the computing devices in the population and back to the remote system 160.

[0037] In order to Figure 1 For example, assume that a corresponding update is received from computing device 1201 during a given round of decentralized learning, but a corresponding update is received from computing device 120 after a given round of decentralized learning. N Receive the corresponding update. In this example, computing device 1201 can be referred to as a "fast computing device" (e.g., so Figure 1 The name "fast computing device 1201" is shown) and the corresponding update reception can be regarded as "fast computing device update 120C1". Further, in this example, computing device 120 N May be referred to as a "straggler computing device" (e.g., Figure 1 The name "straggler computing device 120 N ”) and the corresponding update reception can be regarded as “straggler computing device update 120C N ". Notably, prior to initiating a given round of decentralized learning, none of the computing devices in the population is designated as "fast" or "straggler." Rather, these designations depend on whether the respective computing device sends corresponding updates back to the remote system 160 during the given round of decentralized learning due to computing device considerations and / or network considerations. Thus, a given computing device participating in multiple rounds of decentralized learning may be a fast computing device in some rounds of decentralized learning, but a straggler computing device in other rounds of decentralized learning.

[0038] In various implementations, the global ML model update engine 170 may cause the updated major version of the global ML model to be updated based on the fast computing device update 120C1 and other fast computing device updates received from other fast computing devices in the population, thereby generating updated major weights for the updated major version of the global ML model. In some versions of those implementations, the global ML model update engine 170 may cause the updated major version of the global ML model to be continuously updated based on the fast computing device update 120C1 and other fast computing device updates (as they are received). In additional or alternative implementations, the fast computing device update 120C1 and other fast computing device updates may be stored in one or more databases (e.g., update database 160A) as they are received, and then one or more techniques are utilized to cause the updated major version of the global ML model to be updated based on the fast computing device update 120C1 and other fast computing device updates (e.g., using a federated averaging technique or another technique). In these implementations, the global ML model update engine 170 may cause the updated major version of the global ML model to be updated based on the fast computing device update 120C1 and other fast computing device updates in response to determining that one or more conditions are satisfied (e.g., whether a threshold number of fast computing device updates have been received, whether a threshold duration has passed since a given round of decentralized learning was initiated, etc.).

[0039] In various implementations, the decentralized learning engine 162 may cause a given additional round of decentralized learning for further updating the global ML model to be initiated in the same or similar manner as described above but with respect to the updated primary weight for the updated primary version of the global ML model and with respect to the additional population of additional client devices. N Received stragglers computing device update 120C N , the remote system may have advanced to a given additional round of decentralized learning for further updating the global ML model. Nevertheless, the remote system 160 may still employ various techniques to utilize the stragglers computing device to update 120C N To further update the global ML model. In some implementations, such as Figure 2A As described, the remote system 160 can employ a technique referred to herein as “Federated Asynchronous Regularization and Distillation Using Outdated Teachers” (FARe-DUST), which utilizes the historical global ML model engine 166. In additional or alternative implementations, such as with respect to Figure 2BAs described, the remote system 160 can employ a technique referred to herein as “Federated Asynchronous Straggler Training with Mismatched and Stale Gradients” (MSG-based FeAST), which utilizes a historical global ML model engine 166 and an auxiliary global ML model engine 168.

[0040] Now turn to Figure 2A and Figure 2B , depicts various example techniques for decentralized learning of machine learning (ML) models using outdated updates received from stragglers. For example, Figure 2A depicts a node view 200A of a technique referred to herein as FARe-DUST, and Figure 2B A node view 200B of a technique referred to herein as MSG-based FeAST is depicted. In various implementations, a remote system (e.g., Figure 1 A developer or other user associated with a remote system 160 of a corresponding user may provide an indication of whether to implement FARe-DUST, the MSG-based FeAST technique, or some combination of the two techniques during one or more rounds of decentralized learning for updating the global ML model. In various implementations, a developer or other user associated with the remote system may provide an indication of the type of population of computing devices to be utilized when implementing FARe-DUST, the MSG-based FeAST technique, or some combination of the two techniques. For example, the population of computing devices may include one or more client devices of the corresponding user (e.g., Figure 3 The group of computing devices may also include one or more remote servers other than the remote system (e.g., other than the remote system). Figure 1 Other remote servers or clusters of remote servers other than the remote system 160).

[0041] Specific reference Figure 2A , assuming that node 202 represents the primary weight w of the primary version of the global ML model t , the main weights will be updated through multiple rounds of decentralized learning for updating the global ML model. Further assume that the remote system (e.g., Figure 1 The remote system 160 in t starts a given round of decentralized learning for updating the global ML model by: updating the master weights w for the master version of the global ML model t to a group of computing devices to cause each of the computing devices to communicate via each of the computing devices (e.g., as described with respect to Figure 1 The computing device 1201 and the computing device 120 NThe main weights w for the main version of the global ML model are used as described above. t To generate corresponding updates for the major version of the global ML model. Further, the remote system can asynchronously receive corresponding updates (e.g., as indicated by Δ t It is worth noting that one or more computing devices in the population that provide corresponding updates during a given round of decentralized learning for updating the global ML model can be referred to as fast computing devices (e.g., so Figure 1 In addition, the remote system may cause the master weights w of the master version of the global ML model to t Based on corresponding updates received from one or more of the computing devices in the population during a given round of decentralized learning for updating the global ML model, as represented by an updated primary weight w of the updated primary version of the global ML model t+1 As indicated by node 204 .

[0042] Further assume that the remote system updates the master weights w of the updated major version of the global ML model by t+1 The additional group of additional computing devices is sent to cause each of the additional computing devices to communicate with each of the additional computing devices via each of the additional computing devices (e.g., as described with respect to Figure 1 The computing device 1201 and the computing device 120 N The updated main weights w of the updated main version of the global ML model are used at t+1 The remote system may generate additional corresponding updates for the updated major version of the global ML model to initiate a given additional round of decentralized learning for updating the global ML model. Further, the remote system may asynchronously receive additional corresponding updates (e.g., as indicated by Δ t+1 It is worth noting that one or more additional computing devices in the additional population that provide corresponding updates during a given additional round of decentralized learning for updating the global ML model may also be referred to as fast computing devices (e.g., thus Figure 1 1201 ). In addition, the remote system may cause the updated primary weights w of the updated primary version of the global ML model to be updated. t+1The global ML model is further updated based on additional corresponding updates received from one or more of the additional computing devices in the additional population during a given additional round of decentralized learning for further updating the global ML model, as represented by a further updated primary weight w representing a further updated primary version of the global ML model. t+2 The remote system may continue to advance the major version of the global ML model (e.g., as indicated by the further updated major weight w representing the further updated major version of the global ML model). t+3 , and the ellipsis following the node 208).

[0043] However, further assume that during a given additional round of decentralized learning for updating the global ML model, the remote system asynchronously receives corresponding updates from the given additional round of decentralized learning for updating the global ML model from one or more other computing devices in the swarm (e.g., as determined by Δ t It is worth noting that one or more other computing devices in the population that provide corresponding updates after a given round of decentralized learning for updating the global ML model may also be referred to as stragglers (e.g., Figure 1 In addition, these corresponding updates are received after the remote system has advanced the major version of the global ML model to the updated major version of the global ML model. Therefore, these corresponding updates are generated by one or more other computing devices in the swarm based on the major version of the global ML model, and simply updating the updated major version of the global ML model would be ineffective (e.g., due to a weight mismatch between the major version of the global ML model and the updated major version of the global ML model).

[0044] Nonetheless, the remote system may update the global ML model based on corresponding updates received from one or more computing devices in the population during a given round of decentralized learning for updating the global ML model (e.g., as indicated by Δ t ) and based on outdated updates (as they are received from one or more other computing devices in the population after a given round of decentralized learning for updating the global ML model) (e.g., as represented by Δ t d (represented by d (outdated)) to generate corresponding historical versions of the global ML model. Further, since outdated updates are received asynchronously, multiple corresponding historical checkpoints can be generated during a given round of decentralized learning for updating the global ML model. In other words, Δ tThe outdated updates may include all corresponding updates of the updated primary weights that were utilized during a given round of decentralized learning to generate an updated primary version of the global ML model and a given corresponding update received after the given round of decentralized learning, and the outdated updates received from the stragglers computing devices may be incorporated into the Δ t Therefore, Δ t Out-of-date may represent not only corresponding updates from fast computing devices in the swarm, but also one or more out-of-date updates from one or more stragglers computing devices in the swarm.

[0045] This allows remote systems (e.g. Figure 1 The historical global ML model engine 166 of the remote system 160 can be configured to generate a master weight w for the master version of the global ML model by causing t Based on the distribution of Δ after a given round of decentralized learning for updating the global ML model t The corresponding historical weight h for the corresponding historical version of the global ML model is updated by the corresponding update of the outdated representation to generate the corresponding historical version of the global ML model, as represented by t+1 The remote system may continue to generate and / or update the corresponding historical version of the global ML model (e.g., as indicated by the major version representing the update based on the global ML model and the outdated update from the given additional round of decentralized learning (e.g., Δ t+1 212 , representing additional corresponding historical versions of the global ML model generated by the ML model (e.g., Δ ), further updated major versions based on the global ML model and outdated updates from a given further additional round of decentralized learning (e.g., Δ t+2 Further additional corresponding historical versions of the global ML model generated (outdated) are indicated by a node 214 and an ellipsis after the node 214).

[0046] Thus, in subsequent rounds of decentralized learning (e.g., at least after generating corresponding historical versions of the global ML model), the remote system can send additional data (e.g., in addition to the current primary weights for the current primary version of the global ML model) to each computing device in the corresponding population. In some implementations, the remote system can send to each computing device in the population (1) the current primary weights for the current primary version of the global ML model and (2) one of the corresponding historical versions of the global ML model (e.g., as generated by Figure 2A In these implementations, the corresponding historical version of the global ML model can be used as the corresponding teacher model to generate one or more labels (e.g., as with respect to Figure 1In other words, in these implementations, the corresponding historical version of the global ML model can be used as the corresponding teacher model to distill the knowledge in the corresponding historical version of the global ML model into the corresponding update, thereby incorporating the knowledge in the stragglers into the primary model in a computationally efficient manner that is effective for further updating the current primary version of the primary model. One or more labels can be utilized to derive a distillation regularization term that can influence the corresponding update provided to the remote system for further updating the current primary version of the primary model.

[0047] It is worth noting that in some versions of those implementations, since multiple corresponding historical versions of the global ML model are accumulated through multiple rounds of decentralized learning for updating the global ML model, the corresponding historical version of the global ML model sent to each computing device in the group can be uniformly and randomly selected from the multiple corresponding historical versions of the global ML model. In other words, the computing devices in the corresponding group can utilize different corresponding historical versions of the global ML model when generating corresponding updates. In other versions of those implementations, the computing devices in the corresponding group can utilize the same corresponding historical version of the global ML model when generating corresponding updates. In addition, it is worth noting that in some versions of those implementations, since multiple corresponding historical versions of the global ML model are accumulated through multiple rounds of decentralized learning for updating the global ML model, older corresponding historical versions of the global ML model can be discarded or cleared from the storage of the remote system, so that N corresponding historical versions of a given ML model are maintained at any given time (e.g., where N is a positive integer greater than 1).

[0048] Thus, after decentralized learning for updating the global ML model, the most recently updated major version of the global ML model may be deployed as the global ML model with final weights w as indicated at node 216. T In various implementations, in response to determining that one or more deployment criteria are satisfied, the most recently updated major version of the global ML model can be deployed as the final version of the global ML model. The one or more deployment criteria can include, for example, a threshold number of rounds of decentralized learning for updating the global ML model have been performed, a threshold performance metric for achieving the most recently updated major version of the global ML model, and / or other criteria. Otherwise, the remote system can continue to perform additional rounds of decentralized learning for updating the global ML model.

[0049] Specific turn Figure 2B , similarly assuming that node 222 represents the primary weight w of the primary version of the global ML model t, the main weights will be updated through multiple rounds of decentralized learning for updating the global ML model. Further assume that the remote system (e.g., Figure 1 The remote system 160 in t starts a given round of decentralized learning for updating the global ML model by: updating the master weights w for the master version of the global ML model t to a group of computing devices to cause each of the computing devices to communicate via each of the computing devices (e.g., as described with respect to Figure 1 The computing device 1201 and the computing device 120 N The main weights w for the main version of the global ML model are used as described above. t To generate corresponding updates for the major version of the global ML model. Further, the remote system can asynchronously receive corresponding updates from one or more computing devices in the population during a given round of decentralized learning for updating the global ML model (e.g., also as described by Δ t It is worth noting that one or more computing devices in the population that provide corresponding updates during a given round of decentralized learning for updating the global ML model can be referred to as fast computing devices (e.g., so Figure 1 In addition, the remote system may cause the master weights w of the master version of the global ML model to t Based on corresponding updates received from one or more of the computing devices in the population during a given round of decentralized learning for updating the global ML model, as represented by an updated primary weight w representing an updated primary version for the global ML model. t+1 As indicated by node 224 .

[0050] Further assume that the remote system updates the master weights w for the updated major version of the global ML model by t+1 The additional group of additional computing devices is sent to cause each of the additional computing devices to communicate with each of the additional computing devices via each of the additional computing devices (e.g., as described with respect to Figure 1 The computing device 1201 and the computing device 120 N The updated primary weight w for the updated primary version of the global ML model is utilized at t+1 The remote system may generate additional corresponding updates for the updated major version of the global ML model to initiate a given additional round of decentralized learning for updating the global ML model. Further, the remote system may asynchronously receive additional corresponding updates from one or more additional computing devices in the additional population during the given additional round of decentralized learning for updating the global ML model (e.g., also as described by Δt+1 It is worth noting that one or more additional computing devices in the additional population that provide corresponding updates during a given additional round of decentralized learning for updating the global ML model may also be referred to as fast computing devices (e.g., thus Figure 1 In addition, the remote system may cause updated primary weights w for the updated primary version of the global ML model. t+1 The global ML model is further updated based on additional corresponding updates received from one or more of the additional computing devices in the additional population during a given additional round of decentralized learning for further updating the global ML model, as represented by a further updated primary weight w representing a further updated primary version for the global ML model. t+2 The remote system may continue to advance the major version of the global ML model (e.g., as indicated by the further updated major weight w representing the further updated major version of the global ML model). t+3 , and the ellipsis following the node 228).

[0051] However, further assume that during a given additional round of decentralized learning for updating the global ML model, the remote system asynchronously receives corresponding updates from the given additional round of decentralized learning for updating the global ML model from one or more other computing devices in the swarm (e.g., also as indicated by Δ t It is worth noting that one or more other computing devices in the population that provide corresponding updates after a given round of decentralized learning for updating the global ML model may also be referred to as stragglers (e.g., Figure 1 In addition, these corresponding updates are received after the remote system has advanced the major version of the global ML model to the updated major version of the global ML model. Therefore, these corresponding updates are generated by one or more other computing devices in the swarm based on the major version of the global ML model, and simply updating the updated major version of the global ML model would be ineffective (e.g., due to a weight mismatch between the major version of the global ML model and the updated major version of the global ML model).

[0052] Nonetheless, the remote system may be based on corresponding updates received from one or more computing devices in the population during a given round of decentralized learning for updating the global ML model (e.g., also as described by Δ t) and based on corresponding updates received from one or more other computing devices in the population after a given round of decentralized learning for updating the global ML model (e.g., also as indicated by Δ t In other words, Δ t Out-of-date may include all corresponding updates to the updated primary weights that were utilized during a given round of decentralized learning to generate an updated primary version of the global ML model and a given corresponding update received after the given round of decentralized learning. t Out-of-date may represent not only corresponding updates from fast computing devices in the group, but also corresponding updates from one or more stragglers in the group. This allows remote systems (e.g., via Figure 1 The historical global ML model engine 166 of the remote system 160 can be configured to generate a master weight w for the master version of the global ML model by causing t Based on the distribution of Δ after a given round of decentralized learning for updating the global ML model t The corresponding historical weight h for the corresponding historical version of the global ML model is updated by the corresponding update of the outdated representation to generate the corresponding historical version of the global ML model, as represented by t+1 As indicated by node 230 .

[0053] About Figure 2A Compared to the FARe-DUST technique described above, in which a remote system can generate and / or update a corresponding historical version of a global ML model when an outdated update is asynchronously received by the remote system, Figure 2B The remote system in the described MSG-based FeAST technique can wait for one or more termination criteria to be satisfied before generating a corresponding historical version of the global ML model. The one or more termination criteria can include, for example, receiving a threshold number of outdated updates from stragglers computing devices, a threshold duration of time having passed after a given round of decentralized learning for updating the global ML model ends, and / or other termination criteria. In other words, each of the major versions of the global ML model can be associated with a corresponding historical version of the global ML model that was generated based on outdated updates received from stragglers computing devices during subsequent rounds of decentralized learning. The remote system can continue to generate corresponding historical versions of the global ML model (e.g., as represented by major versions based on updates to the global ML model and outdated updates from a given additional round of decentralized learning (e.g., Δ t+1 232 for additional corresponding historical versions of the global ML model generated by the given further additional rounds of decentralized learning, and nodes 233 for further updated major versions based on the global ML model and outdated updates (e.g., Δt+2 Further additional corresponding historical versions of the global ML model generated (outdated) are indicated by a node 234 and an ellipsis following the node 234).

[0054] Furthermore, with regard to Figure 2A Compared to the FARe-DUST technique described above, which remotely utilizes the corresponding historical versions of the global ML model for distillation, Figure 2B The remote system in the described MSG-based FeAST technique can utilize the corresponding historical version of the global ML model to generate a corresponding auxiliary version of the global ML model (e.g., via Figure 1 The remote system 160 may be a secondary global ML model engine 168 of the remote system 160. For example, the remote system may generate the corresponding secondary versions of the global ML model as a weighted combination of the corresponding historical versions of the global ML model and the corresponding primary version of the global ML model. These secondary versions of the global ML model may be, for example, various exponential moving averages of the corresponding historical versions of the global ML model and the corresponding primary version of the global ML model.

[0055] For example, a remote system (e.g., via Figure 1 The auxiliary global ML model engine 168 of the remote system 160 can be used at step (t-K+1) according to a t-K +1 = β (a t-K -λ Δ t-K stale) + (1 – β) h t-K +1 Generate a corresponding auxiliary model. In this example, t is a positive integer corresponding to a given round of decentralized learning, tK is a positive integer corresponding to a previous round of decentralized learning before the given round of decentralized learning, β is an adjustable scaling factor (e.g., adjustable between 0 and 1 or another range of values) that controls the trade-off between the most recent corresponding historical version of the global ML model and the corresponding auxiliary version of the global ML model, and λ is an adjustable gradient mismatch factor (e.g., adjustable between 0 and 1 or another range of values) that controls the impact of mismatched gradients between the most recent corresponding historical version of the global ML model and the corresponding auxiliary version of the global ML model. In an implementation where λ is zero, the corresponding auxiliary version model will be an exponential moving average of the corresponding historical versions of the global ML model. Further, in an implementation where λ is one, the corresponding auxiliary version model will be an average of the most recent corresponding historical version of the global ML model and the most recent corresponding auxiliary version of the global ML model that incorporates outdated updates from the previous round of decentralized learning.

[0056] Therefore, in Figure 2B In the example of , node 236 represents the auxiliary weights a of the initial auxiliary version of the global ML model t+1In this example, since there is no previous auxiliary version of the global ML model, the remote system (e.g., via Figure 1 The auxiliary global ML model engine 168 of the remote system 160 can generate auxiliary weights a for the initial auxiliary version of the global ML model t+1， As in h t+1 The corresponding historical version of the global ML model at . However, Figure 2B In the example of generating the auxiliary weight a at node 238 t+2 The updated auxiliary version of the global ML model represented by and the auxiliary weights a at node 240 t+3 When further updated auxiliary versions of the global ML model are represented by the remote system, the remote system can generate these updated auxiliary versions of the global ML model based on the corresponding historical versions of the global ML model and the corresponding previous auxiliary versions of the global ML model.

[0057] It is worth noting that in various implementations, since multiple corresponding historical versions of the global ML model are accumulated through multiple rounds of decentralized learning for updating the global ML model, the corresponding historical versions of the global ML model can be discarded or cleared from the storage of the remote system because the corresponding historical versions of the global ML model are incorporated into the corresponding auxiliary version of the global ML model.

[0058] Thus, after decentralized learning for updating the global ML model, the most recently updated auxiliary version of the global ML model may be deployed as the global ML model with final weights a as indicated at node 242. T In various implementations, in response to determining that one or more deployment criteria are met, the most recently updated auxiliary version of the global ML model can be deployed as the final version of the global ML model (e.g., as described in relation to Figure 2A By generating the most recently updated secondary version of the global ML model in this manner, the remote system ensures that the secondary version of the global ML model does not diverge too far from the corresponding primary version of the global ML model, but that the corresponding secondary version of the global ML model gradually incorporates outdated updates received from stragglers computing devices.

[0059] although Figure 2A and Figure 2B Although described with respect to generating and / or updating a particular corresponding history model based on outdated updates received asynchronously during a particular round of decentralized learning, it should be understood that this is for the purpose of illustrating the techniques contemplated herein and is not intended to be limiting. Rather, it should be understood that outdated updates may be received during any subsequent round of decentralized learning and are dependent on computing devices in the population.

[0060] Furthermore, although Figure 2A and Figure 2B The description is about different versions of corresponding ML models stored in the storage of the remote system, but it should be understood that this is for simplicity and is not intended to be limiting. For example, when storing a given version of the global ML model (e.g., any of the primary version, the historical version, and / or the auxiliary version), the corresponding weights of these global ML models can be stored in the storage of the remote system, the optimization states of these global ML models can be stored in the storage of the remote system, and / or other parameters of these global ML models (e.g., momentum parameters) can be stored in the storage of the remote system.

[0061] Now turn to Figure 3 , depicts a block diagram illustrating various aspects of the present disclosure. Figure 3 The block diagram of illustrative embodiments includes a client device 310 having various on-device machine learning (ML) engines that utilize various ML models that can be trained in the manner described herein and is included as part of (or in communication with) an automated assistant client 315. For simplicity, other components of the client device 310 are not shown in FIG. Figure 3 Example in. Figure 3 One example of how automated assistant client 315 can utilize various on-device ML engines and corresponding ML models when performing various actions is illustrated.

[0062] Figure 3 The client device 310 in is illustrated as having one or more microphones 311 for generating audio data, one or more speakers 312 for rendering audio data, one or more visual components 313 for generating visual data, and a display 314 (e.g., a touch-sensitive display) for rendering visual data and / or for receiving various touch inputs and / or typed inputs. The client device 310 may further include pressure sensors, proximity sensors, accelerometers, magnetometers, and / or other sensors for generating other sensor data. The client device 310 selectively executes at least an automated assistant client 315. In Figure 3 In the example of , the automated assistant client 315 includes a hotword detection engine 322, a hotword-free invocation engine 324, a continuous conversation engine 326, an ASR engine 328, an object detection engine 330, an object classification engine 332, a voice recognition engine 334, and a facial recognition engine 336. The automated assistant client 315 further includes a voice capture engine 316 and a visual capture engine 318. It should be understood that Figure 3The ML engines and ML models depicted in are provided for example to illustrate various ML models that can be trained in the manner described herein, and are not intended to be limiting. For example, automated assistant client 315 may further include additional and / or alternative engines, such as a text-to-speech (TTS) engine and corresponding TTS models, a voice activity detection (VAD) engine and corresponding VAD models, an endpoint detector engine and corresponding endpoint detector models, a lip movement engine and corresponding lip movement models, and / or other engines and corresponding ML models. In addition, it should be understood that one or more of the engines and / or models described herein may be combined such that a single engine and / or model can perform the functionality of multiple engines and / or models described herein.

[0063] One or more cloud-based automated assistant components 370 may optionally be implemented on one or more computing systems (collectively, "cloud" computing systems) that are communicatively coupled to client device 310 via one or more networks generally indicated by 399. Cloud-based automated assistant components 370 may be implemented, for example, via a high-performance remote server cluster of high-performance remote servers. In various implementations, an instance of automated assistant client 315, through its interaction with one or more of cloud-based automated assistant components 370, may form what appears to be a logical instance of an automated assistant generally indicated by 395 from the user's perspective, with which the user may engage in human-computer interaction (e.g., verbal interaction, gesture-based interaction, typing-based interaction, and / or touch-based interaction). Figure 3 In the example of , one or more cloud-based automated assistant components 370 include cloud-based counterparts of the ML engine of the client device 310 described above, such as a hotword detection engine 372, a hotword-free invocation engine 374, a continuous conversation engine 376, an ASR engine 378, an object detection engine 380, an object classification engine 382, ​​a speech recognition engine 384, and a facial recognition engine 386. Likewise, it should be understood that Figure 3 The ML engines and ML models depicted in are provided for example purposes only to illustrate various ML models that can be trained in the manner described herein and are not intended to be limiting.

[0064] For example, the client device 310 may be: a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device of a user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), a stand-alone interactive speaker, a smart home appliance (such as a smart TV (or a standard TV equipped with a network-connected dongle with automated assistant capabilities)), and / or a user wearable device including a computing device (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual or augmented reality computing device). Additional and / or alternative client devices may be provided. Notably, the client device 310 may be a personal device of a given user (e.g., a given user of a mobile device), or shared among multiple users (e.g., a user's home, a user's office, etc.). In various implementations, the client device 310 may be an instance of a computing device (e.g., from a given round of decentralized learning for updating a given global ML model). Figure 1 An example of a fast computing device 1201 or from Figure 1 Slow computing device 120 N However, it should be understood that the population of client devices utilized in a given round of decentralized learning for updating a given global ML model is not limited to the client devices of the respective users, and may additionally or alternatively include other remote systems (e.g., Figure 1 other remote servers other than the remote system 160).

[0065] The one or more visual components 313 can take various forms, such as a monocular camera, a stereo camera, a LIDAR component (or other laser-based component), a radar component, etc. The one or more visual components 313 can be used, for example, by a visual capture engine 318 to capture visual data corresponding to visual frames (e.g., image frames, video frames, laser-based visual frames, etc.) of the environment in which the client device 310 is deployed. In some implementations, such visual frames can be utilized to determine whether a user is present in the vicinity of the client device 310 and / or the distance of a given user of the client device 310 relative to the client device 310. This determination of the presence of a user can be used, for example, in determining whether to activate a user interface. Figure 3 310 and / or other engines. Further, speech capture engine 316 can be configured to capture the user's spoken words and / or other audio data captured via one or more of microphones 311, and optionally in response to receiving a specific input to invoke automated assistant 395 (e.g., via actuation of a hardware or software button of client device 310, via a specific word or phrase, via a specific gesture, etc.).

[0066] As described herein, such audio data, visual data, textual data, and / or any other data generated locally at the client device 310 (collectively referred to herein as "client data") may be provided by Figure 3 3 to generate predicted outputs at client device 310 using corresponding ML models and / or at one or more cloud-based automated assistant components in cloud-based automated assistant components 370 using corresponding ML models. Notably, the predicted outputs generated using corresponding ML models may vary based on the client data (e.g., whether the client data is audio data, visual data, textual data, and / or other data) and / or the corresponding ML models utilized in processing the client data.

[0067] As a certain non-limiting example, the corresponding hotword detection engine 322, 372 can use the corresponding hotword detection model 322A, 372A to predict whether the audio data includes one or more specific words or phrases to invoke the automated assistant 395 (e.g., "Ok Assistant", "Hey Assistant", "What is the weather Assistant?", etc.) or one or more specific words or phrases to invoke certain functions of the automated assistant 395 (e.g., "Stop" to stop the alarm or music playback, etc.); the corresponding hotword-free invocation engine 324, 374 can use the corresponding hotword-free invocation model 324A, 374A to predict whether the non-audio data (e.g., visual data) includes a physical motion gesture or other signal (e.g., based on the user's gaze and optionally further based on the user's mouth movement) to invoke the automated assistant 395; the corresponding continuous conversation The corresponding ASR engines 326, 376 can utilize the corresponding continued conversation models 326A, 376A to predict whether further audio data is directed to the automated assistant 395 (e.g., or to an additional user in the environment of the client device 310); the corresponding ASR engines 328, 378 can utilize the corresponding ASR models 328A, 378A to generate recognized text in one or more languages, or predict phonemes and / or tokens corresponding to the audio data detected at the client device 310 and generate recognized text in one or more languages ​​based on the phonemes and / or tokens; the corresponding object detection engines 328, 378 can utilize the corresponding ASR models 328A, 378A to generate recognized text in one or more languages. The corresponding object detection models 330A, 380A may be used by the corresponding object detection models 330A, 380A to predict the location of objects included in the visual data captured at the client device 310; the corresponding object classification engines 332, 382 may be used by the corresponding object classification models 332A, 382A to predict the object classification of objects included in the visual data captured at the client device 310; the corresponding voice recognition engines 334, 384 may be used by the corresponding voice recognition models 334A, 384A to predict whether the audio data captures the spoken utterances of one or more known users of the client device 310 (e.g., via generating a speaker embedding or other representation that can be compared to corresponding actual embeddings of one or more known users of the client device 310); and the respective facial recognition engine 336, 386 can utilize the respective facial recognition model 336A, 386A to predict whether the visual data captures one or more known users of the client device 310 in the environment of the client device 310 (e.g., by generating a facial embedding or other representation that can be compared to corresponding facial embeddings of one or more known users of the client device 310).

[0068] In some implementations, the client device 310 and one or more of the cloud-based automated assistant components 370 may further include natural language understanding (NLU) engines 338, 388 and fulfillment engines 340, 390, respectively. The NLU engines 338, 388 may utilize corresponding NLU models 338A, 388A to perform natural language understanding and / or natural language processing on the recognized text, predicted phonemes, and / or predicted word units generated by the ASR engines 328, 378 to generate NLU data. For example, the NLU data may include, for example, the intent of a spoken utterance captured in audio data, and optionally slot values ​​for parameters of the intent. Further, the fulfillment engines 340, 390 may utilize corresponding fulfillment models or rules 340A, 390A and generate fulfillment data based on processing the NLU data. The fulfillment data may, for example, define a specific fulfillment in response to a user input (e.g., a spoken utterance, a typed input, a touch input, a gesture input, and / or any other user input) provided by a user of the client device 310. The specific performance may include causing automated assistant 395 to interact with a software application accessible at client device 310, causing automated assistant 395 to send a command to an Internet of Things (IoT) device (directly or via a corresponding remote system) based on the user input, and / or other parsing actions to be performed based on processing the user input. The performance data is then provided for local and / or remote performance / execution of the determined actions to cause the specific performance to be performed.

[0069] In other implementations, the NLU engines 338, 388 and the fulfillment engines 340, 390 may be omitted, and the ASR engines 328, 378 may directly generate fulfillment data based on user input. For example, assume that one or more of the ASR engines 328, 378 processes the spoken utterance "turn on the lights" using one or more of the corresponding ASR models 328A, 378A. In this example, one or more of the ASR engines 328, 378 may generate a semantic output that is then sent to a software application associated with the lights and / or sent directly to the lights and indicates that the lights should be turned on, without actively using one or more of the NLU engines 338, 388 and / or one or more of the fulfillment engines 340, 390 to process the spoken utterance.

[0070] It is worth noting that the one or more cloud-based automated assistant components 370 include the Figure 3The cloud-based counterparts of the engines and models described on the client device 310 of the embodiment of the present invention. However, in some implementations, these engines and models of one or more cloud-based automated assistant components 370 may not be utilized because these engines and models can be sent directly to the client device 310 and executed locally at the client device 310. In other implementations, these engines and models can be utilized only when the client device 310 detects any user input and sends the user input to one or more cloud-based automated assistant components 370. In various implementations, these engines and models executed at the client device 310 and one or more cloud-based automated assistant components 370 can be utilized in combination with each other in a distributed manner. In these implementations, a remote execution module can be optionally included to perform remote execution using one or more of these engines and models based on locally or remotely generated NLU data and / or fulfillment data. Additional and / or alternative remote engines can be included.

[0071] As described herein, in various implementations, on-device speech processing, on-device image processing, on-device NLU, on-device fulfillment, and / or on-device execution may be prioritized at least due to the latency and / or network usage reduction they provide when parsing spoken utterances (because no client-server round trip is required to parse spoken utterances). However, one or more of the cloud-based automated assistant components 370 may be at least selectively utilized. For example, such components may be utilized in parallel with on-device components, and outputs from such components may be utilized when local components fail. For example, if any of the on-device engines and / or models fail (e.g., due to relatively limited resources of the client device 310), the more robust resources of the cloud may be utilized.

[0072] Now turn to Figure 4 , depicts a flow chart illustrating an example method 400 for causing a primary version of a global machine learning (ML) model to be updated during a given round of decentralized learning of the global machine learning model but to receive outdated updates from stragglers computing devices after the given round of decentralized learning of the global ML model. For convenience, the operations of method 400 are described with reference to a system that performs the operations. The system of method 400 includes a computing device (e.g., Figure 1 Remote system 160, Figure 3 Cloud-based automated assistant component 370, Figure 6 The method 400 may be performed in a manner that is consistent with the method 400. The method 400 may be performed in a manner that is consistent with the method 400. The method 400 may be performed in a manner that is consistent with the method 400.

[0073] At block 452, the system determines whether a given round of decentralized learning for updating the global ML model has been initiated. If at an iteration of block 452, the system determines that a given round of decentralized learning for updating the global ML model has not been initiated, the system can continue to monitor for a given round of decentralized learning for updating the global ML model to be initiated at block 452. If at an iteration of block 452, the system determines that a given round of decentralized learning for updating the global ML model has been initiated, the system can proceed to block 454.

[0074] At block 454, the system sends the master weights for the master version of the global ML model to the population of computing devices (e.g., as described in relation to Figure 1 At block 456, the system causes each of the computing devices in the population to generate a corresponding update for the major version of the global ML model by utilizing the major version of the global ML model at each of the computing devices in the population (e.g., as described with respect to the decentralized learning engine 162, computing device identification engine 164, and ML model distribution engine 172). Figure 1 The computing device 1201 and the computing device 120 N At block 458, the system asynchronously receives a first subset of corresponding updates to the major version of the global ML model from one or more of the computing devices in the population (e.g., as described with respect to Figure 1 At block 460, the system causes the major version of the global ML model to be updated based on the first subset of corresponding updates to generate updated major weights for the updated major version of the global ML model (e.g., as described with respect to Figure 1 , as described in the global ML model update engine 170 of .

[0075] At block 462, the system determines whether a given round of decentralized learning for updating the global ML model has ended. The system may determine whether a given round of decentralized learning has ended based on receiving a threshold number of corresponding updates from computing devices in the population, based on a threshold duration of time having passed since the given round of decentralized learning was initiated, and / or other criteria. If at an iteration of block 462, the system determines that a given round of decentralized learning for updating the global ML model has not ended, the system may return to block 458 to continue to asynchronously receive a first subset of corresponding updates for the major version of the global ML model from one or more computing devices in the computing devices. If at an iteration of block 462, the system determines that a given round of decentralized learning for updating the global ML model has ended, the system may proceed to block 464.

[0076] In various implementations, the system can wait for a given round of decentralized learning for updating the global ML model to end before causing the major version of the global ML model to be updated to generate updated major weights for the major version of the global ML model. Further, in response to determining that the given round of decentralized learning for updating the global ML model has ended, the system can automatically initiate a given additional round of decentralized learning for updating the given ML model.

[0077] At block 464, the system asynchronously receives, from one or more of the other computing devices in the population, a second subset of corresponding updates for the primary version of the global ML model that were not received during a given round of decentralized learning for updating the global ML model. At block 466, the system determines which technique to implement to utilize the second subset of corresponding updates for the primary version of the global ML model that were not received during a given round of decentralized learning for updating the global ML model. If, at an iteration of block 466, the system determines to implement the “Federated Asynchronous Regularization with Distillation Using Outdated Teachers” (FARe-DUST) technique (e.g., as described with respect to Figure 2A described), the system can proceed to Figure 5A Block 552A of method 500A. Figure 5A Method 500A is described in more detail below. If at the iteration of block 466, the system determines to implement the "Federated Asynchronous Straggler Training Based on Mismatched and Stale Gradients" (MSG-based FeAST) technique (e.g., as described with respect to Figure 2B described), the system can proceed to Figure 5B Block 552B of method 500B. Figure 5B The method 500B is described in more detail below.

[0078] Now turn to Figure 5A , depicts a flow chart illustrating an example method 500A (e.g., “Federated Asynchronous Regularization with Distillation Using Outdated Teachers” (FARe-DUST)) for improving decentralized learning of a machine learning model using outdated updates received from stragglers computing devices. For convenience, the operations of method 500A are described with reference to a system that performs the operations. The system of method 500A includes a computing device (e.g., Figure 1 Remote system 160, Figure 3 Cloud-based automated assistant component 370, Figure 6 The method 500A may be performed in a manner that is consistent with the method 500A. ...

[0079] At block 552A, the system causes the primary version of the global ML model to be updated based on at least a first subset of the corresponding updates and based on a given corresponding update in a second subset of the corresponding updates to generate corresponding historical weights (e.g., as described with respect to the primary version of the global ML model) for the corresponding historical version of the global ML model. Figure 2A 554A). At block 554A, the system determines whether there are one or more additional corresponding updates to the major version of the global ML model. If, at an iteration of block 554A, the system determines that there are one or more additional corresponding updates to the major version of the global ML model, the system can return to block 552A to generate additional corresponding historical weights for additional corresponding historical versions of the global ML model. Further, if, at an iteration of block 554A, the system determines that there are one or more additional corresponding updates to the major version of the global ML model, the system can also proceed to block 556A. However, if, at an iteration of block 554A, the system determines that there are not one or more additional corresponding updates to the major version of the global ML model, the system can still proceed to block 556A without returning to block 552A until one or more additional corresponding updates are received.

[0080] At block 556A, the system determines whether a given additional round of decentralized learning for updating the global ML model has been initiated. If, at an iteration of block 556A, the system determines that a given additional round of decentralized learning for updating the global ML model has been initiated, the system can proceed to block 558A. Further, if, at an iteration of block 554A, the system determines that a given additional round of decentralized learning for updating the global ML model has been initiated, the system can also proceed to block 558A. However, if, at an iteration of block 556A, the system determines that a given additional round of decentralized learning for updating the global ML model has not been initiated, the system can return to block 554A without returning to block 554A.

[0081] In other words, when using the FARe-DUST technique, upon receiving a stragglers update from a stragglers computing device, the system can generate a corresponding historical version of the global ML model. Further, when using the FARe-DUST technique, the system can update a previously generated corresponding historical version of the global ML model. Notably, the system can perform the operations of these blocks as a background process to ensure that the major version of the global ML model is advanced while also generating and / or updating the corresponding historical version of the global ML model. Thus, there is no "no" branch for blocks 554A and 556A because the operations of these blocks can be performed as a background process while the system continues to perform the operations. Figure 5A Method 500A.

[0082] At block 558A, the system sends (eg, except for Figure 4 The additional group of additional computing devices (outside of the group of computing devices in block 454 of method 400 ) sends (1) updated master weights for the updated master version of the global ML model (e.g., from Figure 4 460 of method 400) and (2) a corresponding historical version of the global ML model (e.g., from Figure 5A In various implementations, assuming that multiple corresponding historical versions of the global ML model are available, the system can randomly and uniformly select a given one of the multiple corresponding historical versions of the global ML model to send to a given one of the computing devices. In other words, in these implementations, the system can cause a first corresponding historical version of the global ML model to be sent to a first computing device in the population, cause a second corresponding historical version of the global ML model to be sent to a second computing device in the population, and so on.

[0083] At block 560A, the system causes each of the additional computing devices in the additional population to generate an additional corresponding update for the updated master version of the global ML model. Notably, the corresponding historical version of the global ML model can be used as a corresponding teacher model at the respective computing device (e.g., according to the example of Figure 1 The teacher-student method described by the corresponding gradient engine and the corresponding learning engine) is used to generate additional corresponding updates.

[0084] At block 562A, the system asynchronously receives an additional first subset of additional corresponding updates for the updated major version of the global ML model from one or more additional computing devices in the additional population. At block 564A, the system causes the updated major version of the global ML model to be further updated based on the additional first subset of the additional corresponding updates to generate further updated primary weights for the further updated major version of the global ML model. In other words, the system can continue to advance the major version of the global ML model during a given additional round of decentralized learning based on the corresponding updates received from the one or more additional computing devices in the population.

[0085] At block 566A, the system determines whether a given additional round of decentralized learning for updating the global ML model has ended. The system may determine whether a given round of decentralized learning has ended based on receiving a threshold number of corresponding updates from computing devices in the population, based on a threshold duration of time having passed since the given round of decentralized learning was initiated, and / or other criteria. If, at an iteration of block 566A, the system determines that a given additional round of decentralized learning for updating the global ML model has not ended, the system may return to block 562A to continue to asynchronously receive a first subset of additional corresponding updates for the updated major version of the global ML model from one or more of the additional computing devices. If, at an iteration of block 566A, the system determines that a given additional round of decentralized learning for updating the global ML model has ended, the system may proceed to block 568A.

[0086] In various implementations, the system can wait for a given round of decentralized learning for updating the global ML model to end before causing the major version of the global ML model to be updated to generate updated major weights for the major version of the global ML model. Further, in response to determining that the given round of decentralized learning for updating the global ML model has ended, the system can automatically initiate a given additional round of decentralized learning for updating the given ML model.

[0087] At block 568A, the system asynchronously receives a second subset of additional corresponding updates for the updated major version of the global ML model from one or more other additional computing devices in the additional population, the second subset of additional corresponding updates not received during the given additional round of decentralized learning for updating the global ML model. The system may return to block 552A and perform Figure 5A Additional iterations of method 500A may be performed to continue generating and / or updating additional corresponding historical versions of the global ML model. Notably, for the sake of memory efficiency, the system may only maintain a limited number of corresponding historical versions of the global ML model. Thus, the system may discard or purge older corresponding historical versions of the global ML model because these older corresponding historical versions of the global ML model do not include as much knowledge as the more recently generated and / or updated corresponding historical versions of the global ML model.

[0088] Now turn to Figure 5B , depicts a flow chart illustrating an example method 500A (e.g., “Federated Asynchronous Straggler Training Based on Mismatched and Stale Gradients” (MSG-based FeAST)) for improving decentralized learning of a machine learning model using outdated updates received from a stragglers computing device. For convenience, the operations of method 500B are described with reference to a system that performs the operations. The system of method 500B includes a computing device (e.g., Figure 1 Remote system 160, Figure 3 Cloud-based automated assistant component 370, Figure 6 The method 500B may be performed in a manner that is consistent with the method 500B in terms of the order in which the operations are performed. ...

[0089] At block 552B, the system determines whether one or more termination criteria for generating corresponding historical weights for the corresponding historical version of the global ML model are met. The one or more termination criteria may include, for example, receiving a threshold number of outdated updates from the stragglers computing device, a threshold duration of time after a given round of decentralized learning for updating the global ML model ends, and / or other termination criteria. This enables the system to ensure that each of the major versions of the global ML model is associated with a corresponding historical version of the global ML model, which is generated based on outdated updates received from the stragglers computing device during subsequent rounds of decentralized learning. If at an iteration of block 552B, the system determines that one or more termination criteria are not met, the system may continue to monitor the satisfaction of one or more termination conditions at block 552B. During this period, the system may continue to receive outdated updates from the stragglers computing device. If at an iteration of block 552B, the system determines that one or more termination criteria are met, the system may proceed to block 554B.

[0090] At block 554B, the system performs the following operations based on (e.g., Figure 4 458 of method 400) and based on (e.g., in Figure 4 The method 400 of claim 400 may be used to cause the major version of the global ML model to be updated to generate corresponding historical weights (e.g., as described with respect to the historical version of the global ML model) for the corresponding historical version of the global ML model. Figure 2B At block 556B, the system generates a secondary version of the global ML model based on the updated primary version of the global ML model and based on the corresponding historical version of the global ML model (e.g., as described with respect to Figure 2B described).

[0091] At block 558B, the system determines whether a given additional round of decentralized learning for updating the global ML model has been initiated. If at an iteration of block 558B, the system determines that a given additional round of decentralized learning for updating the global ML model has not been initiated, the system may continue to monitor for the initiation of a given additional round of decentralized learning for updating the global ML model at block 558B. If at an iteration of block 558B, the system determines that a given additional round of decentralized learning for updating the global ML model has been initiated, the system may proceed to block 560B.

[0092] At block 560B, the system sends (eg, except for Figure 4 The additional group of additional computing devices (outside of the group of computing devices in block 454 of method 400 ) sends updated master weights for the updated master version of the global ML model (e.g., from Figure 4 460 of method 400). It is worth noting that Figure 5A Compared to method 500A, the system does not send any corresponding historical models to any computing devices in the additional population of computing devices.

[0093] At block 562B, the system causes each of the additional computing devices in the additional population to generate an additional corresponding update for the updated major version of the global ML model (e.g., as described with respect to Figure 1 At block 564B, the system asynchronously receives an additional first subset of additional corresponding updates for the updated major version of the global ML model from one or more of the additional computing devices in the additional population. At block 566B, the system causes the updated major version of the global ML model to be further updated based on the additional first subset of the additional corresponding updates to generate further updated major weights for the further updated major version of the global ML model. In other words, the system can continue to advance the major version of the global ML model during a given additional round of decentralized learning based on the corresponding updates received from one or more additional computing devices in the population.

[0094] At block 568B, the system determines whether the given additional round of decentralized learning for updating the global ML model has ended. The system may determine whether the given round of decentralized learning has ended based on receiving a threshold number of corresponding updates from computing devices in the population, based on a threshold duration of time having passed since the given round of decentralized learning was initiated, and / or other criteria. If at an iteration of block 568B, the system determines that the given additional round of decentralized learning for updating the global ML model has not ended, the system may return to block 564B to continue to asynchronously receive a first subset of additional corresponding updates for the updated major version of the global ML model from one or more of the additional computing devices. If at an iteration of block 568B, the system determines that the given additional round of decentralized learning for updating the global ML model has ended, the system may proceed to block 570B.

[0095] In various implementations, the system can wait for a given round of decentralized learning for updating the global ML model to end before causing the major version of the global ML model to be updated to generate updated major weights for the major version of the global ML model. Further, in response to determining that the given round of decentralized learning for updating the global ML model has ended, the system can automatically initiate a given additional round of decentralized learning for updating the given ML model.

[0096] At block 570B, the system asynchronously receives a second subset of additional corresponding updates for the updated major version of the global ML model from one or more other additional computing devices in the additional population, the second subset of additional corresponding updates not received during the given additional round of decentralized learning for updating the global ML model. The system may return to block 552B and perform Figure 5B Additional iterations of method 500B are performed to continue generating additional corresponding historical versions of the global ML model.

[0097] Now turn to Figure 6 , depicts a block diagram of an example computing device 610 that can optionally be utilized to perform one or more aspects of the techniques described herein. In some implementations, one or more of a client device, a cloud-based automated assistant component, and / or other components can include one or more components of the example computing device 610.

[0098] The computing device 610 typically includes at least one processor 614 that communicates with a number of peripheral devices via a bus subsystem 612. These peripheral devices may include a storage subsystem 624 (which includes, for example, a memory subsystem 625 and a file storage subsystem 626), a user interface output device 620, a user interface input device 622, and a network interface subsystem 616. The input and output devices allow for user interaction with the computing device 610. The network interface subsystem 616 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0099] The user interface input device 622 may include a keyboard, a pointing device (such as a mouse, a trackball, a touch pad or a graphics tablet, a scanner, a touch screen incorporated into a display), an audio input device (such as a voice recognition system, a microphone), and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and ways of inputting information into the computing device 610 or onto a communication network.

[0100] The user interface output device 620 may include a display subsystem, a printer, a fax machine, or a non-visual display, such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device (such as a liquid crystal display (LCD)), a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide a non-visual display such as via an audio output device. Generally speaking, the use of the term "output device" is intended to include all possible types of devices and methods for outputting information from the computing device 610 to a user or another machine or computing device.

[0101] The storage subsystem 624 stores programming and data constructs that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 624 may include selected aspects of the methods disclosed herein and data for implementing Figure 1 and Figure 3 The logic of the various components depicted.

[0102] These software modules are typically executed by the processor 614 alone or in combination with other processors. The memory 625 used in the storage subsystem 624 may include multiple memories, including a main random access memory (RAM) 630 for storing instructions and data during program execution and a read-only memory (ROM) 632 in which fixed instructions are stored. The file storage subsystem 626 may provide persistent storage for program and data files, and may include a hard drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement the functionality of certain implementations may be stored by the file storage subsystem 626 in the storage subsystem 624, or in other machines accessible to the processor 614.

[0103] The bus subsystem 612 provides a mechanism for the various components and subsystems of the computing device 610 to communicate with each other as intended. Although the bus subsystem 612 is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

[0104] The computing device 610 can be of different types, including a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the Figure 6 The description of computing device 610 depicted in FIG. 6 is intended only as a specific example for illustrating some implementations. Many other configurations of computing device 610 are possible, and these configurations are similar to those of FIG. Figure 6 The computing devices depicted in FIG. 1 may have more or fewer components than those depicted in FIG.

[0105] In situations where the systems described herein collect or otherwise monitor personal information about a user or may make use of the personal information and / or monitored information, the user may be given the opportunity to control whether a program or feature collects user information (e.g., information about the user's social network, social actions or activities, occupation, the user's preferences, or the user's current geographic location), or to control whether and / or how content that may be more relevant to the user is received from a content server. In addition, certain data may be processed in one or more ways before it is stored or used so that personally identifiable information is removed. For example, the user's identity may be processed so that the user's personally identifiable information cannot be determined, or where geographic location information is obtained, the user's geographic location may be generalized (such as to a city, zip code, or state level) so that the user's specific geographic location cannot be determined. Thus, the user may control how information is collected about the user and / or how it is used.

[0106] In some implementations, a method implemented by one or more processors of a remote system is provided, the method comprising: for a given round of decentralized learning for updating a global machine learning (ML) model: sending to a population of computing devices (i) master weights for a major version of the global ML model and (ii) a corresponding historical version of the global ML model; causing each of the computing devices in the population to generate a corresponding update for the major version of the global ML model via utilizing the major version of the global ML model at each of the computing devices in the population and utilizing the corresponding historical version of the global ML model as a corresponding teacher model at each of the computing devices in the population; asynchronously receiving a first subset of the corresponding updates for the major version of the global ML model from one or more of the computing devices in the population; and causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate updated master weights for the updated major version of the global ML model. The method also includes: after the given round of decentralized learning for updating the global ML model: asynchronously receiving a given corresponding update for the major version of the global ML model from one or more other computing devices in the population, the given corresponding update not being received during the given round of decentralized learning for updating the global ML model; causing corresponding historical weights for the corresponding historical version of the global ML model to be updated based on the given corresponding update to generate a corresponding updated historical version of the global ML model for utilization in one or more subsequent rounds of decentralized learning for further updating the global ML model; and in response to determining that one or more deployment criteria are met, causing the most recently updated major version of the global ML model to be deployed as the final version of the global ML model.

[0107] These and other implementations of the technology may include one or more of the following features.

[0108] In some implementations, the corresponding historical version of the global ML model may be one corresponding historical version of a plurality of corresponding historical versions of the global ML model, and sending the corresponding historical version of the global ML model to the group of computing devices may include selecting the corresponding historical version of the global ML model from among the plurality of corresponding historical versions of the global ML model to send to each computing device in the group.

[0109] In some versions of those implementations, the selection of the corresponding historical version of the global ML model from among the multiple corresponding historical versions of the global ML model to send to each of the computing devices in the population can be based on a uniform and random distribution of the multiple corresponding historical versions of the global ML model.

[0110] In additional or alternative versions of those implementations, a first computing device in the computing devices in the population may generate a first corresponding update in the corresponding updates via utilizing the primary version of the global ML model and via utilizing a first corresponding historical version of the global ML model in the plurality of corresponding historical versions of the global ML model. Further, a second computing device in the computing devices in the population may generate a second corresponding update in the corresponding updates via utilizing the primary version of the global ML model and via utilizing a second corresponding historical version of the global ML model in the plurality of corresponding historical versions of the global ML model.

[0111] In additional or alternative versions of those implementations, the method may further include: after causing the corresponding historical weights for the corresponding historical version of the global ML model to be updated to generate the corresponding updated historical version of the global ML model for utilization in one or more subsequent rounds of decentralized learning for further updating the global ML model: clearing the oldest corresponding historical version of the global ML model from the multiple corresponding historical versions of the global ML model.

[0112] In some implementations, causing a given computing device among the computing devices in the population to generate the corresponding update for the major version of the global ML model via utilizing the major version of the global ML model at the given computing device and via utilizing the corresponding historical version of the global ML model as the corresponding teacher model may include causing the given computing device to: process corresponding data obtained by the given computing device using the major version of the global ML model to generate one or more predicted outputs; process the corresponding data obtained by the given computing device using the corresponding historical version of the global ML model to determine a distillation regularization term; generate a given corresponding update for the major version of the global ML model based on at least the one or more predicted outputs and based on the distillation regularization term; and send the given corresponding update for the major version of the global ML model to the remote system.

[0113] In some versions of those implementations, the distillation regularization term may be determined based on one or more labels generated from processing the corresponding data obtained by the given computing device and using the corresponding historical version of the global ML model.

[0114] In some implementations, the primary weight for the primary version of the ML model may have been generated based on the immediately preceding round of decentralized learning for updating the global ML model, and the corresponding historical version of the global ML model may have been generated based on at least one further preceding round of decentralized learning for updating the global ML model, the at least one further preceding round of decentralized learning preceding the immediately preceding round of decentralized learning for updating the global ML model.

[0115] In some implementations, the method may further include: causing a previous corresponding history weight for a previous corresponding history version of the global ML model to be updated based on the given corresponding update to update the updated previous corresponding history version of the global ML model for use in one or more subsequent rounds of decentralized learning for further updating the global ML model, the previous corresponding history version of the global ML model being generated based on at least one further previous round of decentralized learning for updating the global ML model, the at least one further previous round of decentralized learning prior to the immediately previous round of decentralized learning for updating the global ML model.

[0116] In some implementations, the one or more deployment criteria may include one or more of: performing a threshold number of rounds of decentralized learning for updating the global ML model, or achieving a threshold performance metric of the most recently updated major version of the global ML model.

[0117] In some versions of those implementations, causing the most recently updated major version of the global ML model to be deployed as the final version of the global ML model may include: sending the most recently updated major weights for the most recently updated major version of the global ML model to a plurality of computing devices. Sending the most recently updated major weights for the most recently updated major version of the global ML model to a given computing device among the plurality of computing devices may cause the given computing device to: replace any previous weights for a previous version of the global ML model with the most recently updated major weights for the most recently updated major version of the global ML model; and utilize the most recently updated major version of the global ML model when processing corresponding data obtained at the given computing device.

[0118] In some implementations, the computing devices in the group may include client devices of the corresponding user group. In additional or alternative implementations, the computing devices in the group may additionally or alternatively include a remote server.

[0119] In some implementations, a method implemented by one or more processors of a remote system is provided, comprising: for a given round of decentralized learning for updating a global machine learning (ML) model: sending master weights for a major version of the global ML model to a population of computing devices; causing each of the computing devices in the population to generate corresponding updates for the major version of the global ML model via utilizing the major version of the global ML model at each of the computing devices in the population; asynchronously receiving a first subset of the corresponding updates for the major version of the global ML model from one or more of the computing devices in the population; and causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate updated master weights for the updated major version of the global ML model. The method further includes: after the given round of decentralized learning for updating the global ML model: asynchronously receiving a given corresponding update for the major version of the global ML model from one or more other computing devices in the population, the given corresponding update not received during the given round of decentralized learning for updating the global ML model; causing the major version of the global ML model to be updated based on the first subset of the corresponding updates and based on the given corresponding update to generate corresponding historical weights for a corresponding historical version of the global ML model; causing the corresponding historical version of the global ML model to be utilized in one or more subsequent rounds of decentralized learning for further updating the global ML model; and in response to determining that one or more deployment criteria are met, causing the most recently updated version of the global ML model to be deployed as the final version of the global ML model.

[0120] These and other implementations of the technology may include one or more of the following features.

[0121] In some implementations, the method may further include: for a given additional round of decentralized learning for updating the global ML model: sending to an additional population of additional computing devices (i) the updated primary weights for the updated primary version of the global ML model and (ii) the corresponding historical version of the global ML model; causing each of the additional computing devices in the additional population to generate additional corresponding updates for the updated primary version of the global ML model via utilizing the updated primary version of the global ML model at each of the additional computing devices in the additional population and utilizing the corresponding historical version of the global ML model as a corresponding teacher model at each of the additional computing devices in the additional population; asynchronously receiving an additional first subset of the additional corresponding updates for the updated primary version of the global ML model from one or more of the additional computing devices in the additional population; and causing the updated primary version of the global ML model to be updated based on the additional first subset of the additional corresponding updates to generate further updated primary weights for a further updated primary version of the global ML model. The method may further include: after the given additional round of decentralized learning for updating the global ML model: asynchronously receiving a given additional corresponding update for the updated primary version of the global ML model from one or more other additional computing devices in the additional population, the given additional corresponding update not being received during the given additional round of decentralized learning for updating the global ML model; causing the corresponding historical weights for the corresponding historical version of the global ML model to be updated based on the given additional corresponding update to generate a corresponding historical weight for use in the subsequent round for further updating the global ML model. the corresponding updated historical version of the global ML model utilized in one or more subsequent rounds of decentralized learning in the decentralized learning; causing the updated main version of the global ML model to be updated based on the given additional corresponding update to generate additional corresponding historical weights for the additional corresponding historical version of the global ML model utilized in one or more subsequent rounds of decentralized learning in the decentralized learning for further updating the global ML model; and in response to determining that one or more deployment criteria are met, causing the most recently updated version of the global ML model to be deployed as the final version of the global ML model.

[0122] In some implementations, a method implemented by one or more processors of a remote system is provided, comprising: for a given round of decentralized learning for updating a global machine learning (ML) model: sending master weights for a major version of the global ML model to a population of computing devices; causing each of the computing devices in the population to generate corresponding updates for the major version of the global ML model via utilizing the major version of the global ML model at each of the computing devices; asynchronously receiving a first subset of the corresponding updates for the major version of the global ML model from one or more of the computing devices in the population; and causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate updated master weights for the updated major version of the global ML model. The method further includes: after the given round of decentralized learning for updating the global ML model: asynchronously receiving a second subset of the corresponding updates for the primary version of the global ML model from one or more other computing devices in the population, the second subset of the corresponding updates not received during the given round of decentralized learning for updating the global ML model; causing the primary version of the global ML model to be updated based on the first subset of the corresponding updates and based on the second subset of the corresponding updates to generate historical weights for a historical version of the global ML model; generating a secondary version of the global ML model based on the updated primary version of the global ML model and based on the historical version of the global ML model; and in response to determining that one or more deployment criteria are met, causing the secondary version of the global ML model to be deployed as a final version of the global ML model.

[0123] These and other implementations of the technology may include one or more of the following features.

[0124] In some implementations, in response to determining that the one or more deployment criteria are not met, the method may further include: for a given additional round of decentralized learning for updating the global ML model after the given round of decentralized learning for updating the global ML model: sending the updated primary weights for the updated major version of the global ML model to an additional population of additional computing devices; causing each of the additional computing devices in the additional population to generate additional corresponding updates for the updated major version of the global ML model via utilizing the updated major version of the global ML model at each of the additional computing devices in the additional population; asynchronously receiving an additional first subset of the additional corresponding updates for the updated major version of the global ML model from one or more of the additional computing devices in the population; and causing the updated major version of the global ML model to be updated based on the additional first subset of the additional corresponding updates to generate further updated primary weights for further updated major versions of the global ML model. The method may further include: in response to determining that the one or more deployment criteria are not met and after the given additional round of decentralized learning for updating the global ML model: asynchronously receiving an additional second subset of the additional corresponding updates for the updated primary version of the global ML model from one or more other additional computing devices in the population, the additional second subset of the additional corresponding updates not received during the given additional round of decentralized learning for updating the global ML model; causing the updated primary version of the global ML model to be updated based on the additional first subset of the additional corresponding updates and based on the additional second subset of the additional corresponding updates to generate updated historical weights for the updated historical version of the global ML model; generating an updated secondary version of the global ML model based on the secondary version of the global ML model and based on the updated historical version of the global ML model; and in response to determining that the one or more deployment criteria are met, causing the updated secondary version of the global ML model to be deployed as the final version of the global ML model.

[0125] In some versions of those implementations, the one or more deployment criteria may include one or more of: performing a threshold number of rounds of decentralized learning for updating the global ML model, generating a threshold number of auxiliary versions of the global ML model, or achieving a threshold performance metric of the auxiliary version or the updated auxiliary version of the global ML model.

[0126] In some implementations, causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate the updated major weights for the updated major version of the global ML model may be responsive to determining that one or more update criteria are satisfied. In some versions of those implementations, the one or more update criteria may include one or more of: receiving a threshold number of the corresponding updates from the one or more computing devices in the population during the given round of decentralized learning for updating the global ML model, or a threshold duration of time elapsed before the given round of decentralized learning for updating the global ML model ends.

[0127] In some implementations, causing the major version of the global ML model to be updated based on the first subset of the corresponding updates and based on the second subset of the corresponding updates to generate the historical weights for the historical version of the global ML model can be in response to determining that one or more termination criteria are satisfied.

[0128] In some versions of those implementations, the one or more termination criteria may include one or more of: receiving a threshold number of the corresponding updates from the one or more other computing devices in the population, or a threshold duration of time after the given round of decentralized learning for updating the global ML model ends.

[0129] In some implementations, the method may further include: after generating the auxiliary version of the global ML model, discarding the historical version of the global ML model.

[0130] In some implementations, generating the secondary version of the global ML model can be based on a weighted combination of the updated primary version of the global ML model and the historical version of the global ML model.

[0131] In some versions of those implementations, the weighted combination of the updated primary version of the global ML model and the historical version of the global ML model may be weighted using one or more of: an adjustable scaling factor or an adjustable gradient mismatch factor.

[0132] In some implementations, causing the auxiliary version of the global ML model to be deployed as the final version of the global ML model may include: sending auxiliary weights for the auxiliary version of the global ML model to a plurality of computing devices. Sending the auxiliary weights for the auxiliary version of the global ML model to a given computing device among the plurality of computing devices may cause the given computing device to: replace any previous weights for a previous version of the global ML model with the auxiliary weights for the auxiliary version of the global ML model; and utilize the auxiliary version of the global ML model to process corresponding data obtained at the given computing device.

[0133] In some implementations, causing a given computing device among the computing devices in the population to generate the corresponding update for the major version of the global ML model via utilizing the major version of the global ML model at the given computing device may include causing the given computing device to: process corresponding data obtained by the given computing device using the major version of the global ML model to generate one or more predicted outputs; generate a given corresponding update for the major version of the global ML model based on at least the one or more predicted outputs; and send the given corresponding update for the major version of the global ML model to the remote system.

[0134] In some versions of those implementations, causing the given computing device to generate the given corresponding update for the primary version of the global ML model based on at least the one or more predicted outputs may include causing the given computing device to utilize one or more of: a supervised learning technique, a semi-supervised learning technique, or an unsupervised learning technique.

[0135] In some implementations, the corresponding update to the major version of the global ML model may include a corresponding gradient, and causing the major version of the global ML model to be updated to generate the updated major weights for the updated major version of the global ML model may include utilizing a gradient averaging technique.

[0136] In some implementations, the computing devices in the group include client devices of the corresponding user group. In additional or alternative versions of those implementations, the computing devices in the group may include a remote server.

[0137] Various implementations may include a non-transitory computer-readable storage medium storing instructions that may be executed by one or more processors (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), and / or a tensor processing unit (TPU)) to perform a method such as one or more of the methods described herein. Other implementations may include an automated assistant client device (e.g., a client device that includes at least an automated assistant interface for interfacing with a cloud-based automated assistant component), the automated assistant client device including a processor operable to execute stored instructions to perform a method such as one or more of the methods described herein. Still other implementations may include a system of one or more servers, the one or more servers including one or more processors operable to execute stored instructions to perform a method such as one or more of the methods described herein. Still other implementations may include a system of one or more client devices, the one or more client devices each including one or more processors operable to execute stored instructions to perform a method such as one or more of the methods described herein or to select aspects of one or more of the methods described herein.

Claims

1. A method implemented by one or more remote processors of a remote system, the method comprising: For a given round of decentralized learning used to update the global ML model: sending to a population of computing devices (i) master weights for a master version of the global ML model and (ii) corresponding historical versions of the global ML model; causing each of the computing devices in the population to generate a corresponding update for the primary version of the global ML model via utilizing the primary version of the global ML model at each of the computing devices in the population and via utilizing the corresponding historical version of the global ML model as a corresponding teacher model at each of the computing devices in the population; asynchronously receiving, from one or more of the computing devices in the population, a first subset of the corresponding updates to the major version of the global ML model; as well as causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate updated major weights for the updated major version of the global ML model; and After the given round of decentralized learning for updating the global ML model: asynchronously receiving, from one or more other computing devices in the population, a given corresponding update for the major version of the global ML model, the given corresponding update not received during the given round of decentralized learning for updating the global ML model; causing corresponding historical weights for the corresponding historical version of the global ML model to be updated based on the given corresponding update to generate a corresponding updated historical version of the global ML model for utilization in one or more subsequent rounds of decentralized learning for further updating the global ML model; and In response to determining that one or more deployment criteria are satisfied, causing a most recently updated major version of the global ML model to be deployed as a final version of the global ML model.

2. The method of claim 1 , wherein the corresponding historical version of the global ML model is one corresponding historical version of a plurality of corresponding historical versions of the global ML model, and wherein sending the corresponding historical version of the global ML model to the population of computing devices comprises: The corresponding historical version of the global ML model is selected from among the plurality of corresponding historical versions of the global ML model to send to each of the computing devices in the population.

3. The method of claim 2, wherein selecting the corresponding historical version of the global ML model from among the multiple corresponding historical versions of the global ML model to send to each of the computing devices in the population is based on a uniform and random distribution of the multiple corresponding historical versions of the global ML model.

4. The method of claim 2 or claim 3, wherein a first computing device among the computing devices in the group generates a first corresponding update in the corresponding updates by utilizing the main version of the global ML model and by utilizing a first corresponding historical version of the global ML model in the multiple corresponding historical versions of the global ML model, and wherein a second computing device among the computing devices in the group generates a second corresponding update in the corresponding updates by utilizing the main version of the global ML model and by utilizing a second corresponding historical version of the global ML model in the multiple corresponding historical versions of the global ML model.

5. The method according to any one of claims 2 to 4, further comprising: After causing the corresponding historical weights for the corresponding historical version of the global ML model to be updated to generate the corresponding updated historical version of the global ML model for utilization in one or more of the subsequent rounds of decentralized learning for further updating the global ML model: An oldest corresponding historical version of the global ML model is cleared from the plurality of corresponding historical versions of the global ML model.

6. The method of any preceding claim, wherein causing a given one of the computing devices in the population to generate the corresponding update for the primary version of the global ML model via utilizing the primary version of the global ML model at the given computing device and via utilizing the corresponding historical version of the global ML model as the corresponding teacher model comprises causing the given computing device to: processing corresponding data obtained by the given computing device using the primary version of the global ML model to generate one or more predicted outputs; processing the corresponding data obtained by the given computing device using the corresponding historical version of the global ML model to determine a distillation regularization term; generating a given corresponding update for the primary version of the global ML model based on at least the one or more predicted outputs and based on the distillation regularization term; and The given corresponding update for the major version of the global ML model is sent to the remote system.

7. The method of claim 6, wherein the distillation regularization term is determined based on one or more labels generated from processing the corresponding data obtained by the given computing device and using the corresponding historical version of the global ML model.

8. A method as claimed in any preceding claim, wherein the primary weights for the primary version of the ML model are generated based on the immediately preceding round of decentralized learning for updating the global ML model, and wherein the corresponding historical version of the global ML model is generated based on at least one further preceding round of decentralized learning for updating the global ML model, the at least one further preceding round of decentralized learning preceding the immediately preceding round of decentralized learning for updating the global ML model.

9. The method of any preceding claim, further comprising: causing a previous corresponding history weight for a previous corresponding history version of the global ML model to be updated based on the given corresponding update to update the previous corresponding updated history version of the global ML model for utilization in one or more subsequent rounds of decentralized learning for further updating the global ML model, the previous corresponding history version of the global ML model being generated based on at least one further previous round of decentralized learning for updating the global ML model, the at least one further previous round of decentralized learning prior to the immediately previous round of decentralized learning for updating the global ML model.

10. A method as claimed in any preceding claim, wherein the one or more deployment criteria comprise one or more of: performing a threshold number of rounds of decentralized learning for updating the global ML model, or achieving a threshold performance metric of the most recently updated major version of the global ML model.

11. The method of claim 10, wherein causing the most recently updated major version of the global ML model to be deployed as the final version of the global ML model comprises: sending a most recently updated primary weight for the most recently updated primary version of the global ML model to a plurality of computing devices, wherein sending the most recently updated primary weight for the most recently updated primary version of the global ML model to a given computing device among the plurality of computing devices causes the given computing device to: replacing any previous weights for a previous version of the global ML model with the most recently updated primary weights for the most recently updated primary version of the global ML model; as well as The most recently updated primary version of the global ML model is utilized to process corresponding data obtained at the given computing device.

12. A method as claimed in any preceding claim, wherein the computing devices in the community comprise client devices of a respective user community.

13. A method as claimed in any preceding claim, wherein the computing devices in the group comprise a remote server.

14. A method implemented by one or more processors of a remote system, the method comprising: For a given round of decentralized learning used to update the global ML model: sending master weights for a master version of the global ML model to a population of computing devices; causing each of the computing devices in the population to generate a corresponding update for the major version of the global ML model via utilizing the major version of the global ML model at each of the computing devices in the population; asynchronously receiving, from one or more of the computing devices in the population, a first subset of the corresponding updates to the major version of the global ML model; as well as causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate updated major weights for the updated major version of the global ML model; and After the given round of decentralized learning for updating the global ML model: asynchronously receiving, from one or more other computing devices in the population, a given corresponding update for the major version of the global ML model, the given corresponding update not received during the given round of decentralized learning for updating the global ML model; causing the primary version of the global ML model to be updated based on the first subset of the corresponding updates and based on the given corresponding update to generate corresponding historical weights for a corresponding historical version of the global ML model; causing the corresponding historical version of the global ML model to be utilized in one or more subsequent rounds of decentralized learning for further updating the global ML model; and In response to determining that one or more deployment criteria are satisfied, causing the most recently updated version of the global ML model to be deployed as a final version of the global ML model.

15. The method of claim 14, further comprising: For a given additional round of decentralized learning for updating the global ML model: sending (i) the updated primary weights for the updated primary version of the global ML model and (ii) the corresponding historical versions of the global ML model to an additional population of additional computing devices; causing each of the additional computing devices in the additional population to generate additional corresponding updates for the updated major version of the global ML model via utilizing the updated major version of the global ML model at each of the additional computing devices in the additional population and utilizing the corresponding historical version of the global ML model as a corresponding teacher model at each of the additional computing devices in the additional population; asynchronously receiving, from one or more of the additional computing devices in the additional population, an additional first subset of the additional corresponding updates to the updated major version of the global ML model; as well as causing the updated major version of the global ML model to be updated based on the additional first subset of the additional corresponding updates to generate further updated major weights for the further updated major version of the global ML model; After the given additional round of decentralized learning for updating the global ML model: asynchronously receiving, from one or more other additional computing devices in the additional population, a given additional corresponding update for the updated major version of the global ML model, the given additional corresponding update not received during the given additional round of decentralized learning for updating the global ML model; causing the corresponding historical weights for the corresponding historical version of the global ML model to be updated based on the given additional corresponding update to generate a corresponding updated historical version of the global ML model for utilization in one or more of the subsequent rounds of decentralized learning for further updating the global ML model; causing the updated primary version of the global ML model to be updated based on the given additional corresponding update to generate additional corresponding historical weights for the additional corresponding historical version of the global ML model for utilization in one or more subsequent rounds of decentralized learning for further updating the global ML model; as well as In response to determining that one or more deployment criteria are satisfied, causing the most recently updated version of the global ML model to be deployed as the final version of the global ML model.

16. A method implemented by one or more remote processors of a remote system, the method comprising: For a given round of decentralized learning used to update the global ML model: sending master weights for a master version of the global ML model to a population of computing devices; causing each of the computing devices in the population to generate a corresponding update for the major version of the global ML model via utilizing the major version of the global ML model at each of the computing devices; asynchronously receiving, from one or more of the computing devices in the population, a first subset of the corresponding updates to the major version of the global ML model; as well as causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate updated major weights for the updated major version of the global ML model; and After the given round of decentralized learning for updating the global ML model: asynchronously receiving, from one or more other computing devices in the population, a second subset of the corresponding updates for the major version of the global ML model, the second subset of corresponding updates not received during the given round of decentralized learning for updating the global ML model; causing the major version of the global ML model to be updated based on the first subset of the corresponding updates and based on the second subset of the corresponding updates to generate historical weights for historical versions of the global ML model; generating a secondary version of the global ML model based on the updated primary version of the global ML model and based on the historical version of the global ML model; and In response to determining that one or more deployment criteria are satisfied, causing the auxiliary version of the global ML model to be deployed as a final version of the global ML model.

17. The method of claim 16, in response to determining that the one or more deployment criteria are not met, further comprising: For a given additional round of decentralized learning for updating the global ML model after the given round of decentralized learning for updating the global ML model: sending the updated primary weights for the updated primary version of the global ML model to an additional population of additional computing devices; causing each of the additional computing devices in the additional population to generate an additional corresponding update for the updated major version of the global ML model via utilizing the updated major version of the global ML model at each of the additional computing devices in the additional population; asynchronously receiving, from one or more of the additional computing devices in the population, an additional first subset of the additional corresponding updates to the updated major version of the global ML model; as well as causing the updated major version of the global ML model to be updated based on the additional first subset of the additional corresponding updates to generate further updated major weights for the further updated major version of the global ML model; and After the given additional round of decentralized learning for updating the global ML model: asynchronously receiving, from one or more other additional computing devices in the population, an additional second subset of the additional corresponding updates for the updated major version of the global ML model, the additional second subset of the additional corresponding updates not received during the given additional round of decentralized learning for updating the global ML model; causing the updated major version of the global ML model to be updated based on the additional first subset of the additional corresponding updates and based on the additional second subset of the additional corresponding updates to generate updated historical weights for the updated historical version of the global ML model; generating an updated auxiliary version of the global ML model based on the auxiliary version of the global ML model and based on the updated historical version of the global ML model; as well as In response to determining that the one or more deployment criteria are satisfied, causing the updated secondary version of the global ML model to be deployed as the final version of the global ML model.

18. The method of claim 17, wherein the one or more deployment criteria include one or more of: performing a threshold number of rounds of decentralized learning for updating the global ML model, generating a threshold number of auxiliary versions of the global ML model, or achieving a threshold performance metric of the auxiliary version or the updated auxiliary version of the global ML model.

19. The method of any one of claims 16 to 18, wherein causing the major version of the global ML model to be updated based on the first subset of the corresponding updates to generate the updated major weights for the updated major version of the global ML model is in response to determining that one or more update criteria are satisfied.

20. The method of claim 19, wherein the one or more update criteria include one or more of: receiving a threshold number of the corresponding updates from the one or more of the computing devices in the population during the given round of decentralized learning for updating the global ML model, or a threshold duration of time elapsed before the given round of decentralized learning for updating the global ML model ends.

21. The method of any one of claims 16 to 20, wherein causing the primary version of the global ML model to be updated based on the first subset of the corresponding updates and based on the second subset of the corresponding updates to generate the historical weights for the historical version of the global ML model is in response to determining that one or more termination criteria are satisfied.

22. The method of claim 21, wherein the one or more termination criteria include one or more of: receiving a threshold number of the corresponding updates from the one or more other computing devices in the group, or a threshold duration of time after the end of the given round of decentralized learning for updating the global ML model.

23. The method of any one of claims 16 to 22, further comprising: After generating the auxiliary version of the global ML model, discarding the historical version of the global ML model.

24. The method of any one of claims 16 to 23, wherein generating the secondary version of the global ML model is based on a weighted combination of the updated primary version of the global ML model and the historical version of the global ML model.

25. The method of claim 24, wherein the weighted combination of the updated primary version of the global ML model and the historical version of the global ML model is weighted using one or more of: an adjustable scaling factor or an adjustable gradient mismatch factor.

26. The method of any one of claims 16 to 25, wherein causing the auxiliary version of the global ML model to be deployed as the final version of the global ML model comprises: sending auxiliary weights for the auxiliary version of the global ML model to a plurality of computing devices, wherein sending the auxiliary weights for the auxiliary version of the global ML model to a given computing device among the plurality of computing devices causes the given computing device to: replacing any previous weights for a previous version of the global ML model with the auxiliary weights for the auxiliary version of the global ML model; and The auxiliary version of the global ML model is utilized to process corresponding data obtained at the given computing device.

27. The method of any one of claims 16 to 26, wherein causing a given one of the computing devices in the population to generate the corresponding update for the major version of the global ML model via utilizing the major version of the global ML model at the given computing device comprises causing the given computing device to: processing corresponding data obtained by the given computing device using the primary version of the global ML model to generate one or more predicted outputs; generating a given corresponding update for the major version of the global ML model based on at least the one or more predicted outputs; and The given corresponding update for the major version of the global ML model is sent to the remote system.

28. The method of claim 27, wherein causing the given computing device to generate the given corresponding update for the primary version of the global ML model based on at least the one or more predicted outputs comprises causing the given computing device to utilize one or more of: a supervised learning technique, a semi-supervised learning technique, or an unsupervised learning technique.

29. The method of any one of claims 16 to 28, wherein the corresponding update to the major version of the global ML model comprises a corresponding gradient, and wherein causing the major version of the global ML model to be updated to generate the updated major weights for the updated major version of the global ML model comprises utilizing a gradient averaging technique.

30. The method of any one of claims 16 to 29, wherein the computing devices in the group include client devices of a corresponding user group.

31. A method as claimed in any one of claims 16 to 30, wherein the computing devices in the group include a remote server.

32. A system comprising: one or more processors; as well as A memory storing instructions which, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 31.

33. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the operations of the method according to any one of claims 1 to 31.

Images