Method and system for improved efficiency of federated learning for machine learning models

Abstract

Implementations disclosed herein relate to efficient federated learning of machine learning (ML) models at a remote system (e.g., a remote server) based on updates generated at client devices. A processor of a client device can receive client data, process the client data using an on-device ML model to generate a predicted output, generate a gradient based on the predicted output using unsupervised learning, generate an update to a distinct portion of the on-device ML model and / or a global ML model that is a remotely-based counterpart to the on-device ML model based on the gradient. Further, a processor of the remote system can receive the update to the distinct portion of the on-device ML model from the client device and cause the global ML model to be updated based on the update to the distinct portion of the on-device ML model received from the distinct client device. Accordingly, resources consumed at the client devices and / or network resources can be reduced.

Description

Background Technology

[0001] Federated learning of machine learning (ML) models is an increasingly popular ML technique for training ML models. In traditional federated learning, the on-device ML model is stored locally on the user's client device, while the global ML model, as a remote-based counterpart of the on-device ML model, is stored remotely on a remote system (e.g., a remote server or cluster of remote servers). The client device, using the on-device ML model, can process user input detected at the client device to generate predicted outputs and can generate gradients based on the predicted outputs in a supervised or unsupervised manner. Furthermore, the client device can transmit gradients to the remote system. The remote system can update the weights of the global ML model using the gradients and optionally additional gradients generated similarly at the client device or additional client devices. The remote system can transmit the global ML model or its updated weights to the client device. The client device can then update the on-device ML model by replacing it with the updated global ML model or by replacing its weights with the updated weights of the updated global ML model.

[0002] However, federated learning of ML models can lead to unnecessary consumption of significant resources on the client device, and / or the client device may not have sufficient resources to locally store and train a full-scale ML model. For example, suppose an on-device Automatic Speech Recognition (ASR) model is used to generate a transcript that is predicted to correspond to spoken utterance captured in an audio data stream via the client device's microphone. In this example, the predicted output may correspond to the transcript or a portion thereof, and gradients may be generated based on the transcript. When generating gradients, on-device memory can be consumed by storing the activations at each layer of the ASR model on the device, and on-device computational resources can be consumed by computing the error function at each layer of the ASR model on the device and backpropagating the error function across each layer of the ASR model on the device. Therefore, techniques are needed to improve the efficiency of federated learning without unnecessarily consuming on-device memory and computational resources. Summary of the Invention

[0003] The embodiments disclosed herein relate to federated learning based on the efficiency of a machine learning (ML) model on a remote system (e.g., a remote server or cluster of remote servers) generated at a client device. A processor on a given client device can receive client data, process the client data using an ML model on the given device to generate a predicted output, generate gradients based on the predicted output using supervised or unsupervised learning techniques, generate a given update for a given dissimilar portion of the ML model on the given device based on the gradients, and transmit the given update to the remote system. Furthermore, a processor on the remote system can receive the given update from the given client device and cause a given global ML model (e.g., a remote-based counterpart of the ML model on the given device) to be updated based on the given update, thereby updating a given dissimilar portion of the given global ML model corresponding to a given dissimilar portion of the ML model on the given device, without causing any other portions of the given ML model to be updated based on the given update. Notably, this process can be repeated at the given client device (or in parallel at one or more additional client devices) to generate additional given updates for other dissimilar portions of the given global ML model. Therefore, the computational resources consumed at the client devices participating in federated learning of the ML model can be reduced, since the client devices do not generate full updates for the full-size ML model, and the network resources consumed when transmitting updates to remote systems can be reduced, since fewer full updates of the full-size ML model are transmitted to remote systems.

[0004] For example, suppose the audio data capturing a human user's spoken words, "Hey Assistant, turn on the kitchen lights," is generated via the microphone of a given client device of the human user. In this example, the audio data capturing the spoken words can correspond to client data. Furthermore, an on-device automatic speech recognition (ASR) model stored in the on-device memory of the given client device can be used to process the audio data to generate ASR data, such as one or more speech hypotheses predicted to correspond to the spoken words captured in the audio data. In this example, one or more speech hypotheses can correspond to the predicted output. Additionally, gradients can be generated based on processing the audio data and / or one or more speech hypotheses using one or more semi-supervised or self-supervised learning techniques described herein. It is noteworthy that the on-device ASR model can include multiple on-device ASR layers, such as one or more first on-device ASR layers, one or more second on-device ASR layers, one or more third on-device ASR layers, and so on. However, gradients can be generated only for dissimilar portions of the on-device ASR model (such as one or more first on-device ASR layers) without generating gradients for any other on-device ASR layers. By generating gradients only for one or more ASR layers on the first device, the consumption of on-device memory and computational resources at a given client device can be reduced, because the given client device does not need to store activation functions for each ASR layer of the ASR model on the device to generate gradients, and because the given client device does not need to compare activation functions with loss functions for each ASR layer of the ASR model on the device as in traditional federated learning.

[0005] Continuing the example above, suppose a given client device generates gradients for one or more ASR layers on a first device. The given client device may also generate a first update for one or more ASR layers on the first device based on the gradients. In this example, the first update may include, for example, an indication of one or more first on-device ASR layers of the on-device ASR model associated with the first update, an indication of one or more first corresponding global ASR layers of the corresponding global ASR model to be updated based on the first update, an indication of the on-device ASR model for generating the first update for one or more first on-device ASR layers of the on-device ASR model, a gradient generated based on the prediction output, one or more updated first on-device ASR layers of the on-device ASR model updated locally at the client device based on the gradients (e.g., in some implementations, where the on-device ASR is actually updated locally at the given client device based on the gradients), and / or one or more updated on-device weights of the one or more updated first on-device ASR layers of the on-device ASR model (e.g., in some implementations, where the on-device ASR is actually updated locally at the given client device based on the gradients).

[0006] In this example, a given client device may transmit a first update to a remote system. In response to receiving the first update, the remote system may cause one or more first global ASR layers of the global ASR model to be updated based on the first update, resulting in an updated global ASR model. In some cases, the remote system may optionally combine the first update received from the given client device with additional updates received from the given client device or one or more additional client devices. Furthermore, the remote system may transmit the updated global ASR model (e.g., the updated global ASR model itself, one or more of its updated global ASR layers, and / or one or more of its updated global weights) back to the client device and / or one or more of the additional client devices. Each client device may replace its on-device ASR model in its on-device memory with the updated global ASR model and continue generating additional updates for the updated global ASR model, transmitting these additional updates to the remote system to continue updating the updated global ASR model.

[0007] While the above examples describe client-side data as audio data capturing a user's spoken words, it should be understood that this is for illustrative purposes and not limiting. For example, client-side data may additionally or alternatively include visual data, text data, touch data, and / or any other type of client-side data generated at a given client device. In these and other instances, the on-device ML model used to process the client-side data can be a vision-based ML model, a text-based ML model, and / or a touch-based ML model. Furthermore, although the above examples describe ASR models for processing audio data, it should be understood that this is for illustrative purposes and not limiting. For example, other audio-based ML models may additionally or alternatively be used to process client-side data and subsequently updated based on the processing of the client-side data. In these and other instances, audio-based ML models may include, for example, hot word detection models trained to predict whether audio data captures specific words or phrases, voice activity detection (VAD) models trained to predict whether human speech activity is captured in streaming audio data, and / or any other audio-based ML models.

[0008] In some implementations, multiple on-device ML layers can be compressed before gradient generation, such that the multiple on-device ML layers are implemented as a single on-device ML layer at a given client device. For example, a given client device can compress a first on-device ML layer and a second on-device ML layer into one or more first on-device ML layers based on scheduling signals generated by the on-device scheduling engine of the given client device and / or scheduling signals received from a remote scheduling engine. The scheduling signals may include an indication of the on-device ML layers to be compressed before gradient generation. In some versions of these examples, the first and second on-device ML layers may be compressed into one or more on-device ML layers before generating the prediction output and before generating the gradient, while in additional or alternative examples, the first and second on-device ML layers may be compressed into one or more first on-device ML layers after generating the prediction output but before generating the gradient. It is worth noting that neither the first nor the second on-device ML layers are discarded. Instead, compressing the first and second on-device ML layers into one or more first ML layers imposes a constraint on the on-device ML model.

[0009] Furthermore, with the addition of a client device, and before generating additional gradients (e.g., based on additional client data received at the additional client device), multiple on-device ML layers can be compressed, such that the multiple on-device ML layers are implemented as a single on-device ML layer at the additional client device. Continuing the example above, at least a second and a third on-device ML layer of the on-device ML model are compressed into one or more second ML layers at the additional client device. The additional client device can compress the second and third on-device ML layers of the on-device ML model into one or more second ML layers in the same or similar manner as described above, but based on scheduling signals generated by the on-device scheduling engine of the additional client device and / or received from the remote scheduling engine. It is worth noting that in this example, the second on-device ML layer is compressed into both one or more first on-device ML layers at the client device and one or more second on-device ML layers at the additional client device. However, each of the one or more first on-device ML layers and the one or more second on-device ML layers includes at least one unique on-device ML layer—a first on-device ML layer for the one or more first on-device ML layers and one or more third on-device ML layers for the second on-device ML layers. Nevertheless, it should be understood that this is for illustrative purposes only and not for limitation. For example, identical on-device ML layers can be compressed at dissimilar client devices (e.g., a first on-device ML layer and a second on-device ML layer are compressed into one or more first ML layers at both the client device and the additional client device), dissimilar ML layers can be compressed at dissimilar client devices such that at least one unique ML layer is compressed at the dissimilar devices, and / or dissimilar ML layers can be compressed at dissimilar client devices such that only unique ML layers are compressed at different devices. Although these dissimilar ML layers may differ from each other, they are adjacent in the on-device ML model and may include some overlap. Therefore, by generating first and second updates as shared updates, remote systems can leverage this overlap to save computational and / or network resources, resulting in improved efficiency for federated learning.

[0010] In some implementations, the remote scheduling engine can divide a superset of client devices into multiple subsets. In some versions of those implementations, each subset of client devices can iteratively perform gradient snapshot techniques when updating a given global ML model. For example, the remote system can cause a given client device to generate updates for one or more on-device ML layers of the ML model on the given device based on processing client data received at the client device, where N is a positive integer. Furthermore, the remote system can cause additional client devices to generate additional updates for one or more on-device ML layers of the ML model on the given device based on processing additional client data received at the additional client device. In other words, the system can employ gradient snapshot techniques to improve the efficiency of federated learning of the ML model. Gradient snapshot techniques can be implemented based on corresponding scheduling signals generated by the corresponding on-device scheduling engines of the client devices and / or additional client devices and / or corresponding scheduling signals received from the remote scheduling engine. The corresponding scheduling signal can instruct a client device to generate a first update for one or more ML layers on the Nth device of the ML model on the device, instruct an additional client device to generate a second update for one or more ML layers on the (N-1)th device of the ML model on the device, instruct yet another additional client device to generate a third update for one or more ML layers on the (N-2)th device of the ML model on the device, and so on. Therefore, by utilizing gradient snapshot technology, the remote system can ensure the generation of updates for each of the ML layers on the Nth device, as described in this paper.

[0011] In some versions of those implementations, the remote scheduling engine can divide a superset of client devices into multiple subsets. In some versions of those implementations, each subset of client devices may implement an iterative gradient snapshot technique. For example, a first subset of client devices may include a first client device generating updates for one or more ML layers on an Nth device, a second client device generating updates for one or more ML layers on an (N-1)th device, a third client device generating updates for one or more ML layers on an (N-2)th device, and so on for each of the ML layers on the Nth device. Furthermore, a second subset of client devices may generate updates for each of the ML layers on the Nth device in the same or similar manner. Therefore, when updates are sent to the remote system, it can be ensured that the remote system has updates for each of the disparate parts of the ML model that are available for updating.

[0012] In some implementations, unsupervised learning for gradients can correspond to semi-supervised learning techniques, such as teacher-student methods. For example, a predicted output can be generated based on processing client data using an on-device ML model, as described in the examples above. Furthermore, a benchmark output can be generated based on processing client data using a benchmark ML model. In this example, the benchmark ML model can be of the same type as both the on-device ML model and the global ML model, and the benchmark output can be used as a supervisory signal for generating gradients. For example, the benchmark ML model can correspond to a benchmark ASR model used to generate one or more benchmark speech hypotheses based on processing audio data. Furthermore, the on-device ASR model can generate one or more speech hypotheses based on processing audio, as described in the examples above. In this case, one or more benchmark speech hypotheses can be compared with one or more speech hypotheses to generate gradients for one or more first on-device ASR layers.

[0013] In some versions of those implementations, the predicted output may be generated using only the on-device ASR model and / or may be updated using gradients only in response to determining that one or more conditions are met. One or more conditions may include, for example, whether the predicted output meets a predicted output threshold, whether the baseline output meets a baseline output threshold, and / or other conditions. In other words, the predicted output may be generated using only the on-device ASR model, and / or the gradients may be updated using gradients only in response to determining that the baseline output provides sufficient supervision.

[0014] In additional or alternative implementations, the unsupervised learning used to generate gradients can correspond to self-supervised learning techniques. In self-supervised learning, a target portion of the client data can be identified. The target portion of the client data can be after the prepended portion of the client data and before the appended portion of the client data. Furthermore, various masking techniques can be used to mask the target portion of the client data. The target portion of the client data can be arbitrarily selected or can be selected based on one or more criteria—such as a specific segment between n and m seconds of audio data corresponding to the target portion, a visual data portion corresponding to the center portion of an image corresponding to the target portion, the center portion of text data corresponding to the target portion—and / or any other criteria used to select the target portion of the client data. In some implementations where the client data corresponds to audio data, the target portion of the client data can correspond to a target audio waveform portion of the corresponding audio data stream, the prepended portion of the client data can correspond to a prepended audio waveform portion received before the target audio waveform portion, and the appended portion of the client data can correspond to a appended audio waveform portion received after the target audio waveform portion. In additional or alternative implementations where client data corresponds to audio data, the client data may correspond to a representation of the audio, such as a vector representation of the audio data and / or other representations of the audio data. In those implementations, the target portion of the client data may correspond to a target representation portion of the audio data, the preceding portion of the client data may correspond to a preceding representation portion that precedes the target representation portion, and the following portion of the client data may correspond to a following representation portion that follows the target representation portion. In additional or alternative implementations, the client data may correspond to text data or visual data. In those implementations, the target portion, the preceding portion, and the following portion may be identified in the same or similar manner, and the target portion may be masked.

[0015] In embodiments where unsupervised learning for gradients corresponds to the self-supervised learning techniques described above, an on-device ML model can be used to process the preceding and following portions of the client data to generate a predicted output that predicts the target portion of the client data. For example, in an embodiment where the target portion of the client data corresponds to a target audio waveform portion of a corresponding audio data stream, it is further assumed that the audio data captures spoken utterance. In this example, an on-device ASR model can be used to process the preceding and following audio waveform portions to generate a predicted output. In this example, the predicted output may correspond to a predicted target audio waveform and / or predicted audio features predicted to correspond to the target audio waveform portion and / or the audio features of the target audio waveform portion. In other words, the self-supervised learning technique can attempt to reconstruct the target audio waveform portion and / or its audio features based on processing the preceding and following audio waveform portions. Furthermore, for example, in an embodiment where the target portion of the client data corresponds to a target representation portion of a corresponding audio data stream, the preceding and following portions can be processed in the same or similar manner to predict the target portion. Additionally, the predicted output can be compared with the masked target portion to generate a gradient. In other words, the actual target portion of the masked client data can be used as a supervisory signal for comparison with the predicted output. Furthermore, based on this comparison, gradients can be generated and used to generate updates.

[0016] In various implementations, gradients can be generated additionally or alternatively using supervised learning. For example, again suppose that audio data capturing the spoken words of a human user, “Hey Assistant, turn on the kitchen lights,” is generated via a microphone on a given client device of the human user. In this example, the audio data capturing the spoken words may correspond to client data. Furthermore, the audio data can be processed using an on-device ASR model stored in the on-device memory of the given client device to generate ASR data, such as one or more speech hypotheses predicted to correspond to the spoken words captured in the audio data. In this example, one or more speech hypotheses may correspond to a predicted output, and a given speech hypothesis can be selected as corresponding to the spoken words and provided to the user as a transcription via the display of the given client device. Furthermore, in this example, it is further assumed that the user corrects one or more terms of the transcription. The corrected one or more terms of the transcription can be used as a supervisory signal in generating gradients for one or more of the first on-device ASR layers. For example, the given client device can compare one or more terms of the transcription with one or more corrected terms of the transcription to generate gradients for one or more of the first on-device ASR layers.

[0017] Various technical advantages can be achieved by using the techniques described herein. As a non-limiting example, by generating updates based on gradients specific to on-device parts of the ML model, the consumption of on-device memory and computational resources at client devices participating in the federated learning of the ML model can be reduced. For example, client devices store less data when generating gradients and perform less processing when generating gradients by generating only gradients specific to on-device parts of the ML model. Furthermore, network resource consumption can be reduced because updates transmitted to the remote system include less data because they are generated based on these on-device ML model-specific gradients. As another non-limiting example, the remote system can schedule these updates at different client devices so that the remote system can receive all updates for the global ML model, even if no single client device generates updates for the entire global ML model. Therefore, the global ML model (and—and the on-device ML model when transmitting the global ML model back to the client device) has higher accuracy and / or recall, while enabling each client device participating in the federated learning of the ML model to consume fewer on-device resources.

[0018] The above description serves as an overview of some embodiments of this disclosure. Further descriptions of those embodiments and other embodiments follow in more detail. Attached Figure Description

[0019] Figure 1 Example process flows illustrating various aspects of this disclosure according to various embodiments are depicted.

[0020] Figure 2 A block diagram depicts an example environment in which the implementations disclosed herein can be carried out.

[0021] Figure 3 A flowchart illustrating example methods for client-side operations of federated learning, based on various implementations, demonstrates the efficiency of machine learning models.

[0022] Figure 4 A flowchart illustrating example methods of federated learning techniques for achieving the efficiency of machine learning models, according to various implementations, is provided.

[0023] Figure 5 A flowchart illustrating another example method of federated learning techniques for achieving the efficiency of machine learning models, according to various implementations, is presented.

[0024] Figure 6 A flowchart illustrating another example method of federated learning techniques for achieving the efficiency of machine learning models, according to various implementations, is presented.

[0025] Figure 7 A flowchart illustrating example methods for remote system-side operations of federated learning, according to various implementations, demonstrates the efficiency of machine learning models.

[0026] Figure 8 Example architectures of computing devices according to various implementation methods are depicted. Detailed Implementation

[0027] Figure 1 An example process flow illustrating various aspects of this disclosure is described. Client device 150 is illustrated in... Figure 1 In, and at least included in, the representation of client device 150 Figure 1 Components within the frame. Client device 150 may receive client data 101 via various components and / or sensors. For example, client data 101 received at client device 150 may include spoken words captured in audio data generated via one or more microphones of client device 150, typed input captured in text data generated via a touch-sensitive display of client device 150 or a peripheral device (e.g., a keyboard integrated with client device 150 or an external keyboard) communicating with client device 150, touch input captured based on touch data generated via a touch-sensitive display of client device 150 or one or more buttons of client device 150 (e.g., one or more hardware buttons and / or software buttons), gesture input captured in visual data generated via one or more vision components of client device 150, and / or any other type of client data 101 that may be received by client device 150 via other components and / or sensors.

[0028] Furthermore, client device 150 may include various on-device machine learning (ML) models stored in on-device memory of client device 150 (such as in a database 154A of on-device ML models). In some implementations, and based on processing client data 101, on-device ML models may be utilized to enable specific performance of client data 101 to be performed (e.g., as per the context of...). Figure 2 (More detailed description). In additional or alternative implementations, and based on processing client data 101, the on-device ML model can be locally updated at client device 150 and / or the on-device ML model can be used at client device 150 to generate one or more updates (e.g., as per the database 154B of the global ML model) for a corresponding global ML model remotely stored in a remote memory of remote system 160 (e.g., a remote server or a cluster of remote servers) at client device 150. Figure 1 , 3(Described in more detail in points 4, 5, 6, and 7). A remote system includes at least the components represented by remote system 160. Figure 1 The components within the box. Client device 150 and remote system 160 are communicatively coupled via one or more networks (e.g., via one or more local area networks (LANs) and / or wide area networks (WANs)) to transmit and receive data between them.

[0029] As described herein, the implementation relates to improved efficiency of federated learning of ML models. Similar to conventional federated learning, the on-device ML engine 132 can use one or more on-device ML models stored in a database 154A of on-device ML models to process client data 101 to generate a predicted output 102. The one or more on-device ML models utilized by the on-device ML engine 132 may depend on the type of client data 101 received at the client device 150. For example, suppose the client data 101 corresponds to audio data generated via one or more microphones of the client device 150. In this example, the on-device ML model 132 may utilize one or more audio-based ML models to process the client data 101 when generating the predicted output 102, such as automatic speech recognition (ASR) models, hot word detection models, continuous conversation models, speech recognition models, voice activity detection (VAD) models, endpoint detection models, and / or other audio-based ML models. Furthermore, the predicted output 102 in this example (and / or its associated predicted values, such as probability, log-likelihood, binary value, and / or any other predicted value) can be based on one or more audio-based ML models used to process the client data 101, such as those related to... Figure 2 The predictions are described by one or more audio-based ML models. Additionally, for example, it is additionally or alternatively assumed that client data 101 corresponds to visual data generated via one or more visual components of client device 150. In this example, on-device ML model 132 may utilize one or more vision-based ML models to process client data 101 when generating prediction output 102. These vision-based ML models include object detection models, object classification models, face recognition models, and / or other vision-based ML models. Furthermore, the prediction output 102 in this example (and / or its associated prediction values, such as probability, log-likelihood, binary values, and / or any other prediction values) may be based on one or more vision-based ML models used to process client data 101, such as those related to... Figure 2The predictions are described by one or more vision-based ML models. Additionally, for example, it is additionally or alternatively assumed that client data 101 corresponds to text data or touch data generated (or based on processing audio and / or visual data) via one or more user interface input components of client device 150. In this example, the on-device ML engine 132 may utilize one or more text-based and / or touch-based ML models to process client data 101 when generating prediction output 102. These text-based and / or touch-based ML models include Natural Language Understanding (NLU) models, performance models, and / or other text-based and / or touch-based ML models. Furthermore, the prediction output 102 in this example (and / or its associated prediction values, such as probabilities, log-likelihoods, binary values, and / or any other prediction values) may be based on one or more text-based and / or touch-based ML models used to process client data 101, such as... Figure 2 One or more predictions described by a text-based ML model and / or a touch-based ML model.

[0030] In various implementations, to enable the use of the on-device ML model when locally updating the on-device ML model at client device 150 and / or generating one or more updates for the corresponding global ML model at client device 150, client device 150 may provide the prediction output 102 to gradient engine 136. The gradient engine may compare the prediction output 102 with a supervision signal 103 to generate gradient 104. In some implementations, the supervision signal 103 may be an implicit supervision signal generated using unsupervised learning engine 134. Where the supervision signal is unavailable based on explicit user feedback (e.g., feedback from human users at client device 150 and / or feedback from human annotators of client data 101), unsupervised learning engine 134 may utilize one or more self-supervised learning techniques and / or semi-supervised learning techniques (e.g., as per [reference to...]). Figure 4The supervision signal 103 is generated by the client data 101, which is typical of federated learning in a privacy-conscious ML model. For example, suppose client data 101 corresponds to audio data generated via one or more microphones of client device 150, and further suppose that on-device ML engine 132 uses an on-device ASR model to process the audio data to generate and select a given speech hypothesis predicted to correspond to spoken utterances captured in the audio data. In this example, the unsupervised learning engine 134 may generate the supervision signal 103 using one or more self-supervised learning techniques and / or semi-supervised learning techniques described herein, even if the human user of client device 150 does not provide any explicit feedback on the accuracy of the given speech hypothesis. In additional or alternative implementations, the supervision signal 103 may be an explicit supervision signal generated based on client data 101, and the unsupervised learning engine 134 (e.g., feedback from the human user of client device 150) may optionally be omitted. For example, again assuming that client data 101 corresponds to audio data generated via one or more microphones of client device 150, and again assuming that on-device ML engine 132 processes the audio data using an on-device ASR model to generate and select a given speech hypothesis predicted to correspond to spoken utterances captured in the audio data. However, it is further assumed that a human user of client device 150 corrects one or more terms or phrases of the given speech hypothesis. In this example, the correction can be captured in client data 101 and used as supervision signal 103 without using unsupervised learning engine 134. However, in this example, unsupervised learning engine 134 may additionally or alternatively utilize one or more self-supervised learning techniques and / or semi-supervised learning techniques described herein to generate additional or alternative supervision signals.

[0031] In some implementations, gradient 104 (and other gradients) can be derived from a loss function used to train the ML model (e.g., an on-device ML model and / or a global ML model), such that the gradient represents the value of the loss function (or its derivative) obtained from the comparison of the supervision signal 103 with the predicted output 102. For example, gradient engine 136 can generate a zero gradient when the supervision signal 103 and the predicted output 102 match. Furthermore, for example, gradient engine 136 can generate a non-zero gradient when the supervision signal 103 and the predicted output 102 do not match, which may optionally depend on the degree of mismatch. The degree of mismatch may be based on the degree of mismatch between the deterministic comparison of the supervision signal 103 and the predicted output 102. In additional or alternative implementations, gradient 104 (and other gradients) can be derived from a loss function used to train the model, such that gradient 104 represents the value of the loss function (or its derivative) determined based on the predicted output 102 (e.g., without considering any supervision signal).

[0032] In some versions of those implementations, gradient 104 (and other gradients) may be specific to a particular ML layer of a given ML model. For example, again assuming that client data 101 corresponds to audio data generated via one or more microphones of client device 150, and again assuming that on-device ML engine 132 processes the audio data using an on-device ASR model to generate and select a given speech hypothesis predicted to correspond to spoken utterances captured in the audio data. Further assuming that the on-device ASR model includes at least one or more first on-device ASR layers and one or more second on-device ASR layers. In this example, the loss function and / or the value of the loss function may be specific to one or more first on-device ASR layers and / or specific to one or more second on-device ASR layers. For example, a first loss function or a first value of the first loss function may be used to generate gradient 104 for one or more first ASR layers, and a second loss function or a second value of the first loss function may be used to generate gradient 104 for one or more second ASR layers. Additionally or alternatively, gradient 104 may be generated at client device 150 in this manner for one or more first ASR layers, and gradients may not be generated at client device 150 for one or more second ASR layers. Conversely, gradients for one or more second ASR layers can be generated at one or more additional client devices 170 in the same or similar manner described regarding the generation of gradient 104 at client device 150. Each of the one or more additional client devices 170 may include at least the elements contained in the representation of client device 150. Figure 1 The corresponding instance of the component in the box.

[0033] It is worth noting that in traditional federated learning, multiple client devices (e.g., one or more of client device 150 and additional client devices 170) each generate a corresponding gradient for the corresponding ML model on the corresponding device based on corresponding client data received at a given client device among the multiple client devices. For example, each of the multiple client devices may have a corresponding peer ML model stored in the memory of the corresponding device and process the corresponding client data to generate a gradient for the corresponding instance of the ML model on the corresponding device. Although the gradient in this example is generated at dissimilar client devices, it is generated for the ML model on the corresponding peer device. Furthermore, the remote system 160 may use the corresponding gradient (or a corresponding update generated based on the corresponding gradient) to update the remote-based peer of the ML model on the corresponding peer device. However, in traditional federated learning, the corresponding gradient (or a corresponding update generated based on the corresponding gradient) is typically used as a whole for the ML model on the corresponding peer device. In contrast, to improve the efficiency of federated learning, the generated corresponding gradient may be applied only to a portion of the ML model on the corresponding peer device. Therefore, the consumption of computing resources and memory at each of the multiple client devices can be reduced, because the corresponding gradients generated based on the corresponding client data can be applied only to a portion of the ML model on the corresponding client device, and the consumption of network resources when transmitting the corresponding gradients to the remote system 160 can be reduced.

[0034] Therefore, in various implementations, client device 150 (and one or more of additional client devices 170) may include an on-device scheduling engine 138 to ensure that appropriate gradients are generated for dissimilar portions of the corresponding on-device ML model at each of the plurality of client devices. On-device scheduling signal 138 may generate scheduling signal 105 provided to gradient engine 136 to ensure that client device 150 generates gradients for appropriate on-device ML layers of the corresponding on-device ML model. For example, on-device scheduling engine 138 of client device 150 may instruct gradient engine 136 to generate gradients 104 for one or more first on-device ML layers of the corresponding on-device ML model via scheduling signal 105. Furthermore, the corresponding on-device scheduling engine of one or more of the additional client devices 170 may instruct the corresponding gradient engine to generate appropriate gradients for one or more second on-device ML layers of the corresponding on-device ML model via corresponding scheduling signals. Scheduling signal 105 of client device 150 and the corresponding scheduling signals of one or more of the additional client devices 170 may be received at each of the client devices from a remote scheduling engine 146 of the remote system (e.g., as per [reference]). Figure 5 and Figure 6 (as described).

[0035] In some implementations, gradient engine 136 may cause gradient 104, along with other gradients generated at client device 150, to be stored in the on-device memory of client device 150 until one or more client device conditions are met (e.g., see reference 150). Figure 3 (Description) so that gradient 104 (and other gradients) is transmitted to remote system 160. In additional or alternative implementations, gradient engine 136 may cause gradient 104 to be provided to ML update engine 140. ML update engine 140 may generate ML update 106 based on gradient 104, which includes an indication of the type of global ML model to be updated based on ML update 106 (e.g., an ASR update when ML update 106 is based on gradient generation for an ASR model, a hot word update when ML update 106 is based on gradient generation for a hot word model, etc.). In some implementations, ML update 106 may include gradient 104 and an indication of one or more global ML layers of the global ML model to be updated based on the gradient. For example, suppose on-device scheduling engine 138 instructs gradient engine 136 to generate gradient 104 for one or more first on-device ML layers of the corresponding on-device ML model via scheduling signal 105. In this example, ML update engine 140 can generate ML update 106, which includes gradient 104 for one or more ML layers on a first device, and an indication that one or more first global ML layers of the corresponding counterpart global ML model should be updated using gradient 104. Furthermore, suppose the scheduling engine on the corresponding device instructs a given additional client device 170 in the additional client device 170 to instruct the corresponding gradient engine to generate corresponding gradients for one or more ML layers on a second device for the ML model on the corresponding counterpart device via a corresponding scheduling signal. In this example, the corresponding ML update engine can generate additional ML update 107, which includes corresponding gradients for one or more ML layers on a second device, and an indication that one or more second global ML layers of the corresponding counterpart global ML model should be updated using the corresponding gradients. It is noteworthy that in these examples, the ML model on the corresponding device is not locally updated at the given client device 170 in the client device 150 or the additional client device 170 based on gradient 104 or the corresponding gradients, respectively.

[0036] In an additional or alternative implementation, the ML update engine 140 may cause the ML model on the corresponding counterpart device to be updated based on gradient 104. Continuing the example above, again assume that the on-device scheduling engine 138 instructs the gradient engine 136 to generate gradient 104 for one or more first on-device ML layers for the ML model on the corresponding counterpart device via scheduling signal 105. In this example, the ML update engine 140 may cause gradient 104 to backpropagate across one or more first on-device ML layers to update one or more corresponding first on-device weights of one or more first on-device ML layers, while one or more second on-device ML layers (and / or other on-device ML layers) are fixed (e.g., not updated), thereby producing one or more updated first on-device ML layers. Furthermore, the same process may be repeated at a given additional client device 170 in the additional client device 170 to update one or more corresponding second on-device weights of one or more second on-device ML layers, while one or more first on-device ML layers (and / or other on-device ML layers) are fixed, thereby producing one or more updated second on-device ML layers.

[0037] In some versions of these additional or alternative implementations, ML update 106 may include one or more updated first on-device ML layers and an indication of one or more global ML layers of the global ML model that should be replaced in the remote memory of the remote system 160 using the one or more updated first on-device ML layers. Continuing the example above, ML update 106 may include an indication of one or more first global ML layers of the corresponding global ML model that should be replaced using the one or more updated first on-device ML layers. Similarly, additional ML update 107 may include an indication of one or more second global ML layers of the corresponding global ML model that should be replaced using the one or more updated second on-device ML layers.

[0038] In other versions of these additional or alternative implementations, ML update 106 may include one or more updated first on-device weights for one or more first-update on-device ML layers and an indication of one or more global weights of the global ML model that should be replaced at the remote memory of the remote system 160 using the one or more updated first on-device weights. Continuing with the above example, ML update 106 may include an indication of one or more first global weights of one or more first global ML layers of the corresponding global ML model that should be replaced using the one or more updated first on-device weights for one or more first-update on-device ML layers. Similarly, additional ML update 107 may include an indication of one or more second global weights of one or more second global ML layers of the corresponding global ML model that should be replaced using the one or more updated second on-device weights for one or more second-update on-device ML layers.

[0039] In other words, in traditional federated learning, the corresponding global ML model can be updated in a distributed manner because it is updated based on gradients (e.g., gradient 104 and / or one or more additional gradients) or updates (e.g., ML update 106 and / or one or more additional ML updates 107) generated at multiple client devices. However, generating these gradients or updates can consume considerable computational and network resources at each of the multiple client devices. Therefore, by utilizing the technique described herein, both the computational and network resources that would otherwise be consumed at each of the multiple client devices can be saved by enabling each client device to generate and transmit less data compared to traditional federated learning.

[0040] The remote update engine 142 can utilize at least one or more additional ML updates 107, of ML updates 106 and / or additional ML updates 107 received from client device 150, to update one or more global ML models stored in the global ML model database 154B. In some implementations, when one or more additional ML updates 107, of ML updates 106 and / or additional ML updates 107 received from client device 150, are received at remote system 160, they can be stored in the updated database 180. In some versions of those implementations, the remote update engine 142 can obtain updates to update one or more global ML models (e.g., as per the conditions of one or more remote systems) when one or more remote system conditions are met. Figure 7 (as described above). In additional or alternative implementations, the remote update engine 142 may update one or more global ML models upon receiving ML update 106 from client device 150 and / or upon receiving one or more additional ML updates 107 from one or more additional client devices 170.

[0041] It is worth noting that the remote update engine 142 can update the corresponding global ML model in various ways based on the type of one or more of the ML updates 106 received from client device 150 and / or one or more of the additional ML updates 107 received from one or more of the additional client devices 170. For example, the remote update engine 142 can identify a specific global ML model from the global ML models stored in the global ML model database 154B to update its global layer and / or its global weights. In some implementations, the remote update engine 142 can identify a specific global ML model based on an indication of the type of global ML model to be updated based on ML updates 106 and / or one or more of the additional ML updates 107. For example, if multiple updates for an ASR model are received and / or stored in the updated database 180, the remote update engine 142 can identify the corresponding counterpart global ASR model to be updated. Furthermore, how the remote update engine 142 updates the corresponding counterpart global ASR model may be based on the content of one or more of the ML updates 106 and / or the additional ML updates 107. For example, if a given update includes gradients and indicates that it is for one or more first on-device ASR layers of an on-device ASR model, then the remote update engine 142 can backpropagate gradients across one or more corresponding first global ASR layers of the corresponding corresponding global ASR model while at least one or more second global ASR layers of the corresponding global ASR model are fixed. Furthermore, for example, if a given update includes one or more first updated on-device ASR layers of an on-device ASR model and / or one or more updated first on-device weights of one or more first updated on-device ASR layers of an on-device ASR model, then the remote update engine 142 can replace one or more global ASR layers of the corresponding global ASR model with one or more first updated on-device ASR layers and / or replace one or more first global weights for one or more global ASR layers with one or more updated first on-device weights for one or more first updated on-device ASR layers of the on-device ASR model. In various embodiments, updates stored in the update database 180 may include different types of updates. In those embodiments, the remote update engine 142 may utilize one or more criteria to determine how and / or when to utilize different types of updates. One or more criteria may indicate which types of updates should be prioritized over other types of updates. As a non-restrictive example, one or more criteria may indicate that updates replacing individual layers and / or weights should take precedence over updates including gradients, but it should be understood that other criteria are considered in this paper.Furthermore, in some implementations, the remote update engine 142 can identify a single global ML model to be updated at a given time instance, while in other implementations, the remote update engine 142 can identify multiple global ML models to be updated in parallel at a given time instance.

[0042] In some implementations, the remote update engine 142 may assign updates to specific iterations of updating one or more global models based on one or more criteria. For example, one or more criteria may include the types of updates available to the remote update engine 142, a threshold number of updates available to the remote update engine 142 (and optionally for dissimilar portions of one or more global models), a threshold duration required to perform the update, and / or other criteria. Specifically, the remote update engine 142 may identify multiple sets or subsets of updates generated by client devices 150 and / or 170. Furthermore, the remote update engine 142 may update one or more global models based on these sets or subsets of updates. In some other versions of those implementations, the number of updates in the sets or subsets may be the same or different. In other versions of those implementations, each set or subset of updates may optionally include updates from at least two unique client devices. In other implementations, the remote update engine 142 may use updates to update one or more global ML models in a first-in, first-out (FIFO) manner without assigning any updates to specific iterations of updating one or more global ML models.

[0043] The update distribution engine 144 may transmit one or more updated global ML models 108 (e.g., including the updated global ML model itself, its updated layers, and / or its updated weights) to one or more of client devices 150 and / or attached client devices 170. In some implementations, in response to one or more remote system conditions being met at remote system 160 and / or one or more client device conditions being met at one or more of client devices 150 and / or attached client devices 170, the update distribution engine 144 may transmit one or more updated global ML models 108 (e.g., as per the context of...). Figure 3 and Figure 7(As described above). It is worth noting that one or more of the updated global ML models 108 can be transmitted synchronously or asynchronously to one or more of client devices 150 and / or attached client devices 170. Upon receiving one or more of the updated global ML models 108, the updated global ML models 108 can replace the corresponding on-device ML model stored in their on-device memory. Furthermore, client devices 150 and / or 190 can subsequently use the updated on-device ML models one or more to make predictions based on further client data received at one or more of client devices 150 and / or attached client devices 170 (e.g., regarding...). Figure 2 (As described herein). One or more of client devices 150 and / or attached client devices 170 may continue to transmit updates to remote system 160 in the manner described herein. Furthermore, remote system 160 may continue to update one or more of the global ML model in the manner described herein.

[0044] Turn now Figure 2 The embodiment illustrates a client device 250, which depicts various devices incorporating (or communicating with) ML engines as part of (or with) an automation assistant client 240. Interfaces between corresponding ML models and ML engines on various devices are also illustrated. For simplicity, other components of the client device 250 are not shown. Figure 2 The diagram is shown in the image. Figure 2 The illustration shows an example of how the automation assistant client 240 can utilize ML engines and their corresponding ML models on various devices when performing various actions.

[0045] Figure 2 The client device 250 is illustrated as having one or more microphones 211, one or more speakers 212, one or more vision components 213, and a display 214 (e.g., a touch-sensitive display). The client device 250 may also include pressure sensors, proximity sensors, accelerometers, temperature sensors, magnetometers, and / or other sensors for generating sensor data. The client device 250 selectively executes at least the automation assistant client 240. Figure 2 In the example, the automated assistant client 240 includes a hot word detection engine 222, a hot word free-calling engine 224, a persistent conversation engine 226, an ASR engine 228, an object detection engine 230, an object classification engine 232, a voice recognition engine 234, and a face recognition engine 236. The automated assistant client 240 also includes a voice capture engine 216 and a visual capture engine 218. It should be understood that... Figure 2The ML engines and ML models described herein are provided for illustrative purposes only and are not intended to be limiting. For example, the automation assistant client 240 may also include additional and / or alternative engines, such as a text-to-speech (TTS) engine and a corresponding TTS model, a voice activity detection (VAD) engine and a corresponding VAD model, an endpoint detector engine and a corresponding endpoint detector model, a lip movement engine and a corresponding lip movement model, and / or other engines and associated ML models. Furthermore, it should be understood that one or more of the engines and / or models described herein may be combined such that a single engine and / or model can perform the functionality of multiple engines and / or models described herein.

[0046] One or more cloud-based automation assistant components 270 may optionally be implemented on one or more computing systems (collectively, “cloud” computing systems) coupled to client device 250 via one or more network communications generally indicated by 299. The cloud-based automation assistant components 270 may be implemented, for example, via a high-performance server cluster. In various implementations, an instance of the automation assistant client 240, through its interaction with one or more of the cloud-based automation assistant components 270, can form a logical instance of an automation assistant from the user's perspective, as indicated by 295, with which the user can perform human-computer interaction (e.g., typing interaction, verbal interaction, gesture-based interaction, and / or touch-based interaction).

[0047] Client device 250 may be, for example, a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device in a user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), a stand-alone interactive speaker, a smart device—such as a smart TV (or a standard TV equipped with a network-connected dongle with automated assistant functionality)—and / or a wearable device including a computing device (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual or augmented reality computing device). Additional and / or alternative client devices may be provided.

[0048] One or more vision components 213 can take various forms, such as a single-view camera, a stereo camera, a LiDAR component (or other laser-based component), a radar component, etc. One or more vision components 213 can be used, for example, via a vision capture engine 218, to capture image data corresponding to visual frames (e.g., image frames, laser-based visual frames) of the environment in which the client device 250 is deployed. In some embodiments, such visual frames can be used to determine whether a user is present near the client device 250 and / or the distance of a given user (e.g., the user's face) relative to the client device 250. Such determination can, for example, be used to determine whether activation is required. Figure 2 The various devices depicted include ML engines and / or other engines. Furthermore, the voice capture engine 218 can be configured to capture spoken words and / or other audio data captured via one or more of the microphones 211.

[0049] As described in this article, such audio data and / or other non-audio sensor data—such as visual data and text data (collectively referred to herein as client data)—can be generated by... Figure 2 The various engines described in the text are used to process the data so that it can be used at the client device 250 to... Figure 1 , Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 The described method updates the corresponding ML model and / or is used in one or more locations within the cloud-based automated assistant component 270 to update the relevant ML model. Figure 1 , Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 The corresponding ML model is updated in the manner described to make predictions.

[0050] As some non-limiting examples, the corresponding hot word detection engines 222 and 272 can utilize the corresponding hot word detection models 222A and 272A to predict whether the audio data includes one or more specific words or phrases to invoke the automation assistant 295 (e.g., "Ok Assistant," "Hey Assistant," "What is the weather Assistant," etc.) or certain functions of the automation assistant 295 (e.g., "Stop" when an alarm is sounding or music is playing, "Next" when viewing a picture or listening to music, etc.); the corresponding hot word free invocation engines 224 and 274 can utilize the corresponding hot word free invocation models 224A and 274A to predict whether non-audio sensor data (e.g., visual data) includes gestures or signals used to invoke the automation assistant 295 (e.g., based on the user's gaze and...). The selection of the location is further based on the user's mouth movement); the corresponding persistent conversation engines 226, 276 can use the corresponding persistent conversation models 226A, 276A to predict whether other audio data points to the automation assistant 295 (e.g., or to an additional user in the environment of the client device 250); the corresponding ASR engines 228, 278 can use the corresponding ASR models 228A, 278A to generate recognized text, or predict phonemes and / or tags corresponding to the audio data detected at the client device 250, and generate recognition based on the phonemes and / or tags. The text; the corresponding object detection engines 230, 280 can use the corresponding object detection models 230A, 280A to predict the location of objects included in the visual data captured at the client device 250; the corresponding object classification engines 232, 282 can use the corresponding object classification models 232A, 282A to predict the object classification of objects included in the visual data captured at the client device 250; the corresponding speech recognition engines 234, 284 can use the corresponding speech recognition models 234, 284A to predict whether the audio data captures the spoken words of one or more users of the client device 250 (e.g., by generating speaker embeddings or other representations, which can be compared with the corresponding actual embeddings of one or more users of the client device 250); and the corresponding face recognition engines 236, 286 can use the corresponding face recognition models 236A, 286A to predict whether the image data captures one or more users in the environment of the client device 250 (e.g., by generating image embeddings or other representations, which can be compared with the corresponding image embeddings of one or more users of the client device 250). Each of these various ML models (and other ML models) can be updated in the manner described in this article.

[0051] In some implementations, one or more of the client device 250 and the cloud-based automated assistant component 270 may further include Natural Language Understanding (NLU) engines 238, 294 and performance engines 240, 296, respectively. NLU engines 238, 294 may utilize corresponding NLU models 238A, 294A to perform natural language understanding on the identified text, predicted phonemes, and / or predicted tokens generated by ASR engines 228, 278 to generate NLU data. The NLU data may include, for example, intent corresponding to spoken utterances, and optionally slot values ​​corresponding to parameters of the intent. Furthermore, one or more of the client device 250 and the cloud-based automated assistant component 270 may also include performance engines 240, 296, respectively. Performance engines 240, 296 may utilize corresponding performance models or rules 240A, 296A and generate performance data based on the processed NLU data. The performance data can define specific performances in response to user input (e.g., spoken words, typed input, touch input, gesture input, and / or any other type of user input) provided by a user of client device 250. Specific performances may include interactions to be performed with a locally installed application based on user input, commands to be transmitted to an Internet of Things (IoT) device based on user input (directly or via a corresponding remote system), and / or other parsed actions to be performed based on user input. Performance data is then provided for the local and / or remote performance / execution of the determined actions to enable the performance of the specific user input. Execution may include, for example, rendering local and / or remote responses (e.g., visually and / or audibly rendering (optionally utilizing a TTS module on the device)), interacting with a locally installed application, transmitting commands and / or other actions to an IoT device. In other embodiments, NLU engines 238, 294 and performance engines 240, 296 may be omitted, and ASR engines 228, 278 may directly generate performance data based on user input. For example, suppose one or more of ASR engines 228, 278 process the spoken phrase “turn on the lights” using the corresponding ASR model 228A, 278A. In this example, ASR engines 228, 278 may generate semantic output, which is then transmitted to a software application associated with the lights and / or directly to the lights that indicate they should be turned on, and optionally generate a transcription of the spoken phrase for visual rendering to be presented to the user who provided the spoken phrase.

[0052] It is worth noting that the cloud-based automation assistant component 270 includes the references in this article. Figure 2The described engines and models have remote counterparts. However, in some implementations, these engines and models may not be utilized because they can be directly transferred to client device 250, stored in on-device memory of client device 150, and executed locally at client device 250. In other implementations, these engines and models can be exclusively utilized when client device 250 detects any user input (e.g., text-based, audio-based, vision-based, etc.) and transmits the user input to cloud-based automation assistant component 270. In various implementations, these engines and models executed at client device 250 and cloud-based automation assistant component 270 can be utilized in a distributed manner in combination with each other. Nevertheless, a remote execution module may be optionally included, which performs remote execution based on locally or remotely generated NLU data and / or fulfillment data. Additional and / or alternative remote engines may be included. As described herein, in various implementations, on-device speech processing, on-device image processing, on-device NLU, on-device fulfillment, and / or on-device execution can be prioritized, at least due to the reduced latency and / or network usage they provide when parsing spoken utterance (since no client-server round trips are required to parse spoken utterance). However, one or more cloud-based automation assistant components 270 can be selectively utilized at least. For example, such components can be used in parallel with on-device components, and output from such components can be utilized when local components fail. For example, if any of the on-device engine and / or model fails (e.g., due to the relatively limited resources of client device 250), the more stable resources of the cloud can be utilized.

[0053] Now go to Figure 3 The diagram illustrates a flowchart of an example method 300 for client-side operations of federated learning that demonstrates the efficiency of machine learning (ML) models. For convenience, the operation of method 300 is described with reference to the system implementing the operation. The system of method 300 includes computing devices (e.g., Figure 1 Client devices 150 Figure 2 250 client devices Figure 8 The computing device 810 and / or other computing devices may include one or more processors and / or other components. Furthermore, although the operations of method 300 are shown in a specific order, this is not intended to be restrictive. One or more operations may be reordered, omitted, or added.

[0054] In box 352, the system receives client data from the user of the client device at the client device. Client data can be one or more of various types. These types of client data can include, for example, audio data generated via the client device's microphone, visual data generated via the client device's visual components, text or touch data generated via the client device's user interface input components (e.g., via a touchscreen display, mouse and keyboard, and / or other user interface components), and / or any other client data. In some implementations, the client data may be directed to an automated assistant (e.g., to enable the provision of visual and / or audible content, to enable control of IoT devices, etc.), while in other implementations, the client may be provided independently of the automated assistant (e.g., dictating emails or recordings without interaction with the automated assistant).

[0055] In box 354, the system uses an on-device ML model stored in on-device memory on the client device to process client data to generate a predictive output. The on-device ML model includes at least one or more first on-device ML layers and one or more second on-device ML layers. The on-device ML model used to generate the predictive output can be based on the type of client data received at box 352. For example, in an implementation where the client data received at box 352 is audio data, the system can utilize an on-device audio-based model to process the client data to generate a prediction. Furthermore, the predictive output can be based on the type of audio-based ML model used to process the client data. For example, if the on-device audio-based ML model is an on-device ASR model, the predictive output can be a predictive speech hypothesis predicted to correspond to any spoken utterance captured in the audio data. Additionally, for example, if the on-device audio-based ML model is an on-device hot word detection model, the predictive output can include a prediction of whether the audio data includes an indication of a specific word or phrase that invokes or otherwise controls an automation assistant.

[0056] In box 356, the system uses unsupervised learning to generate gradients based on the predicted output. This paper describes the generation of gradients using unsupervised learning in more detail (e.g., regarding...). Figure 1 and 4 In box 358, the system generates a first update for one or more first on-device ML layers of the on-device ML model based on gradients. (See above regarding...) Figure 1The first update may include, for example, an indication of one or more first on-device ML layers of an on-device ML model associated with the first update, an indication of one or more first correspondent global ML layers of a correspondent global ML model updated based on the first update, an indication of an on-device ML model for generating a first update of one or more first on-device ML layers of an on-device ML model, a gradient generated based on the prediction output, one or more updated first on-device ML layers of an on-device ML model locally updated at the client device based on the gradient, and / or one or more updated on-device weights of one or more updated first on-device ML layers of an on-device ML model.

[0057] The system returns box 352 to receive additional client data and performs subsequent iterations of the operations in boxes 352 to 358. In some implementations, during subsequent iterations of the operations in boxes 352 to 358, the system may generate additional first updates for one or more first on-device ML layers of the on-device ML model. In additional or alternative implementations, and during subsequent iterations of the operations in boxes 352 to 358, the system may generate second updates for one or more second on-device ML layers of the on-device ML model. For example, the client device may be used to generate first updates only for one or more first on-device ML layers of the on-device ML model, or the client device may alternate between generating first and second updates for dissimilar on-device ML layers of the on-device ML model. In some implementations, a scheduling engine (e.g., Figure 1 The on-device scheduling engine 138 can use scheduling signals to instruct client devices to generate a first update for one or more specific ML layers on the device. In some versions of those implementations, the on-device scheduling engine can generate scheduling signals based on data received from a remote system. In other words, the remote system can instruct multiple client devices (including the client device receiving client data in block 352 and at least one additional client device) to generate updates for dissimilar portions of the ML model on the corresponding device, and the remote system can combine updates for dissimilar portions of the ML model on the corresponding device to update the entire corresponding global ML model.

[0058] In box 360, the system determines whether one or more client device conditions are met for transmitting the first update (and / or any other updates generated during one or more subsequent iterations of the operations in boxes 352-358) to the remote system. One or more client device conditions may include, for example, a specific time of day, a specific day of the week, whether the client device is charging, whether the client device is at least in a threshold charging state, whether the client device's temperature is below a temperature threshold, and / or whether the client device is held by a user. If the system determines at the iteration in box 360 that one or more client device conditions are not met, the system continues to monitor for the satisfaction of one or more client device conditions in box 360. It is worth noting that while the system continues to monitor for the satisfaction of one or more client device conditions in box 360, the system may continue to generate updates via subsequent iterations of the operations in boxes 352-358. If the system determines at the iteration in box 360 that one or more client device conditions are met, the system proceeds to box 362.

[0059] In box 362, the system transmits a first update for one or more ML layers on a first device to a remote system, such that the remote system updates one or more first global ML layers of the global ML model—which are remote-based counterparts of the ML model on the device—based on the first update, while one or more second global ML layers of the global ML model remain fixed. It is worth noting that the global ML model is a remote-based counterpart of the ML model on the device and includes one or more first global ML layers and one or more second global ML layers. Therefore, one or more first global ML layers can be remote-based counterparts of one or more ML layers on the first device. However, refer to... Figure 3 The first update, described as a non-limiting example, is only for updating one or more first global ML layers, while one or more second global ML layers are fixed (e.g., not updated based on the first update). Therefore, at least one additional update may be required at the remote system to update one or more second global ML layers. In some implementations, as described above, at least one update required to update one or more second global ML layers can be generated at a subsequent iteration of the operations in blocks 352-358. In additional or alternative implementations, at least one update required to update one or more second global ML layers can be generated at the additional client device based on additional client data received at the additional client device.

[0060] In box 364, the system determines whether one or more remote system conditions are met for transmitting an updated global ML model (e.g., the updated global ML model itself, one or more updated global ML layers of the updated global ML model, or one or more updated global weights of the updated global ML model) to the client device. One or more remote system conditions may include, for example, a specific time of day, a specific day of the week, whether the updated global ML model has been updated using a threshold number of updates, and / or whether the performance of the updated global ML model meets a performance threshold. If, at the iteration in box 364, the system determines that one or more client device conditions are not met, the system continues to monitor for the satisfaction of one or more remote system conditions at box 364. Notably, while the system continues to monitor for the satisfaction of one or more remote system conditions at box 364, the system may continue to update the global model based on updates received from the client device and / or one or more additional client devices. If, at the iteration in box 364, the system determines that one or more remote system conditions are met, the system proceeds to box 366.

[0061] In box 366, the system receives an updated global ML model at the client device (e.g., including the updated global ML model itself, one or more updated global ML layers, and / or one or more updated global weights). Although not depicted for brevity, the client device may receive the updated global model only in response to determining that one or more client device conditions are met (e.g., as described with respect to the operation in box 360). In response to receiving the updated global ML model, the client device may replace the on-device ML model in the client device's on-device memory with the updated global ML model, thereby producing an updated device ML model. However, the client device (and one or more additional client devices) may continue to generate additional updates for the updated on-device ML model and may maintain an instance of the updated global ML model at a remote system for further updates based on the additional updates.

[0062] although Figure 3 Method 300 is described only for a single client device; however, it should be understood that this is for illustrative purposes and not for limitation. For example, multiple additional client devices can be used. Figure 3 Method 300 operates to generate updates for the global ML model. Furthermore, as described herein, a scheduling engine (e.g., executing locally at each location on the client device and / or remotely at a remote system) can schedule the generation of updates for dissimilar portions of the corresponding on-device ML model executed locally at each location on the client device, thereby ensuring that the remote system receives updates for each of the dissimilar portions of the corresponding global ML model.

[0063] Now go to Figure 4 The diagram depicts a flowchart of an example method 400 for federated learning techniques used to achieve efficiency in machine learning (ML) models. Specifically, Figure 4 Method 400 illustrates the generation of gradients (e.g., as shown in the diagram regarding...). Figure 3 This is a non-limiting example of method 400 (described in block 356). For convenience, the operation of method 400 is described with reference to a system that performs the operation. The system of method 400 includes a computing device (e.g., Figure 1 Client devices 150 Figure 2 250 client devices Figure 8 The computing device 810 and / or other computing devices may contain one or more processors and / or other components. Furthermore, although the operations of method 400 are displayed in a specific order, this does not imply limitation. One or more operations may be reordered, omitted, or added.

[0064] In box 452, the system identifies client data (e.g., in...). Figure 3In block 352, the target portion of the client data received during the iteration of the operation is located after the preceding portion of the client data and before the following portion. In block 454, the system masks the target portion of the client data. The target portion of the client data can be arbitrarily selected or can be selected based on one or more criteria, such as a specific segment between n and m seconds of audio data corresponding to the target portion, a portion of visual data corresponding to the center portion of an image corresponding to the target portion, the center portion of text data corresponding to the target portion, and / or any other criteria for selecting the target portion of the client data. In some implementations, the client data can correspond to audio data. In those implementations, the target portion of the client data can correspond to a target audio waveform portion of the corresponding audio data stream, the preceding portion of the client data can correspond to a preceding audio waveform portion received before the target audio waveform portion, and the following portion of the client data can correspond to a following audio waveform portion received after the target audio waveform portion. Furthermore, the system can use various masking techniques to mask the target audio waveform portion. In additional or alternative versions of those implementations, the client data may correspond to a representation of audio data processed based on audio data (such as a vector representation of audio data), ASR data generated based on processing the corresponding audio data stream, and / or other representations of audio data. In those implementations, the target portion of the client data may correspond to a target representation portion of the audio data, the preceding portion of the client data may correspond to a preceding representation portion before the target representation portion, and the following portion of the client data may correspond to a following representation portion after the target representation portion. Furthermore, the system may use various masking techniques to mask the target representation portion. In additional or alternative versions of those implementations, the client data may correspond to text data or visual data. In those implementations, the system may identify the target portion, the preceding portion, and the following portion in the same or similar manner, and the system may use various masking techniques to determine the target portion.

[0065] At box 456, the system uses an on-device ML model to process the preceding and following portions of the client data to generate a prediction output that predicts the target portion of the client data. For example, in an implementation where the target portion of the client data corresponds to a target audio waveform portion of the audio data, it is further assumed that the audio data captures spoken utterance. In this example, the system can use an on-device ASR model to process the preceding and following audio waveform portions to generate the prediction output. In this example, the prediction output may correspond to a predicted target audio waveform and / or one or more predicted features of the predicted target audio waveform (e.g., predicted amplitude, predicted wavelength, predicted phase, predicted period, and / or other features) that are predicted to correspond to the target audio waveform portion. Additionally or alternatively, the prediction output may more generally correspond to the predicted target audio data and / or one or more predicted features of the predicted audio data (e.g., predicted MFCC, predicted melbank features, and / or other features) that are predicted to correspond to the target audio data. In other words, the system may attempt to reconstruct the target audio waveform portion and / or features of the audio data based on processing the preceding and following audio waveform portions.

[0066] It is worth noting that in embodiments that utilize on-device ASR models to reconstruct the target audio waveform portion, the on-device ASR model can be a multilingual on-device ASR model, as the dominant language of the given client device generating the audio data may be irrelevant to the reconstructed target audio waveform portion. However, in various embodiments, the system may additionally or alternatively use one or more on-device ASR models specific to the dominant language of the given client device generating the audio data to process the preceding and following audio waveform portions. Furthermore, for example, in an embodiment where the target portion of the client data corresponds to the target representation portion of the audio data, it is also assumed that the audio data captures spoken utterances. In this example, the system may use on-device ASR models and / or on-device NLU models to process the preceding and following representation portions to generate a predicted output. In this example, the predicted output may correspond to a predicted target representation that is predicted to correspond to the target representation portion. In other words, the system may attempt to reconstruct the target representation portion based on processing the preceding and following representation portions.

[0067] In box 458, the system compares the predicted output with the target portion of the client data. In box 460, the system generates gradients based on the comparison of the predicted output with the target portion of the client data. In other words, the system can use the actual target portion of the client data masked in box 454 as a supervision signal, and in box 458, it can compare the predicted output, which is predicted to correspond to that target portion, with it. Furthermore, based on the comparison, the system can perform gradient generation as additional client data is received. Figure 4 Method 400 additional iterations.

[0068] although Figure 4 Method 400 primarily describes client-side data corresponding to the audio data; however, it should be understood that this is for illustrative purposes only and not for limitation. Instead, it should be understood that methods regarding... Figure 4 Method 400 describes techniques for generating one or more gradients using any global ML model described herein (such as other vision-based ML models and / or text-based ML models). Furthermore, although... Figure 4 Method 400 describes the use of self-supervised learning to generate gradients, but it should be understood that this is for illustrative purposes only, not for limitation. For example, Figure 4 Method 400 can additionally or alternatively utilize explicit feedback from human users when generating gradients, such as corrections to transcriptions generated using an on-device ASR model, object classifications generated using an on-device object classification model, calls generated using an on-device hot word model or an on-device hot word free model, etc. Furthermore, for example, Figure 4 Method 400 can additionally or alternatively utilize semi-supervised learning when generating gradients, such as using a teacher-student method at the client device.

[0069] Now go to Figure 5 The diagram depicts a flowchart of another example method 500 of federated learning techniques used to achieve efficiency in machine learning (ML) models. Specifically, Figure 5 Method 500 illustrates a non-limiting example of generating updates for ML models on the device (e.g., as shown in the section on...). Figure 3 (as described in block 358 of method 300), and causes the corresponding global ML model to be updated based on the update (e.g., as per [reference]). Figure 3 (As described in block 362 of method 300). For convenience, the operation of method 500 is described with reference to a system that performs the operation. The system of method 500 includes a computing device (e.g., Figure 1 Client devices 150 Figure 2 250 client devices Figure 1 Remote system 160 Figure 2 Cloud-based automation assistant component 270 Figure 8 The computing device 810 and / or other computing devices may include one or more processors and / or other components. Furthermore, although the operations of method 500 are shown in a specific order, this does not imply limitation. One or more operations may be reordered, omitted, or added.

[0070] In box 552, the system is on the client device (e.g., in...). Figure 3The client data received at the iteration of the operation in box 352 is at the gradient generation (e.g., in Figure 3 Before the gradient generated at the iteration of the operation in box 356, at least the first on-device ML layer of the on-device ML model and the second on-device ML layer of the on-device ML model are compressed into one or more first ML layers. The client device may be based on the on-device scheduling engine of the client device (e.g., Figure 1 Scheduling signals generated by the scheduling engine 138 on the device and / or from a remote scheduling engine (e.g., Figure 1 The remote scheduling engine 146 receives scheduling signals to compress the first on-device ML layer and the second on-device ML layer into one or more first on-device ML layers. The scheduling signals may include an indication of the on-device ML layer to be compressed before gradient generation. In some implementations, the first on-device ML layer and the second on-device ML layer may be compressed into one or more first on-device ML layers before generating the prediction output and before generating the gradient; in other implementations, they may be compressed into one or more first on-device ML layers after generating the prediction output but before generating the gradient. It is noteworthy that neither the first on-device ML layer nor the second on-device ML layer is discarded. Instead, compressing the first on-device ML layer and the second on-device ML layer into one or more first ML layers imposes a constraint on the on-device ML model.

[0071] Furthermore, in box 552A, the system compresses at least the second on-device ML layer and the third on-device ML layer of the on-device ML model into one or more second ML layers before generating additional gradients (e.g., based on additional client data received at the additional client device). The system may do this in the same or similar manner, but based on scheduling signals generated by the on-device scheduling engine of the additional client device and / or from a remote scheduling engine (e.g., Figure 1The remote scheduling engine 146 receives a scheduling signal that compresses the second and third on-device ML layers of the on-device ML model into one or more second ML layers. It is noteworthy that the system compresses the second on-device ML layers into one or more first on-device ML layers at box 552 and into one or more second on-device ML layers at box 552A. However, each of the one or more first on-device ML layers and the one or more second on-device ML layers includes at least one unique on-device ML layer—a first on-device ML layer for the one or more first on-device ML layers and a third on-device ML layer for the one or more second on-device ML layers. Nevertheless, it should be understood that this is for illustrative purposes only and not for limitation. For example, ML layers on the same device can be compressed at dissimilar client devices (e.g., ML layers on a first device and ML layers on a second device are compressed into one or more first ML layers at the client device and additional client devices), dissimilar ML layers can be compressed at dissimilar client devices such that at least one unique ML layer is compressed at the dissimilar devices (e.g., as described with respect to the operation of boxes 552 and 552A), and / or dissimilar ML layers can be compressed at dissimilar client devices such that only unique ML layers are compressed at different devices (e.g., ML layers on a first device and ML layers on a second device are compressed into one or more first ML layers at the client device, and ML layers on a third device and ML layers on a fourth device are compressed into one or more second ML layers). Although these dissimilar ML layers may differ from each other, they are adjacent in the ML model on the device and may include some overlap. Therefore, by generating first and second updates as shared updates, remote systems, as described herein, can utilize this overlap to save computational and / or network resources, resulting in improved efficiency in federated learning.

[0072] In box 554, the system generates the first update at the client device (e.g., in...). Figure 3 The first update generated at the first iteration of the operation in box 358 is a shared update for at least the ML layers on the first device and the ML layers on the second device (e.g., compressed into one or more ML layers on the first device in box 552). The system can generate the first update based on gradients (e.g., as per the gradient). Figure 3 (As shown). Furthermore, in box 554A, the system generates a second update at the attached client device (e.g., to update the update in conjunction with the one shown). Figure 3The first update generated at the first iteration of the operation in box 358 is in the same or similar manner, but at the additional client device and based on the additional client data received at the additional client device, as a shared update for at least the ML layer on the second device and the ML layer on the third device (e.g., compressed to one or more ML layers on the second device in box 552A). The system can generate a second update based on additional gradients (e.g., as shown in the example). Figure 3 (The same or similar methods shown).

[0073] In box 556, the system enables a global ML model (e.g., Figure 3 In a global ML model (to be updated), the first global ML layer and the second global ML layer of the global ML model are updated based on a first shared update, while at least the third global ML layer is fixed. The first global ML layer is a remote counterpart of the ML layer on the first device, the second global ML layer is a remote counterpart of the ML layer on the second device, and the third global ML layer is a remote counterpart of the ML layer on the third device. In block 558, the system causes the second and third global ML layers to be updated based on a second shared update, while at least the first global ML layer is fixed. In other words, the system can update the first global ML layer based on only the first update, update the second global ML layer based on both the first and second updates, and update the third global ML layer based on only the second update.

[0074] Now go to Figure 6 The diagram depicts a flowchart of yet another example method 600 of federated learning techniques for achieving efficiency in machine learning (ML) models. Specifically, Figure 6 Method 600 illustrates the generation of updates for the ML model on the device (e.g., as shown in the diagram). Figure 3 Another non-limiting example of method 300 (described in box 358), and such that the corresponding global ML model is updated based on this update (e.g., as per [reference to...]). Figure 3 (As described in block 362 of method 300). For convenience, the operation of method 600 is described with reference to a system that performs the operation. The system of method 600 includes a computing device (e.g., Figure 1 Client devices 150 Figure 2 250 client devices Figure 1 Remote system 160 Figure 2 Cloud-based automation assistant component 270 Figure 8 The computing device 810 and / or other computing devices may include one or more processors and / or other components. Furthermore, although the operations of method 600 are illustrated in a specific order, this does not imply limitation. One or more operations may be reordered, omitted, or added.

[0075] In box 652, system scheduling targets the global ML model (e.g., Figure 3 One or more updates to the global ML model to be updated. For example, in box 652A, the system causes the client device to generate one or more first updates to the Nth on-device ML layer of the ML model on the client device based on processing client data received at the client device (e.g., in...). Figure 3 The first update generated at the first iteration of the operation in box 358), where N is a positive integer. Furthermore, in box 652B, the system causes the additional client device to generate a second update for one or more on-device ML layers of the ML model on the device (e.g., to update the ML model on the device) based on the additional client data received at the additional client device. Figure 3 The first update generated at the first iteration of the operation in box 358 is in the same or similar manner, but at the attached client device and based on the attached client data received at the attached client device. In other words, the system can employ gradient snapshot techniques to improve the efficiency of federated learning of the ML model. Gradient snapshot techniques can be based on the scheduling engine on the corresponding device of the client device and / or attached client device (e.g., Figure 1 On-device scheduling engine 138) and / or from a remote scheduling engine (e.g., Figure 1 The remote scheduling engine 146 receives corresponding scheduling signals to implement this. These scheduling signals can instruct client devices to generate a first update for one or more Nth-th ML layers of the ML model on the device, instruct additional client devices to generate a second update for one or more (N-1)th-th ML layers of the ML model on the device, instruct yet another additional client device to generate a third update for one or more (N-2)th-th ML layers of the ML model on the device, and so on. Therefore, by utilizing gradient snapshot techniques, the remote system can ensure the generation of updates for each of the Nth-th-th ML layers on the device, as described herein.

[0076] In some implementations, a remote scheduling engine (e.g., Figure 1 The remote scheduling engine 146 can divide a superset of client devices into multiple subsets. In some versions of those implementations, each subset of client devices can perform an iterative gradient snapshot technique. For example, a first subset of client devices may include a first client device generating updates for one or more ML layers on an Nth device, a second client device generating updates for one or more ML layers on an (N-1)th device, a third client device generating updates for one or more ML layers on an (N-2)th device, and so on for each of the ML layers on an Nth device. Furthermore, a second subset of client devices may generate updates for each of the ML layers on an Nth device in the same or similar manner.

[0077] In box 654, the system uses a global ML model (e.g., in...). Figure 3 At least one or more of the (N-1)th global ML layers of the global ML model are fixed while causing one or more of the Nth global ML layers of the global ML model to be updated based on a first update of one or more of the Nth on-device ML layers of the on-device ML model, wherein the one or more Nth global ML layers are remote-based counterparts of one or more of the Nth on-device ML layers, and the one or more (N-1)th global ML layers are remote-based counterparts of one or more of the N-1th on-device ML layers. At block 656, the system causes one or more of the N-1th global ML layers to be updated based on a second update of one or more of the N-1th on-device ML layers of the on-device ML model, while at least one or more of the Nth global ML layers of the global ML model are fixed. In some implementations, such as when a remote scheduling engine (e.g., as described above) Figure 1 When the remote scheduling engine (146) divides the superset of client devices into multiple subsets, the system can assign the first subset of client devices to the first iteration of updating the remote-based counterpart global ML model, assign the second subset of client devices to the subsequent second iteration of updating the remote-based counterpart global ML model, and so on.

[0078] Now go to Figure 7 This document describes a flowchart illustrating an example method for remote system-side operations in federated learning to achieve efficiency of machine learning (ML) models. For convenience, the operation of method 700 is described with reference to the system implementing the operation. The system of method 700 includes computing devices (e.g., Figure 1 Remote system 160 Figure 2 Cloud-based automation assistant component 270 Figure 8 The computing device 810 and / or other computing devices may include one or more processors and / or other components. Furthermore, although the operations of method 600 are shown in a specific order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0079] In box 752, the system receives, at a remote system, one or more updates to a global ML model stored remotely in remote memory on the remote system from one or more client devices. The global ML model includes multiple global ML layers, each including at least one first global ML layer and one or more second global ML layers. The one or more updates to the global ML model can be in any manner described herein (e.g., regarding...). Figure 1 , Figure 3 , Figure 4 , Figure 5 and / or Figure 6 It is generated at one or more locations on the client device.

[0080] In box 754, the system determines whether one or more update conditions for updating the global model are met. These conditions may include, for example, a specific time of day, a specific day of the week, a threshold number of updates required to update the global ML model, and / or a threshold number of dissimilar updates required to update the global ML model. If, at the iteration in box 754, the system determines that one or more update conditions are not met, the system returns to box 752 to receive one or more additional updates from one or more client devices. If, at the iteration in box 754, the system determines that one or more update conditions are met, the system proceeds to box 756.

[0081] In box 756, the system causes the global ML model to be updated based on one or more updates received from one or more client devices to generate an updated global ML model. For example, in box 756A, the system causes only one or more first global ML layers to be updated based on a first update received from a client device among the one or more client devices, while at least one of the one or more second global ML layers is fixed. Furthermore, for example, in box 756B, the system causes one or more second global ML layers to be updated based on a second update received from an additional client device among the one or more client devices, while at least one of the one or more first global ML layers is fixed.

[0082] In box 758, the system determines whether one or more client device conditions are met at one or more locations on the client device. These conditions may include, for example, a specific time of day, a specific day of the week, whether the client device is charging, whether the client device is at least in a threshold charging state, whether the client device's temperature is below a temperature threshold, and / or whether the client device is held by a user. If, at the iteration in box 758, the system determines that one or more client device conditions are not met, the system continues to monitor for the satisfaction of one or more client device conditions in box 758. It is noteworthy that while the system continues to monitor for the satisfaction of one or more client device conditions in box 758, the system can continue to update the global ML model through subsequent iterations of the operations in boxes 752 through 756. If, at the iteration in box 758, the system determines that one or more client device conditions are met, the system proceeds to box 760.

[0083] In box 760, the system transmits an updated global ML model (e.g., the updated global ML model itself, one or more updated global ML layers of the updated global ML model, or one or more updated global weights of the updated global ML model) to one or more client devices. Although not depicted for simplicity, one or more client devices may receive the updated global model only in response to determining that one or more client device conditions are met (e.g., as described with respect to box 360). In response to transmitting the updated global ML model, a client device may cause the updated global ML model to be used to replace the on-device ML model in the on-device memory of the client device, thereby producing an updated on-device ML model. However, the client device (and one or more additional client devices) may continue to generate additional updates for the updated on-device ML model and may maintain an instance of the updated global ML model on a remote system for further updates based on the additional updates.

[0084] Turn now Figure 8 This document depicts a block diagram of an example computing device 810 that can be optionally used to implement one or more aspects of the techniques described herein. In some embodiments, one or more of a client device, a cloud-based automation assistant component, and / or other components may include one or more components of the example computing device 810.

[0085] Computing device 810 typically includes at least one processor 814 that communicates with multiple peripheral devices via a bus subsystem 812. These peripheral devices may include a storage subsystem 824 (including, for example, a memory subsystem 825 and a file storage subsystem 826), a user interface output device 820, a user interface input device 822, and a network interface subsystem 816. The input and output devices allow users to interact with computing device 810. The network interface subsystem 816 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0086] User interface input device 822 may include a keyboard, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen integrated into a display, audio input devices such as a voice recognition system, a microphone, and / or other types of input devices. Generally, the term "input device" is used to encompass all possible types of devices and methods of inputting information onto computing device 810 or a communication network.

[0087] User interface output device 820 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide non-visual displays, such as via an audio output device. Generally, the term "output device" is used to encompass all possible types of devices and the manner in which information is output from computing device 810 to the user or another machine or computing device.

[0088] Storage subsystem 824 stores the programming and data construction capabilities that provide some or all of the functionality of the modules described herein. For example, storage subsystem 824 may include selected aspects for implementing the methods disclosed herein and for implementing... Figure 1 and 2 The logic of the various components described in the text.

[0089] These software modules are typically executed by processor 814 alone or in conjunction with other processors. The memory 825 used in storage subsystem 824 may include multiple memories, including main random access memory (RAM) 830 for storing instructions and data during program execution and read-only memory (ROM) 832 for storing fixed instructions. File storage subsystem 826 can provide persistent storage for program and data files and may include hard disk drives, floppy disk drives and associated removable media, CD-ROM drives, optical drives, or removable media cartridges. Modules implementing the functionality of certain embodiments may be stored by file storage subsystem 826 within storage subsystem 824 or on other machines accessible to processor 814.

[0090] The bus subsystem 812 provides a mechanism for enabling various components and subsystems of the computing device 810 to communicate with each other as intended. Although the bus subsystem 812 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0091] The computing device 810 can be of various types, including workstations, servers, computing clusters, blade servers, server farms, or any other data processing system or computing device. Due to the constantly evolving nature of computers and networks, Figure 8 The description of the computing device 810 depicted herein is intended only as a specific example for illustrating some implementations. Many other configurations of the computing device 810 may have... Figure 8 The computing device depicted in the text has more or fewer components.

[0092] In situations where the systems described herein collect or otherwise monitor personal information about users, or where personal and / or monitoring information may be used, users may have the opportunity to control whether programs or functions collect user information (e.g., information about the user's social networks, social behaviors or activities, occupation, user preferences, or the user's current geographic location), or to control whether and / or how content that may be more relevant to the user is received from content servers. Furthermore, some data may be processed in one or more ways before storage or use to erase personally identifiable information. For example, a user's identity may be processed to make it impossible to determine the user's personally identifiable information, or, where geographic location information is obtained, the user's geographic location may be generalized (e.g., down to the city, zip code, or state level) to make it impossible to determine the user's specific geographic location. Therefore, users can control how information about themselves is collected and / or used.

[0093] In some embodiments, a method implemented by one or more processors of a client device is provided, the method comprising: receiving client data from a user of the client device, the client data being locally generated at the client device; and processing the client data using an on-device machine learning (ML) model locally stored in on-device memory of the client device to generate a predictive output. The on-device machine learning model includes a plurality of on-device ML layers, and the plurality of on-device ML layers includes at least one or more first on-device ML layers and one or more second on-device ML layers. The method further includes generating gradients based on the predictive output using unsupervised learning; generating a first update for one or more first on-device ML layers of the on-device ML model locally stored in on-device memory of the client device based on the gradients; and transmitting the first update to a remote system. Transmitting the first update to the remote system causes the remote system to update a global ML model remotely stored in remote memory of the remote system, the global ML model including at least one or more first global ML layers and one or more second global ML layers. The remote system updating the global ML model including one or more of the second global ML layers is fixed while simultaneously updating one or more first global ML layers based on the first update.

[0094] These and other implementations of this technology may include one or more of the following features.

[0095] In some implementations, the first update transmitted to the remote system may include gradients and an indication to update one or more first global ML layers based on the first update, and one or more first global ML layers of a global ML model stored remotely at the remote system may correspond to one or more first on-device ML layers of an on-device ML model stored locally at the client device. In some versions of those implementations, while one or more of the second global ML layers are fixed, updating one or more first global ML layers based on the first update may include: updating one or more first global ML layers based on gradients and the indication to update one or more first global ML layers based on the first update to generate one or more updated first global ML layers without updating one or more second global ML layers, wherein the one or more updated first global ML layers include one or more updated first global weights for the one or more updated first global ML layers.

[0096] In some implementations, generating a first update for one or more first-device ML layers may include: causing one or more first-device ML layers to be updated based on gradients to generate one or more updated first-device ML layers without updating one or more second-device ML layers, wherein the one or more updated first-device ML layers include one or more updated first-device weights for the one or more updated first-device ML layers.

[0097] In some versions of those implementations, the first update transmitted to the remote system may include one or more updated first on-device ML layers and an instruction to update one or more first global ML layers based on the first update, and one or more first global ML layers of the global ML model stored remotely at the remote system correspond to one or more first on-device ML layers of the on-device ML model stored locally at the client device. In some further versions of those implementations, while one or more of the second global ML layers are fixed, causing one or more first global ML layers to be updated based on the first update may include: based on one or more updated first on-device ML layers and the instruction to update one or more first global ML layers based on the first update, causing one or more first global ML layers to be replaced in remote memory with one or more updated first on-device ML layers, without replacing one or more second global ML layers.

[0098] In additional or alternative versions of those implementations, the first update transmitted to the remote system may include one or more updated first on-device weights for one or more updated first on-device ML layers and an instruction to update one or more first global ML layers based on the first update, and one or more first global ML layers of the global ML model stored remotely at the remote system may correspond to one or more first on-device ML layers of the on-device ML model stored locally at the client device. In some further versions of those implementations, while one or more of the second global ML layers are fixed, updating one or more first global ML layers based on the first update may include: based on one or more updated first on-device weights for one or more updated first on-device ML layers and an instruction to update one or more first global ML layers based on the first update, such that one or more first global weights for one or more first global ML layers are replaced in the remote memory of the remote system using the one or more updated first on-device weights for one or more updated first on-device ML layers, without replacing one or more second global weights for one or more second global ML layers.

[0099] In some implementations, causing the remote system to update the global ML model may further include: while one or more of the first global ML layers are fixed, causing one or more second global ML layers to be updated based on a second update, and the second update may be transmitted to the remote system from an additional client device other than the client device used to generate the first update. In some versions of those implementations, the method may further include receiving an updated global ML model at the client device and from the remote system, the updated global ML model comprising at least one or more updated first global ML layers and one or more updated second global ML layers; and replacing the on-device ML model in the on-device memory of the client device with the updated global ML model.

[0100] In some other versions of those implementations, receiving the updated global ML model may be in response to determining that one or more remote system conditions are met at the remote system. The one or more remote system conditions may include one or more of the following: a specific time of day, a specific day of week, whether the updated global ML model has been updated using a threshold number of updates, or whether the performance of the updated global ML model meets a performance threshold. In still other versions of those implementations, receiving the updated global ML model may also be in response to determining that one or more client device conditions are met at the client device. The one or more client device conditions may include one or more of the following: a specific time of day, a specific day of week, whether the client device is charging, whether the client device is in a charging state with at least a threshold, whether the temperature of the client device is below a temperature threshold, or whether the client device is held by a user.

[0101] In other versions of those implementations, the updated global ML model received at the client device and from the remote system may include one or more of the following: an updated global ML model that, upon receipt, causes the client device to replace the on-device ML model in its on-device memory with the updated global ML model; one or more updated first global ML layers that, upon receipt, cause the client device to replace one or more first on-device ML layers in its on-device memory with the updated first global ML layers; one or more updated second global ML layers that, upon receipt, cause the client device to replace one or more second on-device ML layers in its on-device memory with the updated second global ML layers; one or more updated first global weights for the one or more updated first global ML layers that, upon receipt, cause the client device to replace one or more first local weights for the one or more first on-device ML layers with the updated first global weights in its on-device memory; or one or more updated second global weights for the one or more updated second global ML layers that, upon receipt, cause the client device to replace one or more second local weights for the one or more second on-device ML layers with the updated second global weights in its on-device memory.

[0102] In some implementations, the method may further include identifying a target portion of the client data, the target portion of the client data following a preceding portion of the client data received before the target portion, and the target portion of the client data preceding a following portion of the client data received after the target portion; and masking the target portion of the client data. Processing client data using an on-device ML model to generate a prediction output may include: processing the preceding and following portions of the client data to generate one or more predicted target portions of the client data that are predicted to correspond to the target portion of the client data. In some versions of those implementations, generating gradients based on the prediction output using unsupervised learning may include: comparing the predicted target portion of the client data with the target portion of the client data; and generating gradients based on the comparison between the predicted target portion and the target portion.

[0103] In some implementations, one or more on-device ML layers may include at least a first-device ML layer and a second-device ML layer. The method may further include, before generating gradients based on the prediction output: compressing the first-device ML layer and the second-device ML layer into one or more first-device ML layers, wherein a first update for the one or more on-device ML layers is a first shared update for the first-device ML layer and the second-device ML layer. In some versions of those implementations, while one or more of the second global ML layers are fixed, updating one or more first global ML layers based on the first update may include: using the first update to update the first global ML layer corresponding to the first-device ML layer; and using the first update to update the second global ML layer corresponding to the second-device ML layer. In additional or alternative versions of those implementations, one or more second-device ML layers may include at least a second-device ML layer and a third-device ML layer, which may be compressed at an additional client device into one or more second-device ML layers at the additional client device, and a second update generated locally at the additional client device may be a shared second update for the second-device ML layer and the third-device ML layer. In additional or alternative versions of those implementations, enabling a remote system to update the global ML model may further include: updating one or more second global ML layers based on a second update while one or more of the first global ML layers are fixed, and enabling one or more of the second global ML layers based on a second update while one or more of the first global ML layers are fixed may include: using the second update to update the second global ML layer corresponding to the ML layer on the second device; and using the second update to update the third global ML layer corresponding to the ML layer on the third device.

[0104] In some implementations, a method is provided implemented by one or more processors of a client device, the method comprising: receiving, at a remote system, a first update from a user's client device for a global machine learning (ML) model remotely stored at the remote system. The global ML model includes multiple global ML layers, and the first update for the global ML model applies only to one or more first global ML layers of the global ML model. The method further comprises receiving, at the remote system, a second update from an additional client device of an additional user for a global ML model remotely stored in remote memory of the remote system. The second update for the global ML model applies only to one or more second global ML layers of the global ML model, and the one or more second global ML layers of the global ML model are distinct from the one or more first global ML layers of the global ML model. The method further comprises: at least based on the first update received from the client device and the second update received from the additional client device, causing the global ML model to update to generate an updated global ML model; and responding to determining that one or more conditions are met: the updated global ML model is transmitted to one or more of: the client device, the additional client device, or one or more other additional client devices.

[0105] These and other implementations of this technology may include one or more of the following features.

[0106] In some implementations, updating the global ML model to generate an updated global ML model may include: updating only one or more first global ML layers based on a first update to generate one or more updated first global ML layers while one or more second global ML layers are fixed; updating only one or more second global ML layers based on a second update to generate one or more updated second global ML layers while one or more first global ML layers are fixed; and combining one or more updated first global ML layers and one or more second global ML layers to generate an updated global ML model.

[0107] In some versions of those implementations, having one or more second global ML layers fixed while updating only one or more first global ML layers based on a first update, and having one or more second global ML layers fixed while updating only one or more second global ML layers based on a second update, can occur during the same iteration of updating the global ML model. In additional or alternative versions of those implementations, having one or more second global ML layers fixed while updating only one or more first global ML layers based on a first update can occur during the first iteration of updating the global ML model. Having one or more first global ML layers fixed while updating only one or more second global ML layers based on a second update can occur during the second iteration of updating the global ML model. The second iteration of updating the global ML model can be different from the first iteration of updating the global ML model.

[0108] In some implementations, causing the global ML model to be updated to generate an updated global ML model based at least on a first update received from a client device and a second update received from an additional client device can be in response to determining that one or more update conditions are met. The one or more update conditions may include one or more of the following: a specific time of day, a specific day of week, a threshold number of updates required to update the global ML model, or a threshold number of dissimilar updates required to update the global ML model.

[0109] In some implementations, a remote system may correspond to a remote server or a cluster of remote servers.

[0110] Various implementations may include a non-transitory computer-readable storage medium whose storage can be implemented by one or more processors (e.g., a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), and / or tensor processing unit (TPU)) to perform one or more methods such as those described herein. Other implementations may include an automation assistant client device (e.g., a client device including at least an automation assistant interface for interfacing with a cloud-based automation assistant component), which includes a processor operable to execute stored instructions for performing one or more methods such as those described herein. Yet another implementation may include a system of one or more servers including one or more processors operable to execute stored instructions for performing one or more methods such as those described herein.

Claims

1. A method implemented by one or more processors of a client device, the method comprising: Receive client data from the user of the client device, the client data being generated locally on the client device; The client data is processed using an on-device machine learning (ML) model stored locally in the on-device memory of the client device to generate a predictive output, wherein the on-device machine learning model includes multiple on-device ML layers, and wherein the multiple on-device ML layers include at least one or more first on-device ML layers and one or more second on-device ML layers; Receive a scheduling signal from a remote system, the scheduling signal instructing the client device to generate a first update for the ML layer on the one or more first devices; Unsupervised learning is used to generate gradients specific to the ML layers on the one or more first devices based on the predicted output; Based on the gradient, generate the first update for the one or more first on-device ML layers of the on-device ML model stored locally on the client device in the on-device memory; and The first update is transmitted to a remote system, wherein transmitting the first update to the remote system causes the remote system to update a global ML model remotely stored in a remote memory of the remote system, wherein the global ML model includes at least one or more first global ML layers and one or more second global ML layers, and wherein causing the remote system to update one or more of the second global ML layers included in the global ML model is fixed while causing the one or more first global ML layers to be updated based on the first update.

2. The method according to claim 1, wherein, The first update transmitted to the remote system includes the gradient and an indication to update the one or more first global ML layers based on the first update, wherein the one or more first global ML layers of the global ML model remotely stored at the remote system correspond to the one or more first on-device ML layers of the on-device ML model locally stored at the client device.

3. The method according to claim 2, wherein, While one or more of the second global ML layers are fixed, updating the one or more first global ML layers based on the first update includes: Based on the gradient and an indication to update the one or more first global ML layers based on the first update, the one or more first global ML layers are updated based on the gradient to generate one or more updated first global ML layers without updating the one or more second global ML layers, the one or more updated first global ML layers including one or more updated first global weights for the one or more updated first global ML layers.

4. The method according to claim 1, wherein, Generating the first update for the ML layer on the one or more first devices includes: This allows updating the one or more ML layers on the first device based on the gradient to generate one or more updated ML layers on the first device without updating the one or more ML layers on the second device, wherein the one or more updated ML layers on the first device include one or more updated weights on the first device for the one or more updated ML layers on the first device.

5. The method according to claim 4, wherein, The first update transmitted to the remote system includes the one or more updated first on-device ML layers and an indication to update the one or more first global ML layers based on the first update, wherein the one or more first global ML layers of the global ML model remotely stored at the remote system correspond to the one or more first on-device ML layers of the on-device ML model locally stored at the client device.

6. The method according to claim 5, wherein, While one or more of the second global ML layers are fixed, updating the one or more first global ML layers based on the first update includes: Based on the one or more updated first on-device ML layers and the instruction to update the one or more first global ML layers based on the first update, the one or more first global ML layers are replaced in the remote memory using the one or more updated first on-device ML layers, without replacing the one or more second global ML layers.

7. The method according to claim 4, wherein, The first update transmitted to the remote system includes the first on-device weights of the one or more on-device ML layers for the one or more updates and an indication to update the one or more first global ML layers based on the first update, wherein the one or more first global ML layers of the global ML model remotely stored at the remote system correspond to the one or more first on-device ML layers of the on-device ML model locally stored at the client device.

8. The method according to claim 7, wherein, While one or more of the second global ML layers are fixed, updating the one or more first global ML layers based on the first update includes: Based on the one or more updated first on-device weights for the one or more updated first on-device ML layers and based on the instruction to update the one or more first global ML layers based on the first update, such that in the remote memory of the remote system, the one or more updated first on-device weights for the one or more updated first on-device ML layers are used to replace one or more first global weights for the one or more first global ML layers, without replacing one or more second global weights for the one or more second global ML layers.

9. The method according to claim 1, wherein, The method of causing the remote system to update the global ML model further includes: while one or more of the first global ML layers are fixed, causing the one or more second global ML layers to be updated based on a second update, wherein the second update is transmitted to the remote system from an additional client device other than the client device used to generate the first update.

10. The method of claim 9, further comprising: The client device receives an updated global ML model from the remote system, the updated global ML model including at least one or more updated first global ML layers and one or more updated second global ML layers; as well as The updated global ML model is used to replace the on-device ML model in the on-device memory of the client device.

11. The method according to claim 10, wherein, Receiving the updated global ML model is in response to determining that one or more remote system conditions are met at the remote system, wherein the one or more remote system conditions include one or more of the following: a specific time of day, a specific day of week, whether the updated global ML model has been updated with a threshold number of updates, or whether the performance of the updated global ML model meets a performance threshold.

12. The method according to claim 11, wherein, The global ML model receiving the update also responds to determining that one or more client device conditions are met at the client device, wherein the one or more client device conditions include one or more of the following: a specific time of day, a specific day of week, whether the client device is charging, whether the client device has at least a threshold charging state, whether the temperature of the client device is less than a temperature threshold, or whether the client device is held by a user.

13. The method according to any one of claims 10 to 12, wherein, The updated global ML model received at the client device from the remote system includes one or more of the following: The updated global ML model, when received, causes the client device to replace the on-device ML model in the on-device memory of the client device with the updated global ML model; The one or more updated first global ML layers, when received, cause the client device to replace the one or more first on-device ML layers in the client device's on-device memory with the one or more updated first global ML layers; The one or more updated second global ML layers, when received, cause the client device to replace the one or more second on-device ML layers in the client device's on-device memory with the one or more updated second global ML layers; For one or more updated first global weights of the one or more updated first global ML layers, when the one or more updated first global weights of the one or more updated first global ML layers are received, the client device replaces one or more first local weights of the one or more first on-device ML layers with the one or more updated first global weights in the on-device memory of the client device. or When the one or more updated second global weights of the one or more updated second global ML layers are received, the client device replaces the one or more second local weights of the one or more second on-device ML layers with the one or more updated second global weights in the on-device memory of the client device.

14. The method according to claim 1, further comprising: Identify the target portion of the client data, wherein the target portion of the client data is after a preceding portion of the client data received before the target portion, and the target portion of the client data is before a following portion of the client data received after the target portion; The target portion of the client data is masked; as well as The process of using an on-device ML model to process the client data to generate the prediction output includes: processing the preceding portion and the following portion of the client data to generate one or more of a prediction target portion of the client data that is predicted to correspond to the target portion of the client data.

15. The method according to claim 14, wherein, Generating the gradient based on the predicted output using unsupervised learning includes: Compare the predicted target portion of the client data with the target portion of the client data; and The gradient is generated by comparing the predicted target portion with the target portion.

16. The method according to claim 1, wherein, The one or more ML layers on the first device include at least ML layers on the first device and ML layers on the second device, and the method further includes: Before generating the gradient based on the predicted output: The ML layer on the first device and the ML layer on the second device are compressed into one or more ML layers on the first device, wherein the first update for the ML layer on the one or more devices is a first shared update for the ML layer on the first device and the ML layer on the second device.

17. The method according to claim 16, wherein, While one or more of the second global ML layers are fixed, updating the one or more first global ML layers based on the first update includes: The first update is used to update the first global ML layer corresponding to the ML layer on the first device; and The first update is used to update the second global ML layer corresponding to the ML layer on the second device.

18. The method according to claim 16, wherein, The one or more second-device ML layers include at least a second-device ML layer and a third-device ML layer, wherein the second-device ML layer and the third-device ML layer are compressed at the additional client device into the one or more second-device ML layers at the additional client device, and wherein the second update generated locally at the additional client device is a second shared update for the second-device ML layer and the third-device ML layer.

19. The method according to claim 18, wherein, The method of causing the remote system to update the global ML model further includes: updating the one or more second global ML layers based on the second update while one or more of the first global ML layers are fixed, wherein updating the one or more of the second global ML layers based on the second update while one or more of the first global ML layers are fixed includes: The second update is used to update the second global ML layer corresponding to the ML layer on the second device; and The second update is used to update the third global ML layer corresponding to the ML layer on the third device.

20. A method implemented by one or more processors of a remote system, the method comprising: A first scheduling signal is provided from the remote system, the first scheduling signal instructing the client device to generate a first update for one or more machine learning (ML) layers on a first device; A second scheduling signal is provided from the remote system, the second scheduling signal instructing an attached client device to generate a second update for the ML layer on one or more second devices; The system receives a first update from the user's client device regarding the global ML model remotely stored on the remote system. The global ML model includes multiple global ML layers, and Wherein, the first update for the global ML model is only applied to one or more first global ML layers among the plurality of global ML layers of the global ML model; At the remote system, the second update for the global ML model, which is remotely stored in a remote memory of the remote system, is received from the additional client device of the additional user. Wherein, the second update for the global ML model applies only to one or more second global ML layers among the plurality of global ML layers of the global ML model, and Wherein, the one or more second global ML layers of the global ML model are different from the one or more first global ML layers of the global ML model; Based at least on the first update received from the client device and the second update received from the additional client device, the global ML model is updated to generate an updated global ML model; and In response to determining that one or more conditions are met, The updated global ML model is transferred to one or more of the following: the client device, the additional client device, or one or more other additional client devices.

21. The method according to claim 20, wherein, Updating the global ML model to generate the updated global ML model includes: While the one or more second global ML layers are fixed, the one or more first global ML layers are updated only based on the first update to generate one or more updated first global ML layers; While the one or more first global ML layers are fixed, the one or more second global ML layers are updated only based on the second update to generate one or more updated second global ML layers; and The one or more updated first global ML layers and the one or more second global ML layers are combined to generate the updated global ML model.

22. The method according to claim 21, wherein, The updating of only the one or more first global ML layers based on the first update while the one or more first global ML layers are fixed, and the updating of only the one or more second global ML layers based on the second update while the one or more first global ML layers are fixed, occur during the same iteration of updating the global ML model.

23. The method according to claim 21, wherein, While the one or more second global ML layers are fixed, updating only the one or more first global ML layers based on the first update occurs during a first iteration of updating the global ML model, wherein updating only the one or more second global ML layers based on the second update occurs during a second iteration of updating the global ML model, and wherein the second iteration of updating the global ML model is different from the first iteration of updating the global ML model.

24. The method of claim 20, wherein, The global ML model is updated to generate an updated global ML model based at least on the first update received from the client device and the second update received from the additional client device in response to determining that one or more update conditions are satisfied, wherein the one or more update conditions include one or more of the following: a specific time of day, a specific day of week, whether there is a threshold number of updates for updating the global ML model, or whether there is a threshold number of dissimilar updates for updating the global ML model.

25. The method according to claim 20, wherein, The remote system corresponds to a remote server or a remote server cluster.

26. A system comprising: At least one processor; as well as A memory for storing instructions, which, when executed, cause the at least one processor to perform the method corresponding to any one of claims 1 to 25.

27. A non-transitory computer-readable storage medium storing instructions, which, when executed, cause at least one processor to perform the method corresponding to any one of claims 1 to 25.

Images