Federal learning with base model distillation

Abstract

A method and system for training a neural network using federated learning. The server-maintained machine learning model is sent from the server to the client, resulting in a local machine learning model. At each client, the model is trained with local data to determine respective cross entropy loss and distillation loss based on the base model. At each client, a respective weight of each local machine learning model is updated based on the loss. Transmitting the updated weight to a server without transmitting locally stored data; they are then aggregated and transmitted back to the client. At each client, the local machine learning model is updated with the aggregated updated weights.

Description

Technical Field

[0001] The present disclosure relates to methods and systems for federated learning utilizing a base model in a machine learning model. Background Art

[0002] Federated learning (a type of collaborative learning) is a machine learning technique that trains machine learning algorithms via multiple independent nodes, each using its own dataset. Federated learning aims to train global machine learning algorithms, such as deep neural networks, based on multiple local datasets contained in local nodes (also called clients) without explicitly exchanging data samples. The learning task is solved by a federation of participating devices coordinated by a central server. Each participating device (client) has a local training dataset that is not uploaded to the server or shared with other clients. Instead, each client computes an update to the current global model maintained by the server. The client transmits this update (not the training dataset) to the server. The server aggregates the received updates to update the global model. The resulting shared model can be trained by learning from the training of the client, allowing users to benefit from training the shared model from the client's data without transmitting or centrally storing data from the client. This is particularly useful in situations where the exchange of sensitive or personal data (e.g., medical information, IoT devices, personal devices such as smartphones) is prohibited. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 A system for training a neural network according to an embodiment is shown.

[0004] Figure 2 A computer-implemented method for training and utilizing a neural network according to an embodiment is shown.

[0005] Figure 3A An example diagram or layout of a federated learning system including a base model according to an embodiment is shown.

[0006] Figure 3B A flowchart of a first exemplary method of training a neural network via client-side federated learning according to a first embodiment is shown.

[0007] Figure 3C A flowchart of a second exemplary method for training a neural network via server-side federated learning according to the second embodiment is shown.

[0008] Figure 4 A schematic diagram of a deep neural network with an input layer, multiple hidden layers, and nodes in an output layer is shown according to an embodiment.

[0009] Figure 5Depicted is a schematic diagram of the interaction between a computer-controlled machine and a control system according to an embodiment.

[0010] Figure 6 Depicted is a device configured to control a vehicle according to an embodiment Figure 5 Schematic diagram of a control system of a vehicle, wherein the vehicle can be a partially autonomous vehicle, a fully autonomous vehicle, a partially autonomous robot, or a fully autonomous robot.

[0011] Figure 7 Depicts a system configured to control a manufacturing machine (such as a punch cutter, a cutting machine, or a barrel drill) of a manufacturing system (such as part of a production line) Figure 5 Schematic diagram of the control system.

[0012] Figure 8 A device configured to control a power tool (such as a drill or driver) having at least a partially autonomous mode is depicted. Figure 5 Schematic diagram of the control system.

[0013] Fig. 9 Depicting a device configured to control an automated personal assistant Figure 5 Schematic diagram of the control system.

[0014] Fig.10 Depicts a device configured to control a monitoring system, such as a control access system or a surveillance system Figure 5 Schematic diagram of the control system.

[0015] Fig.11 A device configured to control an imaging system (e.g., an MRI device, an X-ray imaging device, or an ultrasound device) is depicted. Figure 5 Schematic diagram of the control system. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various alternative forms. The drawings are not necessarily drawn to scale; some features may be exaggerated or minimized to show the details of a particular component. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to adopt various embodiments in various ways. As will be understood by those of ordinary skill in the art, the various features illustrated and described with reference to any one of the figures may be combined with the features shown in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combination of features shown provides representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be required for specific applications or implementations.

[0017] As used herein, "a", "an" and "the" refer to both singular and plural references unless the context clearly dictates otherwise. For example, a "processor" programmed to perform various functions refers to one processor programmed to perform each of the functions, or more than one processor programmed uniformly to perform each of the various functions.

[0018] The present disclosure relates to federated learning. References herein to a "server" of a federated learning system refer to a centralized server(s) that communicates with a client to exchange data therewith. For example, one or more servers in the cloud or remote from an end user may constitute a server. References herein to a "client" of a federated learning system refer to a local node or device corresponding to a particular end user. For example, a smartphone, an IoT device, etc. would constitute a client. Each of the server and the client may utilize a corresponding computer system for machine learning training, such as those described herein.

[0019] Federated learning (also known as collaborative learning) is a machine learning technique that trains a machine learning algorithm via multiple independent sessions, each using its own dataset. Federated learning aims to train a global machine learning model (e.g., a deep neural network) based on multiple local datasets contained in local nodes (also known as clients) without explicitly exchanging data samples. The learning task is solved by a federation of participating devices coordinated by a central server. Each participating device (client) has a local training dataset that is not uploaded to the server. Instead, each client computes an update to the current global model maintained by the server. The client transmits the update (not the training dataset) to the server. The server aggregates the received updates to update the global model. The resulting shared model can be trained by learning from the training of the clients, allowing users to benefit from a shared model trained from the client's data without transmitting or centrally storing the data from the clients.

[0020] Federated learning is particularly useful in situations where the exchange of sensitive or personal data is prohibited (e.g., medical information, IoT devices, personal devices such as smartphones). Take a smartphone as a simple everyday example. When people type an email or text message, they may use an autocorrect or autocomplete feature that will automatically correct people's spelling mistakes or suggest corrections or additions to the text. Such systems typically rely on machine learning models that are trained based on the user's previous words and may vary from user to user. Due to the sensitivity of the information used, the raw data of the words entered by the user is not sent to a central server to train the model. Instead, the user's smartphone can receive the base model from the server and the model can be trained locally to perform autocorrection in a way that is tailored to the word history of that specific user. Information about how the model was trained locally can be sent to the server without sending the raw data (e.g., the user's words and / or corrections) to the server.

[0021] Federated learning is a technique that enables data to remain stored locally on client devices, while the server acts as a coordinator of the learning process, aggregating the learned information and synchronizing it with clients (e.g., users, AI tasks, etc.). However, due to issues such as data heterogeneity, distribution variations, and the fact that communications may be unreliable and even in the event of hardware failures, the efficiency and robustness of the system may be reduced. In addition, various federated learning architectures have been proposed, such as a single global model and a peer-to-peer model with individual models, some of which include personalized versions.

[0022] FedAvg is a federated learning technique in which a distributed machine learning algorithm allows clients to collaboratively train a global server model while keeping their data locally stored and private. It works by sending an initial model from the server to the clients, aggregating the client updates into a global model by averaging their updates, and repeating this process until convergence is achieved. FedAvg achieves updating the global model by aggregating the knowledge and insights of multiple clients. Typical federated learning systems suffer from problems such as performance degradation and robustness degradation, especially when faced with issues such as data heterogeneity and distribution changes. Recently, base models pre-trained on large amounts of data have been widely adapted as the initialization of model parameters. In cases where clients have access to their own base models but have strong privacy concerns, their maximum achievable performance will be limited.

[0023] In view of the above challenges and the diversity of federated learning architectures and frameworks, this disclosure aims to address these issues by proposing a novel federated learning framework that utilizes a base model local to the client to improve its performance while using a smaller proxy model to coordinate and synchronize information between clients. While others have suggested using a base model within the client to improve its personalization capabilities, they suggest customizing the base model directly by adding adaptation layers or hints. They do not use the knowledge distilled from the base model to guide the training of a smaller-sized proxy model.

[0024] To address the issues of insufficient representation and computational efficiency, the present disclosure proposes embodiments including the following methods. Instead of using the same base model used in the literature, the client is free to choose its own base model. By consistently distilling the performance of the base model to a smaller student model in the client terminal and updating the student model with aggregated weights from the server, our framework leverages prior knowledge from pre-trained base models while retaining information presented by the client population. Clients distill knowledge from pre-trained base models of their own choice, fine-tune on local data, and have access to aggregated knowledge in a federated learning setting. Therefore, our framework strikes a balance between an efficient communication setting across servers and clients while achieving privacy by not directly exposing the base model and a high level of performance and robustness achievable using the base model.

[0025] Federated learning systems can take advantage of Figure 1-2 The machine learning training and process shown in . Figure 1 A system 100 for training a neural network (e.g., a deep neural network) is shown. The neural network being trained can reside on a server or a client. In other words, both the server and the client can utilize Figure 1 The system 100 may include an input interface for accessing training data 102 of a neural network. For example, Figure 1 As shown in , the input interface can be constituted by a data storage interface 104, which can access the training data 102 from a data storage device 106. For example, the data storage interface 104 can be a memory interface or a persistent storage interface, such as a hard disk or SSD interface, but can also be a personal, local or wide area network interface, such as a Bluetooth, Zigbee or Wi-Fi interface or an Ethernet or fiber optic interface. The data storage device 106 can be an internal data storage device of the system 100, such as a hard disk drive or SSD, but can also be an external data storage device, such as a network accessible data storage device.

[0026] In some embodiments, the data storage device 106 may also include a data representation 108 of an untrained version of the neural network, which data representation 108 may be accessed by the system 100 from the data storage device 106. However, it should be understood that the training data 102 and the data representation 108 of the untrained neural network may also be accessed from different data storage devices, for example, via different subsystems of the data storage interface 104. Each subsystem may be of the type described above for the data storage interface 104. In other embodiments, the data representation 108 of the untrained neural network may be generated internally by the system 100 based on the design parameters of the neural network, and thus may not be explicitly stored on the data storage device 106. The system 100 may also include a processor subsystem 110, which may be configured to provide an iterative function to replace a layer stack of the neural network to be trained during operation of the system 100. Here, the layers of the replaced layer stack may have mutually shared weights and may receive the output of the previous layer as input, or, for the first layer of the layer stack, receive the initial activation and a portion of the input of the layer stack. The processor subsystem 110 may also be configured to iteratively train the neural network using the training data 102. Here, the training iterations performed by the processor subsystem 110 may include a forward propagation portion and a backward propagation portion. The processor subsystem 110 may be configured to perform the forward propagation portion by: determining an equilibrium point of the iterative function at which the iterative function converges to a fixed point, in addition to other operations that may be performed to define the forward propagation portion, wherein determining the equilibrium point includes using a numerical root finding algorithm to find a root solution of the iterative function minus its input; and providing the equilibrium point as a substitute for the output of the layer stack in the neural network. The system 100 may also include an output interface for outputting a data representation 112 of the trained neural network; the data may also be referred to as trained model data 112. For example, as also Figure 1 As shown in , the output interface can be composed of a data storage interface 104, which in these embodiments is an input / output ("IO") interface, via which the trained model data 112 can be stored in the data storage device 106. For example, the data representation 108 defining the "untrained" neural network can be at least partially replaced by the data representation 112 of the trained neural network during or after training, because the parameters of the neural network (such as weights, hyperparameters, and other types of parameters of the neural network) can be adapted to reflect the training on the training data 102. This is also Figure 1106 by reference numerals 108, 112 referring to the same data records on the data storage device 106. In other embodiments, the data representation 112 may be stored separately from the data representation 108 defining the "untrained" neural network. In some embodiments, the output interface may be separate from the data storage interface 104, but may generally be of the type described above for the data storage interface 104.

[0027] The structure of system 100 is one example of a system that may be used to train the models utilized by the federated learning systems described herein. Figure 2 Additional structures for operating and training these machine learning models are shown in .

[0028] Figure 2 A federated learning system 200 configured to execute and train a machine learning model (e.g., a neural network or a deep neural network) as described herein is depicted. System 200 can be implemented to perform the federated learning process described herein. System 200 can include at least one computing system 202. Computing system 202 can be a client device (such as a smartphone, an IoT device, a medical device, or a computer such as described herein with reference to the following). Figure 6-11 206. The computing system 202 may be part of an embedded computer system, a system on a chip (SoC), a single board computer system (SBC) (such as, for example, a computer on a module (COM) or a system on a module (SOM)), a laptop computer, a personal device (such as a smart phone or tablet), a personal device network, or a combination of these. The computing system 202 may include at least one processor 204, which is operably connected to a memory unit 208. The processor 204 may include one or more integrated circuits that implement the functions of a central processing unit (CPU) 206. The CPU 206 may be a commercially available processing unit that implements an instruction set such as one of the x86, ARM, Power, or MIPS instruction set families. During operation, the CPU 206 may execute stored program instructions retrieved from the memory unit 208. The stored program instructions may include software that controls the operation of the CPU 206 to perform the operations described herein. In some examples, processor 204 may be a system on a chip (SoC) that integrates the functionality of CPU 206, memory unit 208, network interface, and input / output interface into a single integrated device. Computing system 202 may implement an operating system to manage various aspects of operation. Figure 2 One processor 204, one CPU 206, and one memory 208 are shown, but of course multiple processors 204, multiple CPUs 206, and multiple memories 208 may be utilized throughout the system.

[0029] The memory unit 208 may include volatile memory and non-volatile memory for storing instructions and data. Non-volatile memory may include solid-state memory, such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when the computing system 202 is deactivated or powered off. Volatile memory may include static and dynamic random access memory (RAM) that stores program instructions and data. For example, the memory unit 208 may store a machine learning model 210 or algorithm, a training data set 212 for the machine learning model 210, and an original source data set 216.

[0030] The computing system 202 may include a network interface device 222 configured to provide communications with external systems and devices. For example, the network interface device 222 may include a wired and / or wireless Ethernet interface defined by the Institute of Electrical and Electronics Engineers (IEEE) 802.11 series of standards. The network interface device 222 may include a cellular communication interface for communicating with a cellular network (e.g., 3G, 4G, 5G). The network interface device 222 may be further configured to provide a communication interface to an external network 224 or a cloud so that a device (e.g., a client device) executing the computing system 202 can communicate with the server 230.

[0031] The external network 224 may be referred to as the World Wide Web or the Internet. The external network 224 may establish a standard communication protocol between computing devices. The external network 224 may allow information and data to be easily exchanged between computing devices and the network.

[0032] One or more servers 230 may communicate with an external network 224. Each server may include a computing system, such as computing system 202, such that server 230 is configured to perform machine learning and train a neural network. Of course, consistent with the spirit of the present disclosure, some personal or sensitive original source data 216 originating from a particular client device may not be transmitted to server 230, and thus the original source data at the server may not exist or may be completely independent of the original source data on the computing system 202 of the client device. During operation of the federated learning system, as will be described below, computing systems 202 associated with client devices may exchange portions of training data 212, but not original source data 216 or any personal data, in order to protect the privacy of any sensitive personal data residing on the client device. Server 230 may then access this information via a connection to network 224 and update its stored model on the server side.

[0033] The computing system 202 may include an input / output (I / O) interface 220, which may be configured to provide digital and / or analog input and output. The I / O interface 220 is used to transfer information between an internal storage device and an external input and / or output device (e.g., an HMI device). The I / O 220 interface may include an associated circuit or bus network for transferring information to or between (one or more) processors and storage devices. For example, the I / O interface 220 may include digital I / O logic lines that can be read or set by (one or more) processors, handshake lines for supervising data transmission via I / O lines, timing and counting facilities, and other structures known to provide such functions. Examples of input devices include keyboards, mice, cameras, sensors, etc. Examples of output devices include monitors, screens, printers, speakers, etc. The I / O interface 220 may include additional serial interfaces (e.g., universal serial bus (USB) interfaces) for communicating with external devices. The I / O interface 220 may be referred to as an input interface (because it transmits data from an external input, such as a sensor) or an output interface (because it transmits data to an external output, such as a display).

[0034] The computing system 202 may include a human-machine interface (HMI) device 218, which may include any device that enables the system 200 to receive control inputs. Examples of input devices may include human-machine interface inputs, such as keyboards, mice, touch screens, voice input devices (e.g., microphones), and other similar devices. The computing system 202 may include a display device 232. The computing system 202 may include hardware and software for outputting graphics and text information to the display device 232. The display device 232 may include an electronic display screen, a projector, a printer, a speaker, or other devices suitable for displaying information to a user or operator. The computing system 202 may be further configured to allow interaction with a remote HMI and a remote display device via a network interface device 222.

[0035] System 200 may be implemented using one or more computing systems. Although the example depicts a single computing system 202 that implements all of the described features, it is intended that the various features and functions may be separated and implemented by multiple computing units that communicate with each other. Specifically, a client device may implement computing system 202, and server 230 may also include its own computing system 202. The specific system architecture selected may depend on a variety of factors.

[0036] The federated learning system 200 may implement a machine learning algorithm 210 that is configured to analyze a raw source data set 216. The raw source data set 216 may include raw or unprocessed sensor data that may represent an input data set for the machine learning system. The raw source data set 216 may include videos, video segments, images, text-based information, audio or human speech, time series data (e.g., pressure sensor signals that vary over time), and raw or partially processed sensor data (e.g., a radar map of an object). The raw source data set 216 may include sensitive data or personal data that has a high security necessity, and therefore the raw source data set 216 may not be transmitted from the client device to the server 230. Reference Figure 5-11 Several different input examples are shown and described. In some examples, the machine learning algorithm 210 can be a neural network algorithm (e.g., a deep neural network) designed to perform a predetermined function. For example, the neural network algorithm can be configured in an automotive application to recognize street signs or pedestrians in an image. The neural network algorithm can be configured to automatically correct text or speech based on the context of words from an individual.

[0037] The computing system 202 may store a training data set 212 for the machine learning algorithm 210. The training data set 212 may represent a previously constructed data set used to train the machine learning algorithm 210. The training data set 212 may be used by the machine learning algorithm 210 to learn weighting factors associated with the neural network algorithm. The training data set 212 may include a source data set having a corresponding output or result that the machine learning algorithm 210 attempts to reproduce via the learning process.

[0038] The machine learning algorithm 210 may operate in a learning mode using the training data set 212 as input. The machine learning algorithm 210 may be executed in multiple iterations using data from the training data set 212. In each iteration, the machine learning algorithm 210 may update the internal weighting factors based on the achieved results. For example, the machine learning algorithm 210 may compare the output results (e.g., a reconstructed or supplemented image in the case where image data is input) with the results included in the training data set 212. Since the training data set 212 includes the expected results, the machine learning algorithm 210 may determine when the performance is acceptable. After the machine learning algorithm 210 achieves a predetermined performance level (e.g., 100% consistent with the results associated with the training data set 212) or converges, the machine learning algorithm 210 may be executed using data that is not in the training data set 212. It should be understood that in the present disclosure, "convergence" may mean that a set (e.g., predetermined) number of iterations have occurred, or that the residual is small enough (e.g., the change in the approximate probability in the iteration is less than a threshold), or other convergence conditions. The trained machine learning algorithm 210 may be applied to a new data set to generate annotated data.

[0039] The machine learning algorithm 210 may be configured to identify specific features in the original source data 216. The original source data 216 may include multiple instances or input data sets that need to supplement the results. For example, the machine learning algorithm 210 may be configured to identify the presence of a person in a video image and annotate the event (occurrence). The machine learning algorithm 210 may be programmed to process the original source data 216 to identify the presence of specific features. The machine learning algorithm 210 may be configured to identify the features in the original source data 216 as predetermined features (e.g., specific words in the case where the input is text or spoken words). The original source data 216 may be derived from various sources. For example, the original source data 216 may be actual input data collected by the machine learning system. The original source data 216 may be machine generated for testing the system. As an example, the original source data 216 may include original images or videos from a camera, spoken words from a microphone, or typed or written words from a keyboard or touch screen, etc.

[0040] Federated learning (FL) is a distributed training paradigm that enables clients dispersed in the real world to collaboratively learn a global model while protecting data privacy. These clients usually have strict constraints on their communication bandwidth and inference speed. However, heterogeneous data distribution among clients and the increased computational cost to maintain high performance without increasing the risk of data privacy exposure in this setting are key bottlenecks that hinder the application of FL in practice. Typically, heterogeneous data distribution among clients leads to client drift, where the global model cannot reach its optimal point due to the divergent behavior between local models. When coupled with small-scale models to match the inference speed and communication bandwidth constraints, the achievable performance and robustness of both global and local models are degraded overall. These problems are further exacerbated by data privacy concerns when information about local data can be re-engineered from standard model updates. To address these issues, this paper proposes a framework that leverages the performance of the base model to help small-scale surrogate models correct their local biases and improve the overall performance of the global model in a non-IID (independent and identically distributed) setting, while maintaining differential privacy guarantees and fixed communication and inference budgets. A comprehensive evaluation of the framework in various settings is presented, highlighting its effectiveness in improving the performance of small models under strong non-IID conditions. In addition, the impact of fine-tuning the base model on local data and how it detracts from obtaining a better global model under strongly heterogeneous data distributions is presented.

[0041] Federated learning (FL) is a popular training paradigm in large-scale machine learning that can address collaborative learning across distributed clients, similar to real-world settings, while keeping the data stored therein private. A typical FL framework can include a central server that coordinates the training of shared models by periodically averaging local models from multiple clients, each of which can only use data stored locally on the client for training. In practice, due to their deployment in real-world environments, clients impose strict constraints on inference speed and communication bandwidth, which in turn restricts them to using small-scale models. When combined with heterogeneous data distribution and data confidentiality constraints, these issues pose significant challenges to the overall achievable performance and robustness.

[0042] Base models present a potential solution to poor performance under heterogeneous data distributions, with the known advantages of being transferable representations across a wide range of downstream tasks and being highly robust to distribution changes. Several recent approaches explore fine-tuning transformer-based architectures and base models in a federated setting. However, in some cases, federated training of base models leads to worse performance than pure local training under extreme non-IID conditions. In addition, existing methods do not discuss the relatively large increase in inference time when switching to base models instead of small-scale alternatives such as EfficientNet, MobileNet, etc. In addition, most federated training algorithms assume that the full set or partial subset of weights are exchanged with the server, which does not comply with privacy constraints, as it has been previously shown that by capturing gradient updates or weights, malicious users can infer multiple properties of the original data distribution; and subsequently use this information in an adversarial manner.

[0043] Considering heterogeneous data distribution, inference speed, and privacy, a novel federated training framework is proposed to improve the overall performance of small-scale models by distilling information from base models stored locally on clients. The framework allows clients flexibility in the number and type of base models stored in each client, which enables small-scale models to benefit from the improved performance and robustness for non-IID distributions as well as privacy-based optimization schemes, both of which are known to degrade the overall performance. In addition, the framework eliminates the computational burden of running large base models during inference by relying on small-scale models to measure the overall performance of the client. By treating the base models as local models, the small-scale models as proxies that can be shared, and training them with privacy-guaranteed optimization techniques, data privacy can be maintained.

[0044] Unique aspects of the framework include:

[0045] First, the framework is the first to exploit the base model in a federated learning setting under a constrained computational budget. The federated training framework raises the upper bound on the performance and robustness achievable in small-scale client models under the strict non-IID data setting.

[0046] Then, by distilling knowledge from the base model, the framework can overcome the overall performance degradation observed when using optimization schemes under privacy guarantee.

[0047] Finally, the framework allows for flexible selection of base models stored locally, which enables customers to benefit from their choice of base model and scale of compute resources without sacrificing the privacy or inference speed of their models.

[0048] Federated Learning

[0049] Federated learning is a distributed machine learning approach that enables multiple clients to collaboratively train a shared model while keeping their data local, thus keeping the data private. Consider using FedAvg to provide an efficient solution to collaboratively train a global model using periodic model averaging. Federated learning is then adopted in a range of application areas including computer vision, natural language processing, speech recognition, healthcare, IoT, etc. Recently, efforts have been made to address some key aspects of federated learning, such as performance in heterogeneous data settings, reduced communication efficiency, and improved privacy.

[0050] Heterogeneous data distribution:

[0051] Typically, client data may naturally have heterogeneous distributions, which leads to a significant drop in accuracy during federated training, making it impossible for naive aggregations such as FedAvg to guarantee that the model converges to a local minimum. One way to address such challenges is personalized FL, which adds proximal terms to local training objectives to allow client models to produce variations without overfitting. Other ways to address this issue include regularization, model mixing, and clustering clients, which can be used to stabilize FL with personal models. In addition, multi-task learning using simultaneous execution of different tasks and meta-learning that can quickly adapt the global model to the local data distribution of each client can be used to alleviate this problem. In this work, the problem of heterogeneous data distribution is addressed by using base models, as these models can provide more general solutions while maintaining the same amount of data as conventional FL methods.

[0052] Basic Model

[0053] Base models may have advantages in specific areas such as language, vision, and integration of multimodal pre-training methods. The ability to pre-train on large-scale datasets can make these models seamlessly applicable to a wide range of tasks. The framework can apply base models to FL to improve the client's robustness to distribution changes and heterogeneous data distributions, or to improve the overall performance of the system. Although base models are very beneficial to performance in heterogeneous data settings, training and evaluating directly on base models increases the computational burden on the client and increases latency during inference. This hinders their deployment in real-world settings. Instead, the framework takes advantage of locally stored base models to help improve the performance and robustness of small-scale models while maintaining low communication and inference overheads.

[0054] Inference efficiency

[0055] We turn to FL strategies that address the inference efficiency issue. For example, consider employing dropout to ensure that inference remains efficient, or methods to adaptively prune parameters during federated training, or provide theoretical analysis for the design strategies involved in local model pruning masks. Consider utilizing personalized and structured sparse masks for federated learning, or creating diverse local models as subnetworks of the global network by selecting key continuous parameters layer by layer via structured pruning and with the help of static batch normalization. An important difference between existing efforts to address inference efficiency and the approach presented here is that they use pruning as a platform to personalize and improve the communication and inference time of each client. In contrast, the present disclosure fixes the available budget and continues to improve the performance ceiling for smaller models. The two approaches are complementary and can be combined to further amplify their impact.

[0056] Privacy

[0057] To alleviate data privacy concerns in FL, consider a FL framework that allows clients to train memes / proxy models and private models using mutual learning. Memes / proxy models may be shared only with the server, reducing the risk of data exposure. However, there may be vulnerabilities associated with the capture of model weights / updates during the exchange of information between the client and the server. To provide stronger privacy protection, methods such as differential privacy, secure multi-party computation, secret sharing, homomorphic encryption, and hybrid methods can be used. To address privacy concerns in FL, ProxyFL can use a concept in which clients maintain private and proxy models trained using mutual learning. Clients communicate exclusively with other clients by exchanging their proxy models, ensuring the privacy of data and models. However, in ProxyFL, each client can use DP-SGD during training, which provides a measurable guarantee for differential privacy. In a similar way, consider a FL framework based on differential privacy (DP) to confuse and protect information about the distribution of client data by adding artificial noise to the parameters during local training in the client. Inspired by methods proposed to actively address privacy concerns, our framework can adopt the approach proposed in ProxyFL and exploit the impact of distillation from a base model on the performance of DP-based optimizers.

[0058] Distillation

[0059] Knowledge distillation is a teaching technique that transfers valuable insights and generalization capabilities of teacher models to student models. Within the FL domain, the framework explores adaptive aggregation methods that utilize ensemble distillation to fuse models on the server, and then the framework considers auxiliary datasets to weight and integrate local models from each client. Consider FedDistill, a method that extracts statistics related to logit (log odds) vectors from client models, which are then shared with other clients to facilitate knowledge distillation. In addition, a data-free knowledge distillation method (FEDGEN) is considered to address data heterogeneity in FL, where at the server, they train a generative model by combining client information and use data from the generator to help improve the performance of the client model. Alternatively, a personalized federated learning method based on co-distillation allows cross-architecture training. In this framework, the impact of the basic idea of ​​knowledge distillation on the performance of small-scale models while avoiding the use of excessive data, augmentation, or model sharing is considered, providing guidance on how the base model can be effectively used in federated learning.

[0060] Leveraging the Base Model in Federated Learning

[0061] Consider the conventional FL framework FedAvg shown below, which is then enhanced by the key components of the framework of the present disclosure.

[0062] Standard FL framework

[0063] FedAvg can be concisely described by the following equation,

[0064]

[0065] where θ∈Rd represents the parameters of the global model and N represents the number of clients. i ) is the loss function of the i-th client, where each client contains its local data distribution, denoted by D i express.

[0066] method

[0067] This disclosure proposes a new framework in which each of N clients contains two different types of local models: (a) a pre-trained base model (private) and (b) a small-scale model (agent), denoted as In this framework, the objective of FL is the same as Equation 1, where the model being optimized is a small-scale surrogate model.

[0068] Framework Core

[0069] An overview of the core of the framework is illustrated in Algorithm 1. During training, the server obtains locally updated small-scale models from each client. These models are aggregated using an idea similar to Equation 1 or a more complex weighting scheme (e.g., Equation 2 below) that takes into account the data distribution among clients.

[0070]

[0071] After model aggregation is complete, the server broadcasts the aggregated model to each client, where the client uses the aggregated model to update the weights of its local small-scale model. Within each client, the small-scale model is trained on the local data distribution using the following loss,

[0072]

[0073]

[0074]

[0075] Where L CE is the cross entropy loss, L Distill is the Kullback Leibler (KL) divergence loss, which is used to distill knowledge from the base model to the small-scale proxy model. The parameter λ controls the ratio of knowledge extracted from the base model compared to the ground truth label. Once the local training is completed, each client pushes its local small-scale model to the server and repeats the whole process of aggregation, broadcasting, and client update until convergence.

[0076] Figure 3A : An overview of the proposed lightweight federated learning framework. For each client, there are two models, namely the student model and the frozen base model. The frozen base model generates logits to guide the training of the student model. The aggregate model is completed by the server aggregating information from different local models without observing their data. In the inference phase, the base model is discarded.

[0077] Considering the first exemplary embodiment, recall that federated learning is the process of collaborative learning between clients while maintaining the privacy of the data stored therein. The server acts as a supervisor of the learning process by receiving and aggregating information provided from each client and then sharing the aggregated model among all clients. Typical federated learning systems experience problems such as performance degradation and robustness degradation, especially when faced with issues such as data heterogeneity and distribution changes. Base models pre-trained on large amounts of data can be widely adapted for initialization of model parameters. In cases where clients have access to their own base models but have strong privacy concerns, their maximum achievable performance will be limited.

[0078] In view of the above challenges, this embodiment of the framework aims to address these issues by proposing to use the base model locally within the client to improve its performance, while using a smaller student (proxy) model to coordinate and synchronize information between various clients. The knowledge distilled from the base model is then used to guide the training of the smaller model. This embodiment proposes a framework that takes advantage of the diverse feature embeddings from the base model and distills knowledge to improve the learning performance of various clients. This allows the client to choose its own base model instead of using the same base model. By consistently distilling the performance of the base model to a smaller student model in the client terminal and updating the student model with aggregated weights from the server, the framework takes advantage of the prior knowledge from the pre-trained base model while retaining the information provided by the client population. Clients distill knowledge from the pre-trained base model of their own choice, fine-tune on local data, and have access to aggregated knowledge in a federated learning setting. Therefore, the framework balances the high level of performance and robustness that can be achieved by using the base model while achieving an efficient communication setting between the server and the client while obtaining privacy by not directly exposing the base model.

[0079] The focus here is on distilling knowledge from the base model and training smaller models. The distilled smaller models are broadcast to the server and receive an updated aggregated global model from the server. The prototypes generated from the pre-trained models are then combined with a contrastive loss to update the representations learned within each client. This embodiment provides an alternative solution that approaches the performance of the base model while training small models. By distilling knowledge from a pre-trained base model of its own choice, combined with updates from the client and adaptation to local client data, the framework provides a high-performance, stable, and small-scale foundation for the client. Since the choice of the base model is private to the client, this embodiment maintains more privacy for the client compared to using the same base model across clients.

[0080] This embodiment can be used to train models with many different sensor input systems using federated / decentralized / collaborative learning in industrial IoT, sensor networks, and automation of production scenarios.

[0081] Figure 3Bis a flowchart of privacy-enhanced federated learning with client-side base model distillation, where the server sends the model weights obtained and aggregated from all clients to each client. At each client, a smaller model is initialized with the weights sent from the server. The model is then updated using two losses, one loss distilling information from the pre-trained base models stored in the client, and the other loss is the cross-entropy loss. Once the smaller client model is updated, its weights are transmitted to the server, and the whole process is repeated.

[0082] Assume there are N clients, each storing M i (i∈[N]) pre-trained base models and a smaller student model. The choice of base model can cover a wide range of variations, including but not limited to changes in architecture, loss function, pre-training dataset, etc. The server contains the smaller model (θ S ), this smaller model is matched to the backbone of the client's model and serves as the main synchronization point for the client. Overall, the server acts as a coordinator for federated training, sending and receiving information from the clients.

[0083] In each client i, M i The parameters of each base model and the distilled smaller model in the base model are expressed as and Here, each client i can access the locally stored data, represented as (D i ,y i ). Additionally, in the context of federated training, there are key parameters such as the number of epochs the client trains between each communication (to the server) round, batch size, learning rate, optimizer type, etc.

[0084] enter:

[0085] The central server (coordinator) hosts a smaller student model that mirrors the clients' models.

[0086] There are N clients in the federated learning system, with their locally stored data (D i ,y i ) and initialized local model And store M pre-trained base models.

[0087] step:

[0088] At the beginning of each round (r) of federated training, the server receives model weights from each client in [N]. The model weights are aggregated and used to initialize θ DS , the method is as follows,

[0089]

[0090] Here, we assume that the data is uniformly sampled across all clients, thus allowing us to assign re-weighted importance to each client model.

[0091] After the model aggregation described in the previous step is completed, the student model will be broadcast to each client.

[0092] After each client receives the information from the server, they reset their models by accepting their weights and reinitializing them using the aggregate model, and start training on their local data (D i ,y i ) is trained on the local dataset (D i ,y i ) based model pre-trained on In the part of distilling knowledge, the corresponding loss term is given by:

[0093]

[0094] It is called KL-divergence loss[5], where Z i represents the logit of the i-th class, and T represents the distillation temperature. Combined with the adaptation of local data training, the loss function is as follows:

[0095]

[0096] Where l() is the cross entropy loss function.

[0097] These sets of operations are then repeated until a stopping criterion (e.g., total number of rounds, loss / performance tolerance, etc.) is reached.

[0098] Considering the second exemplary embodiment, recall that federated learning is the process of collaborative learning between clients while maintaining the privacy of the data stored therein. The server acts as a supervisor of the learning process by receiving and aggregating information provided from each client and then sharing the aggregated model among all clients. Typically, clients are deployed to solve problems in real-world scenarios, which limit their infrastructure to limited computing resources, runtime, memory, etc. With this in mind, clients may not be suitable for storing and running state-of-the-art models, which are typically very large. Therefore, clients may be restricted to using small-scale models, which limits the maximum possible performance that can be achieved. With such adjustments, the performance and robustness of the entire system may experience a decline, especially in the face of issues such as data heterogeneity and distribution changes that are common in federated learning.

[0099] In view of the above challenges, this second embodiment aims to solve these problems by proposing to use the base model in the server to coordinate and synchronize information between various clients. Although the use of the base model in the client improves its personalization ability, the limitation of the client computing resources may not allow the base model to be stored and executed in all clients, whether the entire content or a subset of the content of the base model is shared between the server and the client. Consider combining the feature embeddings generated by the pre-trained network in a federated manner, while assuming that the pre-trained model exists in each client. This second embodiment proposes a framework that uses the benefits of the diversified feature embeddings of the base model placed in the central server to coordinate and improve the learning performance between various clients without the additional overhead of directly sharing the base model. By consistently distilling the performance of the frozen base model to a smaller student model in the server (which mirrors the model backbone that is more suitable for running on the client device), combined with updating the student model with the aggregated weights from the client, our framework takes advantage of the prior knowledge from the pre-trained base model while retaining the information provided by the client. The client then obtains a better initial model, i.e., the student model distilled in the server, and further fine-tunes itself based on local data to improve its performance. Thus, the framework balances an efficient communication setup between server and client with a high level of performance and robustness achievable by using the underlying model.

[0100] In addition to using the base model in each client of the federated system, this second embodiment also focuses on adapting the base model placed in the central server by distilling it into smaller models. The distilled smaller base model receives updates from all clients at the same time, and then synchronously broadcasts the updated global model to each client. The prototype generated from the pre-trained model is combined with the contrastive loss to update the representation learned in each client. And an alternative solution is to use a small model that is more suitable for the available resources to approach the performance base model. By placing the base model on a central server and utilizing the distilled version of the model, combined with updates from the clients, the distilled global model provides a high-performance, stable, and small-scale foundation for the clients, which can be further optimized using local data.

[0101] This embodiment can be used to train models with many different sensor input systems using federated / decentralized / collaborative learning in industrial IoT, sensor networks, and automation of production scenarios.

[0102] Figure 3C is a flow chart of federated learning with server-side base model distillation, where the client sends the locally adapted model weights to the server. At the server, the models of all clients are aggregated. Once the aggregation is complete, the model is updated by distillation from the base model using a common dataset, and then the distilled model is broadcast to all clients. At each client, the smaller model is initialized with the weights sent from the server, and the whole process is repeated.

[0103] Assume that the central server stores M pre-trained base models, each of which is parameterized as The choice of base model can cover a wide range of variations, including but not limited to changes in architecture, loss function, pre-training dataset, etc. In addition to the base model, the server also contains a smaller student model (θ DS ), which matches the backbone of the client's model and serves as the main synchronization point for the client. Overall, the server acts as a coordinator for federated training, sending and receiving information from clients.

[0104] A federated learning system consists of N clients. Typically, clients are limited to using the minimum amount of available hardware resources. Therefore, consider using a common small model on all clients by default to mimic this behavior.

[0105] Generally speaking, there is a common public dataset (D p ,y p For each client i, its locally stored data and model are represented by (D i ,y i ) and θi In addition, in the context of federated training, there are some key parameters, such as the number of epochs the client trains between each communication (to the server), batch size, learning rate, optimizer type, etc.

[0106] enter:

[0107] The central server (coordinator) stores M pre-trained base models (deep neural networks) and a smaller student model θ DS , the student model mirrors the client's model.

[0108] There are N clients in the federated learning system, where each client i has locally stored data (D i ,y i ) and with initialization parameters θ i The local model.

[0109] Both the server and the client can access public data (D p ,y p ).

[0110] step:

[0111] 1. At the beginning of each round (r) of federated training, the server receives model parameters θ from each client in [N] i The model parameters are aggregated and used to initialize Here is how:

[0112]

[0113] Here, we re-weight the importance of each client model to match the amount of local data within each client relative to the total amount of data in the system.

[0114] 2. After model aggregation is completed, use public datasets to distill knowledge from the pre-trained base model to a model with aggregated parameters The loss used for distillation is given by:

[0115]

[0116] It is called KL-divergence loss[5], where Z i denotes the logit of the ith class, and T denotes the distillation temperature.

[0117] 3. After distillation is completed, the student model is broadcast to each client, where the local model weights Reinitialized to the student model broadcast from the server The client then starts to read the data in its local data (D i ,y i ) using the loss given by

[0118]

[0119] Where l() is the loss function.

[0120] Then repeat steps 1-3 until a stopping criterion is reached (e.g., total number of rounds, loss / performance tolerance, etc.).

[0121] The machine learning models described herein can be used in many different applications. As described above, the locally stored raw source data can be image data, sound data, etc., and thus, as an example, Figure 6-11 Various applications that retrieve or use this data are shown in FIG. Figure 5 The structure for training and using machine learning models for these applications (and others) is illustrated in . Figure 5 A schematic diagram of the interaction between a computer-controlled machine 500 and a control system 502 is depicted. The computer-controlled machine 500 includes an actuator 504 and a sensor 506. The actuator 504 may include one or more actuators, and the sensor 506 may include one or more sensors. The sensor 506 is configured to sense a condition of the computer-controlled machine 500. The sensor 506 may be configured to encode the sensed condition into a sensor signal 508 and transmit the sensor signal 508 to the control system 502. Non-limiting examples of the sensor 506 include video, radio radar, lidar, microphone, ultrasonic, and motion sensors. In one embodiment, the sensor 506 is an optical sensor configured to sense an optical image of an environment close to the computer-controlled machine 500.

[0122] The control system 502 is configured to receive sensor signals 508 from the computer controlled machine 500. As described below, the control system 502 may be further configured to calculate actuator control commands 510 based on the sensor signals and transmit the actuator control commands 510 to the actuators 504 of the computer controlled machine 500.

[0123] like Figure 5As shown in , the control system 502 includes a receiving unit 512. The receiving unit 512 may be configured to receive a sensor signal 508 from the sensor 506 and transform the sensor signal 508 into an input signal x. In an alternative embodiment, the sensor signal 508 is directly received as an input signal x without the receiving unit 512. Each input signal x may be a portion of each sensor signal 508. The receiving unit 512 may be configured to process each sensor signal 508 to generate each input signal x. The input signal x may include data corresponding to an image recorded by the sensor 506.

[0124] The control system 502 includes a classifier 514. The classifier 514 may be configured to classify the input signal x into one or more labels using a machine learning algorithm, such as the neural network described above. The classifier 514 is configured to be parameterized by parameters, such as the parameters described above (e.g., parameter θ). The parameter θ may be stored in and provided by a non-volatile storage device 516. The classifier 514 is configured to determine an output signal y based on the input signal x. Each output signal y includes information for assigning one or more labels to each input signal x. The classifier 514 may transmit the output signal y to a conversion unit 518. The conversion unit 518 is configured to convert the output signal y into an actuator control command 510. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504, and the actuator 504 is configured to actuate the computer-controlled machine 500 in response to the actuator control command 510. In another embodiment, the actuator 504 is configured to actuate the computer-controlled machine 500 directly based on the output signal y.

[0125] After the actuator 504 receives the actuator control command 510, the actuator 504 is configured to perform an action corresponding to the associated actuator control command 510. The actuator 504 may include control logic configured to transform the actuator control command 510 into a second actuator control command for controlling the actuator 504. In one or more embodiments, instead of or in addition to controlling the actuator, the actuator control command 510 may also be used to control a display.

[0126] In another embodiment, instead of or in addition to the computer-controlled machine 500 including the sensor 506, the control system 502 also includes the sensor 506. Instead of or in addition to the computer-controlled machine 500 including the actuator 504, the control system 502 may also include the actuator 504.

[0127] like Figure 5As shown in , the control system 502 also includes a processor 520 and a memory 522. The processor 520 may include one or more processors. The memory 522 may include one or more memory devices. The classifier 514 of one or more embodiments (e.g., a machine learning algorithm, such as those described above) can be implemented by the control system 502, which includes a non-volatile storage device 516, a processor 520, and a memory 522.

[0128] The non-volatile storage device 516 may include one or more persistent data storage devices, such as a hard drive, an optical drive, a tape drive, a non-volatile solid-state device, a cloud storage device, or any other device capable of persistently storing information. The processor 520 may be the processor described above with reference to Figure 1-2 Any processor or processor subsystem described, and may include one or more devices selected from a high performance computing (HPC) system, including a high performance core, a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, a tensor processing unit, a graphics processing unit, an ASIC, or any other device that manipulates signals (analog or digital) based on computer executable instructions residing in the memory 522. The memory 522 may include a single memory device or multiple memory devices, including but not limited to random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information.

[0129] The processor 520 may be configured to read the memory 522 and execute computer executable instructions residing in the non-volatile storage device 516 and embodying one or more machine learning algorithms and / or methods of one or more embodiments. The non-volatile storage device 516 may include one or more operating systems and applications. The non-volatile storage device 516 may store compiled and / or interpreted computer programs created using various programming languages ​​and / or technologies, including but not limited to Java, C, C++, C#, object-oriented C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL, alone or in combination.

[0130] When executed by processor 520, the computer executable instructions of non-volatile storage device 516 may cause control system 502 to implement one or more machine learning algorithms and / or methods disclosed herein. Non-volatile storage device 516 may also include machine learning data (including data parameters) that support the functions, features, and processes of one or more embodiments described herein.

[0131] The program code embodying the algorithms and / or methods described herein can be distributed individually or collectively as a program product in a variety of different forms. The program code can be distributed using a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement aspects of one or more embodiments. Computer-readable storage media are non-transient in nature and can include volatile and non-volatile, removable and non-removable tangible media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can also include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disk read-only memory (CD-ROM) or other optical storage devices, cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired information and can be read by a computer. Computer-readable program instructions can be downloaded from a computer-readable storage medium to a computer, another type of programmable data processing device, or another device, or downloaded to an external computer or external storage device via a network.

[0132] Computer-readable program instructions stored in a computer-readable medium can be used to instruct a computer, other types of programmable data processing devices, or other devices to operate in a particular manner so that the instructions stored in the computer-readable medium produce a manufactured product including instructions for implementing the functions, actions, and / or operations specified in the flowchart or diagram. In certain alternative embodiments, the functions, actions, and / or operations specified in the flowchart and diagram can be reordered, processed serially, and / or processed simultaneously according to one or more embodiments. In addition, any flowchart and / or diagram may include more or fewer nodes or blocks than those shown in one or more embodiments.

[0133] A process, method or algorithm may be embodied in whole or in part using suitable hardware components (such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a state machine, a controller or other hardware components or devices) or a combination of hardware, software and firmware components.

[0134] Figure 6-11 Embodiments of environments in which the federated learning system described herein may be implemented are illustrated. Each of these embodiments illustrates an embodiment of a client device. In these embodiments, data originating from sensor 506 may be the original source data used to train a machine learning model on the device (client), but is not transmitted to a server to protect the data. Figure 6A schematic diagram of a control system 502 is depicted, the control system 502 being configured to control a vehicle 600, which may be an at least partially autonomous vehicle or an at least partially autonomous robot. The vehicle 600 includes an actuator 504 and a sensor 506. The sensor 506 may include one or more video sensors, cameras, microphones, radio radar sensors, ultrasonic sensors, lidar sensors, and / or location sensors (e.g., GPS). One or more of the one or more specific sensors may be integrated into the vehicle 600. The sensor 506 may include a software module that is configured to determine the state of the actuator 504 when executed. A non-limiting example of a software module includes a weather information software module that is configured to determine the current or future state of the weather near the vehicle 600 or other location.

[0135] The classifier 514 of the control system 502 of the vehicle 600 can be configured to detect objects near the vehicle 600 based on the input signal x. In such an embodiment, the output signal y may include information characterizing the proximity of the object relative to the vehicle 600. The actuator control command 510 can be determined based on this information. The actuator control command 510 can be used to avoid a collision with the detected object. The original source data for federated learning may include raw images of the vehicle's surroundings, but the vehicle's processing of objects in the surrounding environment may change the weights in the machine learning model used onboard; these adjusted weights can then be sent back to the server's model for updating.

[0136] In embodiments where the vehicle 600 is at least partially autonomous, the actuator 504 may be embodied in the brakes, propulsion system, engine, transmission, or steering of the vehicle 600. Actuator control commands 510 may be determined such that the actuator 504 is controlled so that the vehicle 600 avoids a collision with the detected object. The detected object may also be classified according to what the classifier 514 considers the detected object to be most likely to be, such as a pedestrian or a tree. The actuator control commands 510 may be determined based on the classification. In scenarios where a hostile attack may occur, the above system may be further trained to better detect objects or recognize changes in lighting conditions or angles of sensors or cameras on the vehicle 600.

[0137] In other embodiments where the vehicle 600 is an at least partially autonomous robot, the vehicle 600 may be a mobile robot configured to perform one or more functions such as flying, swimming, diving, and stepping. The mobile robot may be an at least partially autonomous lawn mower or an at least partially autonomous cleaning robot. In such embodiments, the actuator control commands 510 may be determined such that a propulsion unit, a steering unit, and / or a braking unit of the mobile robot may be controlled so that the mobile robot may avoid a collision with the identified object.

[0138] In another embodiment, the vehicle 600 is an at least partially autonomous robot in the form of a gardening robot. In such an embodiment, the vehicle 600 may use an optical sensor as a sensor 506 to determine the state of plants in the environment near the vehicle 600. The actuator 504 may be a nozzle configured to spray a chemical. Based on the identified plant species and / or the identified plant state, the actuator control command 510 may be determined to cause the actuator 504 to spray the plant with an appropriate amount of a suitable chemical.

[0139] The vehicle 600 may be an at least partially autonomous robot in the form of a household appliance. Non-limiting examples of household appliances include a washing machine, a stove, an oven, a microwave, or a dishwasher. In such a vehicle 600, the sensor 506 may be an optical sensor configured to detect the state of an object to be processed by the household appliance. For example, in the case where the household appliance is a washing machine, the sensor 506 may detect the state of the laundry inside the washing machine. The actuator control command 510 may be determined based on the detected state of the laundry.

[0140] Figure 7 A schematic diagram of a control system 502 is depicted, the control system 502 being configured to control a system 700 (e.g., a manufacturing machine), such as a punch cutter, a cutter, or a barrel drill, of a manufacturing system 702 (e.g., a portion of a production line). The control system 502 may be configured to control an actuator 504, which is configured to control the system 700 (e.g., a manufacturing machine).

[0141] The sensor 506 of the system 700 (e.g., a manufacturing machine) may be an optical sensor configured to capture one or more attributes of the manufactured product 704. The classifier 514 may be configured to determine a state of the manufactured product 704 based on the captured one or more attributes. The actuator 504 may be configured to control the system 700 (e.g., a manufacturing machine) based on the determined state of the manufactured product 704 to perform subsequent manufacturing steps of the manufactured product 704. The actuator 504 may be configured to control the function of the system 700 (e.g., a manufacturing machine) on the subsequent manufactured product 706 of the system 700 (e.g., a manufacturing machine) based on the determined state of the manufactured product 704.

[0142] Figure 8 A schematic diagram of a control system 502 configured to control a power tool 800 (such as a drill or driver) having at least a partially autonomous mode is depicted. The control system 502 may be configured to control an actuator 504 configured to control the power tool 800 .

[0143] The sensor 506 of the power tool 800 may be an optical sensor configured to capture one or more properties of the working surface 802 and / or the fastener 804 driven into the working surface 802. The classifier 514 may be configured to determine the state of the working surface 802 and / or the fastener 804 relative to the working surface 802 based on the captured one or more captured properties. The state may be that the fastener 804 is flush with the working surface 802. The state may alternatively be the hardness of the working surface 802. The actuator 504 may be configured to control the power tool 800 so that the driving function of the power tool 800 is adjusted based on the determined state of the fastener 804 relative to the working surface 802 or one or more captured properties of the working surface 802. For example, if the state of the fastener 804 is flush with the working surface 802, the actuator 504 may stop the driving function. As another non-limiting example, the actuator 504 may apply additional or less torque based on the hardness of the working surface 802.

[0144] Fig. 9 A schematic diagram of a control system 502 configured to control an automated personal assistant 900 is depicted. The control system 502 may be configured to control an actuator 504, which may be configured to control the automated personal assistant 900. The automated personal assistant 900 may be configured to control a household appliance, such as a washing machine, a stove, an oven, a microwave, or a dishwasher.

[0145] Sensor 506 may be an optical sensor and / or an audio sensor. The optical sensor may be configured to receive a video image of user 902's gesture 904. The audio sensor may be configured to receive a voice command of user 902.

[0146] The control system 502 of the automated personal assistant 900 can be configured to determine an actuator control command 510, which is configured to control the system 502. The control system 502 can be configured to determine the actuator control command 510 based on the sensor signal 508 of the sensor 506. The automated personal assistant 900 is configured to transmit the sensor signal 508 to the control system 502. The classifier 514 of the control system 502 can be configured to execute a gesture recognition algorithm to recognize the gesture 904 made by the user 902, determine the actuator control command 510, and transmit the actuator control command 510 to the actuator 504. The classifier 514 can be configured to retrieve information from the non-volatile storage device in response to the gesture 904 and output the retrieved information in a form suitable for receipt by the user 902.

[0147] Fig.10A schematic diagram of a control system 502 is depicted, which is configured to control a monitoring system 1000. The monitoring system 1000 can be configured to physically control access through a door 1002. A sensor 506 can be configured to detect scenes relevant to deciding whether to allow access. The sensor 506 can be an optical sensor configured to generate and transmit image and / or video data. The control system 502 can use this data to detect the face of a person.

[0148] The classifier 514 of the control system 502 of the monitoring system 1000 can be configured to interpret the image and / or video data by matching the identities of known persons stored in the non-volatile storage device 516 to determine the identity of the person. The classifier 514 can be configured to generate an actuator control command 510 in response to the interpretation of the image and / or video data. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504. In this embodiment, the actuator 504 can be configured to lock or unlock the door 1002 in response to the actuator control command 510. In other embodiments, non-physical logical access control is also possible.

[0149] The monitoring system 1000 may also be a surveillance system. In such an embodiment, the sensor 506 may be an optical sensor configured to detect a scene under surveillance, and the control system 502 is configured to control the display 1004. The classifier 514 is configured to determine the classification of the scene, for example, whether the scene detected by the sensor 506 is suspicious. The control system 502 is configured to transmit the actuator control command 510 to the display 1004 in response to the classification. The display 1004 may be configured to adjust the displayed content in response to the actuator control command 510. For example, the display 1004 may highlight objects that are considered suspicious by the classifier 514. Using embodiments of the disclosed system, the surveillance system can predict objects that appear at certain times in the future.

[0150] Fig.11A schematic diagram of a control system 502 is depicted, which is configured to control an imaging system 1100, such as an MRI device, an X-ray imaging device, or an ultrasound device. The sensor 506 can be, for example, an imaging sensor. The classifier 514 can be configured to determine the classification of all or part of the sensed image. The classifier 514 can be configured to determine or select an actuator control command 510 in response to the classification obtained by the trained neural network. For example, the classifier 514 can interpret an area of ​​the sensed image as a possible abnormality. In this case, the actuator control command 510 can be determined or selected to cause the display 1102 to display the imaging and highlight the potential abnormal area. The sensed image can be used within the hospital environment to train the machine learning system (client) within the hospital, but this data will not be sent to the server to train the server's model. Instead, the adjusted weights of the hospital based on the sensed image adjustment can be sent to the server to adjust the weights on the server side.

[0151] Although exemplary embodiments are described above, these embodiments are not intended to describe all possible forms covered by the claims. The words used in the specification are descriptive rather than restrictive, and it should be understood that various changes can be made without departing from the spirit and scope of the present disclosure. As previously mentioned, the features of various embodiments can be combined to form further embodiments of the present invention that may not be explicitly described or illustrated. Although various embodiments may be described as providing advantages or being superior to other embodiments or prior art implementations in terms of one or more desired characteristics, it is recognized by those of ordinary skill in the art that one or more features or characteristics can be sacrificed to achieve the desired overall system properties, depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, applicability, weight, manufacturability, ease of assembly, etc. Therefore, even if any embodiment is described as not as ideal as other embodiments or prior art implementations in terms of one or more characteristics, these embodiments do not exceed the scope of the present disclosure and may be ideal for specific applications.

Claims

1. A method for training a neural network using federated learning, the method comprising: sending at least a portion of the machine learning model maintained by the server from the server to the plurality of clients, thereby generating a plurality of local machine learning models; At each of the plurality of clients, training the plurality of local machine learning models using locally stored data, the locally stored data being stored locally at the respective client, wherein the training at each client comprises determining a respective cross entropy loss and a distillation loss for each of the plurality of local machine learning models based on a plurality of base models local to each of the plurality of clients; At each of the plurality of clients, updating a corresponding weight of each of the plurality of local machine learning models based on a cross entropy loss and a distillation loss; transmitting the corresponding updated weights from each client to the server without transmitting the client's locally stored data; aggregating, at the server, the corresponding updated weights from each of the plurality of local machine learning models; transmitting the aggregated corresponding updated weights to each of the plurality of clients; and At each of the plurality of clients, each of the plurality of local machine learning models is updated using the aggregated corresponding updated weights.

2. The method according to claim 1, wherein the distillation loss is calculated according to the following formula:

3. The method according to claim 2, wherein the corresponding weights are updated according to:

4. The method of claim 3, wherein the polymerization is carried out according to:

5. The method of claim 4, wherein the plurality of base models comprises pre-trained base models.

6. A system for training a neural network using federated learning, the system comprising: Memory, which stores instructions; as well as A plurality of processors, when executing instructions stored in the memory, the plurality of processors jointly perform: sending at least a portion of the machine learning model maintained by the server from the server to the plurality of clients, thereby generating a plurality of local machine learning models; At each of the plurality of clients, training the plurality of local machine learning models using locally stored data, the locally stored data being stored locally at the respective client, wherein the training at each client comprises determining a respective cross entropy loss and a distillation loss for each of the plurality of local machine learning models based on a plurality of base models local to each of the plurality of clients; At each of the plurality of clients, updating a respective weight of each of the plurality of local machine learning models; transmitting the corresponding updated weights from each client to the server without transmitting the client's locally stored data; aggregating, at the server, respective weights from each of the plurality of local machine learning models; transmitting the aggregated corresponding weight to each of the plurality of clients; and At each of the plurality of clients, each of the plurality of local machine learning models is updated using the aggregated corresponding weight.

7. The system of claim 6, wherein the distillation loss is calculated according to the following formula:

8. The system of claim 7, wherein updating the corresponding weights is based on:

9. The system of claim 8, wherein the aggregation is based on:

10. The system of claim 9, wherein the plurality of base models comprises pre-trained base models.

11. A robotic system operated by a neural network, comprising: Memory, which stores instructions; as well as At least one processor, when executing instructions stored in the memory, jointly trains the neural network using federated learning by following steps: At each of the plurality of clients, training the plurality of local machine learning models using locally stored data, the locally stored data being stored locally at the respective client, wherein the training at each client comprises determining a respective cross entropy loss and a distillation loss for each of the plurality of local machine learning models based on a plurality of base models local to each of the plurality of clients; At each of the plurality of clients, updating a respective weight of each of the plurality of local machine learning models; transmitting the corresponding updated weights from each client to the server without transmitting the client's locally stored data; aggregating, at the server, respective weights from each of the plurality of local machine learning models; transmitting the aggregated corresponding weight to each of the plurality of clients; and At each of the plurality of clients, each of the plurality of local machine learning models is updated using the aggregated corresponding weight.

12. The system of claim 11, wherein the distillation loss is calculated according to the following formula:

13. The system of claim 12, wherein updating the corresponding weights is based on:

14. The system of claim 13, wherein the aggregation is based on:

15. The system of claim 14, wherein the plurality of base models comprises pre-trained base models.

16. The robotic system of claim 15, wherein the robotic system is an autonomous vehicle.

17. The robotic system of claim 15, wherein the robotic system is a medical system.

Images