A federated learning method and related apparatus

Abstract

A federated learning method applied to the field of artificial intelligence technology. In the method, federated learning is realized by interacting prior distribution and posterior distribution of model parameters between nodes, which can realize learning of data distribution of training data in the learning node in the model training process; and when the node obtains multiple models corresponding to different data distributions, the node selects a model closest to the training data distribution from the multiple models based on the performance of each model in processing the training data to perform training, thereby solving the problem of different training data distributions on different nodes and effectively improving the effect of the trained model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and more particularly to a federated learning method and related apparatus. Background Technology

[0002] As users become increasingly concerned about protecting their personal privacy, the inability of data owners to share user data has created numerous "data silos." These "data silos" pose a new challenge to artificial intelligence (AI) based on massive amounts of data: how to train machine learning models without access to sufficient training data.

[0003] To address the challenges posed by "data silos," federated learning emerged. Federated learning effectively helps clients perform joint training and build shared machine learning models without sharing data resources (i.e., training data does not leave their local machines). During the local training phase, each client trains its local model based on the training data. In the model aggregation phase, each client uploads its local model to the cloud server, which aggregates the various local models to obtain a global model and distributes it. Clients then update their local models using the training data based on the global model, repeating this process until the global model converges.

[0004] In federated learning, the clients participating in federated learning usually belong to different users or organizations. Therefore, the distribution of training data on different clients often varies greatly, that is, the data is non-independent and identically distributed (Non-IID). This can easily lead to poor performance of the trained model, or even the model failing to converge. Summary of the Invention

[0005] This application provides a federated learning method that can effectively improve the performance of trained models.

[0006] This application provides a federated learning method for the field of artificial intelligence. The federated learning method includes: first, a first node acquires the prior distribution of parameters of multiple models, where each model can be a machine learning model whose parameters follow a distribution. The prior distribution of parameters of the multiple models can be, for example, a Gaussian distribution, a delta distribution, or other distributions. The first node can be, for example, a client node, used to train the acquired models based on training data.

[0007] Then, the first node determines the performance of each model in processing the training data based on the prior distribution of the parameters of multiple models and the training data of the first node. For example, the first node samples the parameter values ​​of each model based on the prior distribution of the parameters of multiple models, thus obtaining the specific values ​​of each parameter in each model. The first node then determines the performance of each model in processing the training data based on the parameter values ​​of each model and the training data. The training data of the first node can be training data stored locally on the first node; or it can be training data stored on a cloud server or database connected to the first node.

[0008] Secondly, the first node trains the posterior distribution of the parameters of the first model based on the prior distribution of the first model's parameters and the training data. The first model is one of multiple models, determined based on the performance of each model when processing the training data. For example, the first node can select the model with the best performance when processing the training data from among multiple models. The optimal performance of the first model when processing the training data indicates that the prior distribution of its parameters best matches the distribution of the training data in the first node; therefore, the first node selects the first model for further training.

[0009] Finally, the first node sends the posterior distribution of the parameters of the first model to the second node, so that the second node can update the prior distribution of the parameters of the first model based on the posterior distribution of the parameters of the first model uploaded by each node. The second node can be, for example, an aggregation node, used to update the prior distribution of the model's parameters based on the posterior distribution of the parameters of the models uploaded by each client node.

[0010] In this scheme, federated learning is achieved by exchanging the prior and posterior distributions of model parameters between nodes. This enables the learning of the data distribution of training data in nodes during model training. Furthermore, when a node obtains multiple models corresponding to different data distributions, it selects the model that best matches the training data distribution from among the multiple models based on the performance of each model in processing training data. This solves the problem of different training data distributions on different nodes and can effectively improve the performance of the trained model.

[0011] Furthermore, machine learning models with parameters following a distribution can predetermine the possible values ​​of the parameters, and these possible values ​​can characterize the merits of various possible improvement directions for the machine learning model. Therefore, federated learning on machine learning models with parameters following a distribution helps participating nodes find better improvement directions for the machine learning model, thereby reducing training time and communication overhead between nodes.

[0012] In one possible implementation, the federated learning architecture includes an aggregation node and multiple client nodes. The second node is the aggregation node, and the first node is one of the multiple client nodes. The first node obtains the prior distribution of the parameters of multiple models, specifically by receiving the prior distribution of the parameters of multiple models from the second node.

[0013] In addition, while the first node sends the posterior distribution of the parameters of the first model to the second node, the first node also sends an indication message to the second node, which indicates that the posterior distribution of the parameters sent by the first node corresponds to the first model.

[0014] In other words, after the first node selects the first model to train from among multiple models, when uploading the posterior distribution of the parameters of the first model, the first node also needs to notify the second node that the model it has selected is the first model.

[0015] In this scheme, the aggregation node pre-builds multiple models corresponding to different data distribution types based on the potential data distribution on each client node, and distributes these models to each client node. The client node then selects the model that best matches its local data distribution based on its training data. Furthermore, after each client node uploads the posterior distribution of the corresponding model's parameters, the aggregation node aggregates the models accordingly. This allows for model aggregation based on data distribution type, ensuring that the prior distribution of the final model's parameters more accurately reflects the distribution of training data on the client nodes. This solves the problem of inconsistent training data distribution across different nodes and effectively improves the performance of the trained model.

[0016] In one possible implementation, the federated learning architecture includes multiple aggregation nodes and multiple client nodes, with each aggregation node responsible for one model and the first node being one of the multiple client nodes.

[0017] The first node obtains the prior distributions of parameters for multiple models, including: the first node receives the prior distributions of parameters for different models from multiple nodes respectively, to obtain the prior distributions of parameters for multiple models. The prior distribution of the parameters for the first model is received by the first node from the second node. Therefore, after the first node trains and obtains the posterior distribution of the parameters for the first model, the first node sends the posterior distribution of the parameters for the first model to the second node.

[0018] In summary, this solution involves multiple aggregation nodes pre-building models corresponding to different data distribution types based on the potential data distribution on each client node. These aggregation nodes then distribute these models to their respective client nodes, which select the model that best matches their local data distribution based on their training data. Furthermore, after each client node uploads the posterior distribution of its model's parameters to its corresponding aggregation node, the aggregation node aggregates the models accordingly. This data-distribution-based aggregation ensures that the prior distribution of the final model's parameters more accurately reflects the distribution of training data on the client nodes, thus resolving the issue of inconsistent training data distribution across different nodes and effectively improving the performance of the trained models.

[0019] Meanwhile, when client nodes obtain the posterior distribution of model parameters during training, they only need to send the posterior distribution of model parameters to the aggregation node corresponding to that model. This avoids all client nodes sending the posterior distribution of model parameters to the same aggregation node, thus preventing network congestion, reducing the processing burden on aggregation nodes, lowering the risk of failure of a single aggregation node, and improving information security.

[0020] In one possible implementation, the performance of each model when processing training data includes one or more of the following: model accuracy (i.e., model precision), model confidence, model convergence speed, and the gradient forward direction of the model during training.

[0021] In one possible implementation, the first node trains to obtain the posterior distribution of the parameters of the first model based on the prior distribution of the parameters and the training data. Specifically, this includes: the first node trains to obtain the posterior distribution of the target parameters in the first model based on the prior distribution of the parameters of the first model, the training data, and the selection probability of each parameter in the first model. The selection probability of each parameter indicates the probability of selecting each parameter as the target parameter in the first model. The target parameters are a subset of all parameters in the first model. In other words, the target parameters are a subset of all parameters in the first model selected based on the selection probability of each parameter; these are the parameters that need to be retained during the training of the first model. The other parameters besides the target parameters are the parameters that need to be discarded during the training of the first model.

[0022] After obtaining the posterior distribution of the target parameters in the first model through training, the first node sends the posterior distribution of the target parameters in the first model to the second node, thereby reducing the amount of communication between the first node and the second node.

[0023] In this scheme, during the training process of the model by the client node, sparsification parameters are introduced to filter the original parameters in the model, thereby eliminating some parameters in the model. This can effectively reduce the number of parameters in the model, reduce the amount of computation in the training process, and reduce the amount of communication between nodes, thus effectively improving the efficiency of federated learning.

[0024] In one possible implementation, the selection probability of each parameter in the first model is a probability value that can change dynamically during training. That is, the selection probability of each parameter in the first model can change as training progresses, rather than being a fixed value.

[0025] In this way, during the training process, the first node learns the selection probability of each parameter while learning the posterior distribution of the parameters. This allows the selection probability of each parameter to be automatically adjusted according to the training data, thereby better learning the optimal parameter sparsity result and ensuring the performance of the final trained model.

[0026] In one possible implementation, the prior distribution of the parameters of the first model is either the probability distribution of the parameters of the first model or the probability distribution of the probability distribution of the parameters of the first model.

[0027] A second aspect of this application provides a federated learning method applied to an aggregation node in federated learning. The method includes: a second node sending a prior distribution of parameters of a first model to multiple first nodes, wherein the first model is a machine learning model whose parameters follow a distribution. The second node is the aggregation node, and the multiple first nodes are all client nodes.

[0028] Then, the second node receives the posterior distribution of the parameters of the first model sent by some of the multiple first nodes. These multiple first nodes also obtain the prior distribution of the parameters of other models besides the first model, and based on the training data, they select one model from the multiple models to train, thereby obtaining the posterior distribution of the parameters of the selected model.

[0029] The second node updates the prior distribution of the parameters of the first model based on the posterior distribution of the parameters of the first model, thus obtaining the updated prior distribution of the parameters of the first model. In other words, after obtaining the posterior distribution of the parameters of the first model sent by a portion of the first node, the second node updates the prior distribution of the parameters of the first model based on the posterior distribution of the parameters of the first model sent by the first node.

[0030] The second node then sends the prior distribution of the updated parameters of the first model to some of the first nodes, so that some of the first nodes can perform the next round of model training based on the prior distribution of the updated parameters of the first model. Alternatively, the second node can also send the prior distribution of the updated parameters of the first model to multiple first nodes, so that each first node can continue to select the appropriate model to perform the next round of model training.

[0031] In this scheme, federated learning is achieved by exchanging the prior and posterior distributions of model parameters between nodes. This enables the learning of the data distribution of training data in nodes during model training. Furthermore, when a node obtains multiple models corresponding to different data distributions, it selects the model that best matches the training data distribution from among the multiple models based on the performance of each model in processing training data. This solves the problem of different training data distributions on different nodes and can effectively improve the performance of the trained model.

[0032] In one possible implementation, the federated learning architecture includes an aggregation node and multiple client nodes. The second node is the aggregation node, and the multiple first nodes are the client nodes. The method further includes: the second node sending the prior distributions of parameters for multiple models to the multiple first nodes, the prior distributions of parameters for the multiple models including the prior distribution of parameters for the first model; and the second node receiving indication information sent by some of the first nodes, which indicates that the posterior distribution of parameters sent by some of the first nodes corresponds to the first model.

[0033] In other words, the second node sends the prior distributions of the parameters of multiple models to each of the multiple first nodes; and some of the first nodes select a first model and send the posterior distribution of the parameters of the trained first model to the second node. In this way, the second node updates the prior distribution of the parameters of the first model based on the posterior distribution of the parameters of the first model sent by these first nodes.

[0034] In one possible implementation, the method further includes: a second node receiving a posterior distribution of parameters of a second model sent by another portion of the multiple first nodes, wherein the second model is one of the multiple models. The second node updates the prior distribution of the parameters of the second model based on the posterior distribution of the parameters of the second model.

[0035] In other words, when the second node sends the prior distributions of parameters for multiple models to each of the multiple first nodes, some first nodes choose to train the first model, while others choose to train the second model. Ultimately, the second node updates the prior distribution of the model's parameters based on the posterior distribution of the parameters of the same model, according to the model selected and trained by each first node.

[0036] In one possible implementation, the federated learning architecture includes multiple aggregation nodes and multiple client nodes, with each aggregation node responsible for one model. A second node is one of the multiple aggregation nodes, and each aggregation node is used to send the prior distribution of the model's parameters to multiple first nodes, with different aggregation nodes sending different prior distributions of the model's parameters.

[0037] In one possible implementation, the second node receives the posterior distribution of the parameters of the first model sent by a portion of the first nodes from among a plurality of first nodes, including: the second node receiving the posterior distribution of some parameters of the first model sent by a portion of the first nodes from among a plurality of first nodes; the second node updates the prior distribution of the parameters of the first model according to the posterior distribution of the parameters of the first model, including: the second node updating the prior distribution of the parameters of the first model according to the posterior distribution of some parameters of the first model.

[0038] In one possible implementation, the prior distribution of the parameters of the first model is either the probability distribution of the parameters of the first model or the probability distribution of the probability distribution of the parameters of the first model.

[0039] A third aspect of this application provides a federated learning device, which belongs to a first node, comprising:

[0040] The acquisition module is used to obtain the prior distribution of parameters for multiple models;

[0041] The processing module is used to determine the performance of each of the multiple models when processing the training data, based on the prior distribution of the parameters of multiple models and the training data of the device.

[0042] The processing module is also used to train the posterior distribution of the parameters of the first model based on the prior distribution of the parameters of the first model and the training data, wherein the first model is one of multiple models, and the first model is determined among multiple models based on the performance of each model when processing the training data;

[0043] The sending module is used to send the posterior distribution of the parameters of the first model to the second node.

[0044] In one possible implementation, the acquisition device is specifically used to receive the prior distribution of parameters of multiple models from the second node;

[0045] The sending module is also used to send indication information from the first node to the second node. The indication information is used to indicate that the posterior distribution of the parameters sent by the first node corresponds to the first model.

[0046] In one possible implementation, the acquisition device is specifically used to receive the prior distributions of parameters of different models from multiple nodes respectively, so as to obtain the prior distributions of parameters of multiple models, wherein the prior distribution of parameters of the first model is received by the first node from the second node.

[0047] In one possible implementation, the processing module is specifically used for:

[0048] Based on the prior distribution of parameters of multiple models, the parameter values ​​of each model in the multiple models are sampled.

[0049] Based on the parameter values ​​and training data of each model, determine the performance of each model when processing the training data.

[0050] In one possible implementation, the processing module is specifically used to train a posterior distribution of the target parameters in the first model based on the prior distribution of the parameters of the first model, the training data, and the selection probability of each parameter in the first model, wherein the selection probability of each parameter is used to indicate the probability of selecting each parameter as the target parameter in the first model, and the target parameters are a subset of all parameters in the first model.

[0051] The sending module is specifically used to send the posterior distribution of the target parameters in the first model to the second node.

[0052] The fourth aspect of this application provides a federated learning device, which belongs to a second node, comprising:

[0053] The sending module is used to send the prior distribution of the parameters of the first model to multiple first nodes;

[0054] The receiving module is used to receive the posterior distribution of the parameters of the first model sent by a portion of the multiple first nodes;

[0055] The processing module is used to update the prior distribution of the parameters of the first model based on the posterior distribution of the parameters of the first model, so as to obtain the updated prior distribution of the parameters of the first model.

[0056] The sending module is also used to send the prior distribution of the updated parameters of the first model to some of the first nodes.

[0057] In one possible implementation, the sending module is further configured to send the prior distribution of parameters of multiple models to multiple first nodes, wherein the prior distribution of parameters of multiple models includes the prior distribution of parameters of the first model.

[0058] The receiving module is also used to receive indication information sent by a portion of the first nodes, the indication information being used to indicate that the posterior distribution of the parameters sent by the portion of the first nodes corresponds to the first model.

[0059] In one possible implementation, the second node is one of a plurality of aggregation nodes, each of which is used to send the prior distribution of the model's parameters to a plurality of first nodes, and different aggregation nodes send the prior distribution of the parameters of different models.

[0060] In one possible implementation, the receiving module is further configured to receive the posterior distribution of some parameters in the first model sent by some of the multiple first nodes;

[0061] The processing module is also used to update the prior distribution of the parameters of the first model based on the posterior distribution of some parameters in the first model.

[0062] The fifth aspect of this application provides a federated learning apparatus, which may include a processor and a memory coupled together. The memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method of any implementation of the first or second aspect described above is implemented. For details regarding the steps of the various possible implementations of the first or second aspect executed by the processor, please refer to the first or second aspect; further details will not be provided here.

[0063] The sixth aspect of this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method of any implementation of the first or second aspect described above.

[0064] The seventh aspect of this application provides a circuit system including a processing circuit configured to perform the method of any implementation of the first or second aspect described above.

[0065] The eighth aspect of this application provides a computer program product that, when run on a computer, causes the computer to perform any implementation of the first or second aspect described above.

[0066] A ninth aspect of this application provides a chip system including a processor for supporting a server or threshold value acquisition device in implementing the functions involved in any implementation of the first or second aspect described above, such as transmitting or processing data and / or information involved in the methods described above. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the server or communication device. This chip system may be composed of chips or may include chips and other discrete devices.

[0067] The beneficial effects of aspects two through nine mentioned above can be referred to the introduction of aspect one above, and will not be repeated here. Attached Figure Description

[0068] Figure 1A schematic diagram illustrating a scenario of federated learning provided for an embodiment of this application;

[0069] Figure 2 This is a schematic diagram illustrating the execution process of federated learning in related technologies.

[0070] Figure 3 This is a schematic diagram of the structure of a convolutional neural network provided in an embodiment of this application;

[0071] Figure 4 This is a schematic diagram of another convolutional neural network structure provided in an embodiment of this application;

[0072] Figure 5 A schematic diagram of a system architecture 500 provided for an embodiment of this application;

[0073] Figure 6 A schematic diagram illustrating the execution flow of a federated learning method provided in an embodiment of this application;

[0074] Figure 7A A schematic diagram illustrating the process of federated learning performed by an aggregation node and multiple client nodes, provided for an embodiment of this application;

[0075] Figure 7B A schematic diagram of an aggregation node update model provided in an embodiment of this application;

[0076] Figure 8A A schematic diagram illustrating a process for multiple aggregation nodes and multiple client nodes to perform federated learning, provided for an embodiment of this application;

[0077] Figure 8B A schematic diagram of a multiple aggregation node update model provided in an embodiment of this application;

[0078] Figure 9 This is a schematic diagram of the structure of a parameter-sparsed model provided in an embodiment of this application;

[0079] Figure 10 A schematic diagram of the structure of a federated learning device provided in an embodiment of this application;

[0080] Figure 11 A schematic diagram of the structure of a federated learning device provided in an embodiment of this application;

[0081] Figure 12 A schematic diagram of the structure of the execution device provided in the embodiments of this application;

[0082] Figure 13 A schematic diagram of the structure of a chip provided in an embodiment of this application;

[0083] Figure 14This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application are described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some, and not all, of the embodiments of this application. Those skilled in the art will understand that, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0085] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such descriptions can be used interchangeably where appropriate to allow embodiments to be implemented in a sequence other than that illustrated or described in this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved. The division of units in this application is a logical division. In practical applications, there may be other division methods. For example, multiple units may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the shown or discussed mutual coupling, direct coupling, or communication connection may be through some interface, and the indirect coupling or communication connection between units may be electrical or other similar forms, none of which are limited in this application. Furthermore, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed among multiple circuit units. Some or all of the units can be selected to achieve the purpose of the solution in this application according to actual needs.

[0086] For ease of understanding, some technical terms involved in the embodiments of this application will be introduced below.

[0087] (1) Federated Learning

[0088] Federated learning is essentially a model training method that enables data sharing and collaborative model building while ensuring data privacy, security, and legal compliance. The core idea of ​​federated learning is that when multiple data sources participate in model training, the original data does not need to be transferred; instead, the model is trained jointly only by exchanging intermediate parameters. The original data can remain locally. This approach achieves a balance between data privacy protection and data sharing analysis, resulting in a data application model that is "usable but not visible."

[0089] For example, please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating a federated learning scenario provided in an embodiment of this application. The federated learning scenario may include multiple client nodes and a central node, where the client nodes and central node can be any node supporting data transmission (such as a network node). For example, a client node can be a mobile terminal or a personal computer. The central node can be a server or a cluster server. In some embodiments, the client nodes may be referred to as the owners of the training data, and the central node may be referred to as the coordinator of the federated learning process.

[0090] The central node is used to maintain the federated model. Client nodes can obtain the federated model from the central node and train it locally using their local training data to obtain a local model. After training the local model, the client node can send it to the central node so that the central node can update or optimize the federated model. This process is repeated multiple times until the federated model converges or reaches a preset iteration stopping condition.

[0091] The following will combine Figure 2 This section introduces the general process of federated learning. For example... Figure 2 As shown, the execution process of federated learning includes the following steps 201-205.

[0092] Step 201: The central node constructs the federated model.

[0093] The central node can build general machine learning models or specific machine learning models according to needs. Taking image recognition tasks as an example, the central node can build a convolutional neural network (CNN) as a federated model.

[0094] Step 202: Multiple client nodes obtain or receive the federated model from the central node.

[0095] In this system, multiple client nodes can obtain or receive the same federated model from the central node. For example, in one implementation, a client node can proactively request the central node to distribute the federated model. Alternatively, in another implementation, the central node proactively distributes the federated model to the client nodes. Taking a personal computer as the client node and a server as the central node, the personal computer can download the federated model from the server.

[0096] Step 203: Multiple client nodes train the federated model using local training data to obtain a local model.

[0097] Specifically, multiple client nodes can use the federated model as the initial model for their local model, and then use the local training data to train the initial model in one or more steps to obtain the local model.

[0098] Because different client nodes have different local training data, different client nodes can train different local models based on the same federated model.

[0099] Step 204: Multiple client nodes upload the trained local models to the central node.

[0100] Step 205: The central node aggregates multiple local models to obtain the updated federated model.

[0101] For example, in one implementation, the central node can perform a weighted summation of the parameters of the local models of multiple client nodes and use the result of the weighted summation as the updated federated model.

[0102] It should be noted that the process described in steps 201-205 can be viewed as an iteration in the federated learning process. The central node and client nodes can repeat steps 201-205 until the federated model converges or achieves the preset effect.

[0103] Generally, federated learning can be used to train machine learning models. The most common machine learning model is the neural network. The relevant concepts of neural networks will be explained below.

[0104] (2) Neural Network

[0105] A neural network can be composed of neural units, which can be defined as a computational unit that takes xs (i.e., input data) and an intercept of 1 as input. The output of this computational unit can be:

[0106] ;

[0107] Where s = 1, 2, ..., n, where n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer, and the activation function can be the sigmoid function. A neural network is a network formed by connecting multiple of the above-mentioned individual neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0108] (3) Deep Neural Networks

[0109] Deep Neural Networks (DNNs), also known as multilayer neural networks, can be understood as neural networks with many hidden layers, though there's no specific metric for "many." DNNs can be categorized into three layers based on their position: input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. All layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer. Although DNNs appear complex, the operation of each layer is actually quite simple, resembling a linear relationship as follows: ,in, It is the input vector. It is the output vector. It is an offset vector. It is the weight matrix (also called coefficients). ( ) is the activation function. Each layer is simply an activation function applied to the input vector. The output vector is obtained through such a simple operation. .

[0110] Because DNNs have many layers, the coefficients... and offset vector The number is therefore quite large. These parameters in DNN are defined as follows: (in terms of coefficients) For example: Suppose in a three-layer DNN, the linear coefficient from the fourth neuron in the second layer to the second neuron in the third layer is defined as... The superscript 3 represents the coefficient. The layer number is used, and the index corresponds to the output third layer index 2 and the input second layer index 4. In summary: the coefficients from the k-th neuron in layer L-1 to the j-th neuron in layer L are defined as... It's important to note that the input layer does not have... Parameters. In deep neural networks, more hidden layers allow the network to better depict complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can accomplish more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers in the trained deep neural network (a weight matrix formed by vectors W from many layers).

[0111] (4) Convolutional Neural Network (CNN)

[0112] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers. This feature extractor can be viewed as a filter, and the convolution process can be seen as using a trainable filter to convolve with an input image or feature map. A convolutional layer refers to the layer of neurons in a CNN that performs convolution processing on the input signal (e.g., the first and second convolutional layers in this embodiment). In a convolutional layer of a CNN, a neuron can be connected to only some of the neurons in neighboring layers. A convolutional layer typically contains several feature planes, each of which can be composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The underlying principle is that the statistical information of one part of the image is the same as that of other parts. This means that image information learned in one part can also be used in another part. Therefore, for all locations on the image, we can use the same learned image information. In the same convolutional layer, multiple convolutional kernels can be used to extract different image information. Generally, the more convolutional kernels there are, the richer the image information reflected by the convolution operation.

[0113] Convolutional kernels can be initialized as matrices of random size, and during the training of a convolutional neural network, they can learn appropriate weights. Furthermore, sharing weights directly reduces the number of connections between layers in the convolutional neural network, while also lowering the risk of overfitting.

[0114] Specifically, such as Figure 3 As shown, the convolutional neural network (CNN) 100 may include an input layer 110, a convolutional / pooling layer 120, wherein the pooling layer is optional, and a neural network layer 130.

[0115] The structure consisting of the convolutional layer / pooling layer 120 and the neural network layer 130 can be the first convolutional layer and the second convolutional layer described in this application. The input layer 110 is connected to the convolutional layer / pooling layer 120, and the convolutional layer / pooling layer 120 is connected to the neural network layer 130. The output of the neural network layer 130 can be input to the activation layer, and the activation layer can perform non-linear processing on the output of the neural network layer 130.

[0116] Convolutional / pooling layers: 120. Convolutional layers: such as... Figure 3 The convolutional / pooling layer 120 shown may include layers 121-126 as in Examples 121-126. In one implementation, layer 121 is a convolutional layer, layer 122 is a pooling layer, layer 123 is a convolutional layer, layer 124 is a pooling layer, layer 125 is a convolutional layer, and layer 126 is a pooling layer. In another implementation, layers 121 and 122 are convolutional layers, layer 123 is a pooling layer, layers 124 and 125 are convolutional layers, and layer 126 is a pooling layer. That is, the output of the convolutional layer can be used as the input of a subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.

[0117] Taking convolutional layer 121 as an example, it can include multiple convolution operators, also known as kernels. In image processing, a convolution operator acts as a filter, extracting specific information from the input image matrix. Essentially, a convolution operator can be a weight matrix, which is usually predefined. During the convolution operation, the weight matrix processes the input image pixel by pixel (or two pixels by two pixels, depending on the stride) along the horizontal direction, thus extracting specific features. The size of the weight matrix should be related to the image size. It's important to note that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image; during convolution, the weight matrix extends to the entire depth of the input image. Therefore, convolution with a single weight matrix produces a single-depth convolutional output. However, in most cases, multiple weight matrices of the same dimension are applied instead of a single weight matrix. The outputs of each weight matrix are stacked to form the depth dimension of the convolutional image. Different weight matrices can be used to extract different features from an image. For example, one weight matrix can be used to extract image edge information, another weight matrix can be used to extract specific colors from the image, and yet another weight matrix can be used to blur unwanted noise in the image. These multiple weight matrices have the same dimension, and the feature maps extracted by these multiple weight matrices with the same dimension also have the same dimension. Then, the multiple feature maps with the same dimension are merged to form the output of the convolution operation.

[0118] The weight values ​​in these weight matrices need to be obtained through extensive training in practical applications. The weight matrices formed by the weight values ​​obtained through training can extract information from the input image, thereby helping the convolutional neural network 100 to make correct predictions.

[0119] When a convolutional neural network 100 has multiple convolutional layers, the initial convolutional layers (e.g., 121) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 100 increases, the features extracted by later convolutional layers (e.g., 126) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0120] Pooling layers: Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, such as... Figure 3 In the example of 120, each layer 121-126 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers.

[0121] Neural Network Layer 130: After processing by the convolutional / pooling layers 120, the convolutional neural network 100 is still insufficient to output the required output information. As before, the convolutional / pooling layers 120 only extract features and reduce the parameters introduced by the input image. However, to generate the final output information (the required class information or other relevant information), the convolutional neural network 100 needs to utilize neural network layer 130 to generate one or a set of required class numbers of output. Therefore, neural network layer 130 may include multiple hidden layers (such as...). Figure 3 As shown in 131, 132 to 13n) and output layer 140, the parameters contained in these multi-layer hidden layers can be pre-trained based on relevant training data for specific task types, such as image recognition, image classification, image super-resolution reconstruction, etc.

[0122] After the multiple hidden layers in neural network layer 130, the final layer of the entire convolutional neural network 100 is the output layer 140. This output layer 140 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 100 has undergone forward propagation (such as...), the loss function is applied. Figure 4 The propagation from 110 to 140 is completed (forward propagation), and the reverse propagation (such as...) Figure 4 The propagation from 140 to 110 (backpropagation) will begin to update the weight values ​​and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 100 and the error between the output of the convolutional neural network 100 through the output layer and the ideal result.

[0123] It should be noted that, as Figure 3 The convolutional neural network 100 shown is merely an example of a convolutional neural network. In specific applications, convolutional neural networks can also exist in the form of other network models, such as... Figure 4 The multiple convolutional / pooling layers shown are parallelized, and the extracted features are all input into the full neural network layer 130 for processing.

[0124] (5) Bayesian Neural Network

[0125] Unlike typical neural networks, Bayesian neural networks use random variables as their weights, rather than fixed values, following a certain probability distribution. This type of neural network can not only provide predicted values ​​but also the uncertainty of those predictions.

[0126] (6) Loss function

[0127] During neural network training, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value and update the weight vector of each layer based on the difference. (Of course, there's usually an initialization process before the first update, where parameters are pre-configured for each layer.) For example, if the network's prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the neural network can predict the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, so training the neural network becomes a process of minimizing this loss as much as possible.

[0128] (7) Backpropagation algorithm

[0129] Neural networks can employ backpropagation (BP) to correct the initial parameters of the model during training, thereby minimizing the model's error loss. Specifically, forward propagation of the input signal to the output generates error loss; this error loss information is then propagated back to update the initial model parameters, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining optimal model parameters, such as the weight matrix.

[0130] (8) Non-independent and identically distributed (Non-IID)

[0131] In probability theory and statistics, independent and identically distributed (IID) means that each variable in a set of random variables has the same probability distribution, and these random variables are independent of each other. However, an IID set of random variables does not mean that every event in their sample space has the same probability of occurring. For example, the sequence of results from rolling a non-fair die is IID, but the probability of rolling each face up is not the same.

[0132] So, Non-IID means that the variables are not independent or are not identically distributed. In federated learning, Non-IID generally refers to situations where the data does not conform to the same distribution, because the data distributions are certainly independent, but they do not necessarily follow the same sampling method. For example, if the entire dataset contains 100 types of images, one device contains only landscape images, while another device contains only images of people and plants. The former follows one distribution (1 / 100), and the latter follows another distribution (2 / 100). Conversely, if one device contains all 100 types of images, and other devices also contain all 100 types of images, then they are identically distributed.

[0133] (9) Bayesian learning

[0134] Bayesian learning directly derives the population distribution by utilizing the prior distribution of parameters and the posterior distribution derived from sample information. Bayesian learning theory uses probability to represent all forms of uncertainty, implementing the learning and inference process through probability rules. Bayesian learning has unique advantages in handling small sample data and is a natural fit with federated learning.

[0135] In the process of applying Bayesian learning to model training, it is assumed that the model parameters to be estimated are random variables that follow a certain distribution. We can first give the prior distribution (also known as the subjective distribution) of the parameters to be estimated based on experience, that is, the distribution that the parameters to be estimated themselves follow (such as Gaussian distribution). Then, based on the prior distribution of the parameters to be estimated and combined with the sample information, we apply Bayes' theorem to find the posterior distribution of the parameters to be estimated.

[0136] (10) Neural networks whose parameters follow a distribution

[0137] Neural networks with parameters following a distribution are a type of machine learning model where parameters follow a distribution. Specifically, the parameters of a traditional neural network (such as the weights of neurons mentioned earlier) are fixed values. However, this type of neural network suffers from overfitting; that is, it tends to make overly confident predictions in regions where training data is missing, and cannot accurately measure the uncertainty of the prediction results.

[0138] Compared to neural networks with fixed parameters, some neural networks have parameters that follow a specific distribution. For example, the parameters of a Bayesian neural network are random variables following a certain distribution, such as a Gaussian distribution. The training process for neural networks with parameters following a probability distribution is not aimed at obtaining fixed parameter values, but rather at optimizing the probability distribution of the parameters. After training, the parameter distribution can be sampled, with each sampled value corresponding to a neural network with fixed parameters. If a large number of sampled neural networks make similar predictions for a given input, then the neural network can be considered to have low uncertainty in its prediction of that input; otherwise, the uncertainty in its prediction is high. In this way, neural networks with parameters following a probability distribution can characterize the uncertainty in predictions due to missing data, thus avoiding the problem of overfitting.

[0139] (11) Prior distribution, posterior distribution and likelihood estimation

[0140] The training problem of a machine learning model whose parameters follow a probability distribution can be viewed as an estimation problem of the probability distribution of the parameters based on Bayes' theorem. In Bayes' theorem, the prior distribution, the posterior distribution, and the likelihood estimation are three important concepts.

[0141] The prior distribution of parameters is a pre-assumption about the posterior distribution; that is, it refers to the assumption about the posterior distribution of parameters before the training data is observed. The prior distribution can be manually specified (e.g., randomly initialized) or learned through data. In contrast, the posterior distribution describes the distribution of parameters after the training data has been observed. In other words, it describes the distribution of parameters given the training data. According to Bayes' theorem, the prior distribution, posterior distribution, and likelihood estimate of parameters satisfy the following relationship: Posterior distribution = (Prior distribution × Likelihood estimate) / Probability of the training data occurring.

[0142] (12) Parametric and nonparametric descriptions of parameter distribution

[0143] Both the prior and posterior distributions of parameters describe the distribution of parameters, but there can be many ways to describe the distribution of parameters, and the embodiments of this application do not limit this.

[0144] In some embodiments, the prior and / or posterior distributions of the parameters can be described using parametric distribution methods. For example, assuming the parameter distribution is Gaussian, the prior and / or posterior distributions can be described using the mean and variance. In other embodiments, the prior and / or posterior distributions can also be described using non-parametric distribution methods. For example, the prior and / or posterior distributions of the parameters can be described using probability histograms, probability densities, cumulative function curves, etc.

[0145] (13) The "point description" and "distribution description" of the prior distribution to the posterior distribution.

[0146] The prior distribution of model parameters can be either the probability distribution of the model parameters or the probability distribution of the probability distribution of the model parameters.

[0147] There is a relationship between prior and posterior distributions; the prior distribution can be viewed as a pre-defined description of the posterior distribution, a hypothetical description made before the training data is observed. If the prior distribution of the model parameters is the probability distribution of the model parameters, this type of prior distribution can be understood as a "point description" of the posterior distribution; if the prior distribution of the model parameters is the probability distribution of the probability distribution of the model parameters, this type of prior distribution can be understood as a "distribution description" of the posterior distribution.

[0148] For example, assuming the model parameters follow a Gaussian distribution, when the prior distribution of the model parameters is the probability distribution of the model parameters, the prior distribution of the model parameters can be the mean and variance of the model parameter distribution. From the perspective that the prior distribution describes the posterior distribution, it is equivalent to the prior distribution using a point [mean, variance] to "describe" the posterior distribution.

[0149] For example, assuming the model parameters follow a Gaussian distribution, when the prior distribution of the model parameters is the probability distribution of the model parameter probability distribution, the prior distribution of the model parameters does not give the mean and variance of the model parameter distribution, but rather describes the probability that the mean and variance of the model parameter distribution take different values. From the perspective that the prior distribution describes the posterior distribution, it is equivalent to the prior distribution using a probability distribution to "describe" the probability (or the penalty or reward) of the posterior distribution taking different values ​​of the mean and variance.

[0150] (14) Measurement of the difference between two distributions

[0151] Some embodiments of the present application relate to the measurement of the difference between the prior distribution and the posterior distribution. There can be various ways to measure the difference between the prior distribution and the posterior distribution, and different distribution difference measurement functions can be designed according to the different ways of describing the posterior distribution by the prior distribution to measure the difference between the two distributions. Several examples are given below.

[0152] As an example, if the prior distribution uses "point description" for the posterior distribution and the prior distribution uses a parameterized distribution description method, the difference between the prior distribution and the posterior distribution can be measured by the KL divergence (Kullback - Leibler divergence) of the two distributions. In other words, the KL divergence of the prior distribution and the posterior distribution can be used as the distribution difference measurement function for the two distributions.

[0153] As another example, if the prior distribution uses "point description" and the prior distribution uses a non - parameterized distribution description method (such as description based on histograms, probability density curves, etc.), the difference between the prior distribution and the posterior distribution can be measured by calculating the similarity of the histograms (or probability density curves) corresponding to the two distributions. In other words, the similarity of the histograms (or probability density curves) corresponding to the prior distribution and the posterior distribution can be used as the distribution difference measurement function for the two distributions. The similarity of the histograms (or probability density curves) corresponding to the two distributions can be obtained by calculating the difference in the areas of the two histograms (or probability density curves) or the cosine distance.

[0154] As yet another example, if the prior distribution uses "distribution description" for the posterior distribution, the probability of the prior distribution at the values of the posterior distribution can be used to describe the difference between the two distributions. In other words, the probability of the prior distribution at the values of the posterior distribution can be used as the distribution difference measurement function for the two distributions.

[0155] (15)The Bernoulli distribution (

[0156] The Bernoulli distribution, also known as the two - point distribution or the 0 - 1 distribution, is a discrete probability distribution. If the random variable X only takes two values, 0 and 1, and the probability that the random variable X takes the value 1 is p (0 < p < 1), and the probability that the random variable takes the value 0 is 1 - p, then it can be said that the random variable X follows the Bernoulli distribution with parameter p.

[0157] The number of successes in a Bernoulli trial follows the Bernoulli distribution, and the parameter p is the probability of success in the trial. The Bernoulli distribution is a discrete probability distribution and is a special case of the binomial distribution when N = 1.

[0158] (16)Monte Carlo method

[0159] The Monte Carlo method, also known as statistical simulation or statistical experiment, is a numerical simulation method that takes probabilistic phenomena as its research object. It is a calculation method that uses sampling surveys to obtain statistical values ​​to estimate unknown characteristic quantities.

[0160] The basic idea of ​​the Monte Carlo method is as follows: To solve a problem, first, a probabilistic model or stochastic process is established, whose parameters or numerical characteristics are equal to the solution to the problem. Then, these parameters or numerical characteristics are calculated through observation or sampling experiments on the model or process. Finally, an approximate value of the solution is given. The accuracy of the solution is expressed by the standard error of the estimated value. The main theoretical basis of the Monte Carlo method is probability and statistics theory, and its main methods are random sampling and statistical experiments. Generally, the basic steps for solving practical problems using the Monte Carlo method are: 1. Based on the characteristics of the practical problem, construct a simple and easily implementable probabilistic and statistical model so that the solution obtained is exactly the probability distribution or mathematical expectation of the problem; 2. Give the sampling methods for various random variables with different distributions in the model; 3. Statistically process the simulation results and give the statistical estimate and accuracy estimate of the solution to the problem.

[0161] (17) Repeated parameters

[0162] Reparameterization involves sampling from a distribution p with parameters θ. Direct sampling (which is discrete and non-differentiable) lacks gradient information, meaning the gradients won't be updated during backpropagation (BP). Reparameterization ensures sampling from distribution p while preserving gradient information.

[0163] For ease of understanding, the system architecture applied to the federated learning method provided in the embodiments of this application will be introduced below.

[0164] Please see Figure 5 , Figure 5 This is a schematic diagram of a system architecture 500 provided in an embodiment of this application. For example... Figure 5 As shown, in this system architecture 500, the aggregation node 501 is implemented by one or more servers. Optionally, the aggregation node 501 can work with other computing devices, such as data storage, routers, load balancers, etc. The aggregation node 501 can be deployed on a single physical site or distributed across multiple physical sites.

[0165] Users can interact with aggregation node 501 by operating their own user devices (such as client node 502 and client node 503). Each client node can represent any computing device, such as a personal computer, computer workstation, smartphone, tablet, smart camera, smart car or other type of cellular phone, media consumption device, wearable device, set-top box, game console, etc.

[0166] Each client node can interact with the aggregation node 501 through any communication mechanism / standard communication network. The communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0167] In the specific implementation process, the aggregation node 501 sends the prior distribution of the model's parameters to each client node. Each client node then trains the model based on its local training data to obtain the posterior distribution of the model's parameters. Next, the client nodes upload the posterior distribution of the model's parameters to the aggregation node 501, which updates the prior distribution of the model's parameters. By repeatedly executing the aforementioned steps, the aggregation node 501 and each client node cooperate to train the model.

[0168] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the execution flow of a federated learning method provided in an embodiment of this application. Figure 6 As shown, the federated learning method includes the following steps 601-604.

[0169] Step 601: The first node obtains the prior distribution of parameters of multiple models.

[0170] In this embodiment, the first node can be, for example, the client node described above, used to train the acquired model based on the training data. The prior distribution of the parameters of the multiple models can be obtained by the first node from one or more nodes. For example, the first node obtains the prior distribution of the parameters of the multiple models from a second node; or, for another example, the first node obtains the prior distribution of the parameters of each model from multiple nodes respectively, to obtain the prior distribution of the parameters of the multiple models.

[0171] In this embodiment, the various models mentioned are all machine learning models whose parameters follow a distribution. In some embodiments, the model is a neural network whose parameters follow a distribution, and the parameters of the model can refer to the parameters of the neurons in the neural network. For example, the model can be a Bayesian neural network. Further, in some embodiments, the parameters in the Bayesian neural network can follow a Gaussian distribution, a delta distribution, or other distributions.

[0172] Furthermore, the aforementioned multiple models can have the same structure, but the distribution of parameters in the multiple models is not the same. Therefore, the prior distribution of parameters of the multiple models obtained by the first node actually includes the prior distribution of parameters of each of the multiple models.

[0173] In simple terms, suppose the above multiple models comprise five models, namely Model 1 to Model 5. Since Models 1 to 5 have the same structure, they all have the same number of parameters, for example, parameter 1 to parameter 1000. The distributions of the parameters in different models are not the same. For example, parameter 1 in Model 1 follows a Gaussian distribution A1; parameter 1 in Model 2 follows a Gaussian distribution A2… and parameter 1 in Model 5 follows a Gaussian distribution A5; similarly, parameter 2 in Model 1 follows a Gaussian distribution B1; parameter 2 in Model 2 follows a Gaussian distribution B2… and parameter 2 in Model 5 follows a Gaussian distribution B5.

[0174] Step 602: The first node determines the performance of each of the multiple models when processing the training data based on the prior distribution of the parameters of the multiple models and the training data of the first node.

[0175] In this embodiment, since the first node obtains the prior distribution of the parameters in the model, rather than the parameters with fixed values, the first node can sample each parameter in the model based on the prior distribution of each parameter, thereby obtaining the sampled values ​​of each parameter in the model. For example, if the prior distribution of the model's parameters is a Gaussian distribution, the first node can determine the range of values ​​for the model's parameters and the probability of each parameter taking each value based on the mean and variance of the Gaussian distribution; thus, the first node can sample the values ​​of the parameters based on the probabilities of each parameter taking each value, and thereby obtain the sampled values ​​of the parameters.

[0176] For example, the first node samples the parameter values ​​of each of the multiple models based on the prior distribution of the parameters, thus obtaining the specific values ​​of each parameter in each model. Then, based on the parameter values ​​of each model and the training data, the first node determines the performance of each model when processing the training data. After determining the parameter values ​​in each model, the first node obtains a model with fixed parameter values. Therefore, the first node can input the training data into each model separately, thereby obtaining the performance of each model when processing the training data.

[0177] In another possible example, regarding the prior distribution of parameters for the same model, the first node could be based on this prior distribution, sampling each parameter multiple times to obtain multiple sampled values ​​for each parameter, resulting in multiple sampled models with different parameter sampled values. The first node could then determine the performance of each of these sampled models with different parameter sampled values ​​when processing training data, and combine their performance to obtain the performance of the corresponding model. In this way, by sampling the parameters multiple times to obtain multiple sampled models and combining their performance to obtain the performance of the corresponding model, the randomness of random sampling can be effectively avoided, allowing for an accurate evaluation of the performance of each model based on its prior parameter distribution.

[0178] The training data for the first node can be training data stored locally on the first node; or it can be training data stored on a cloud server or database connected to the first node.

[0179] Optionally, the performance of each model when processing training data includes one or more of the following: model accuracy (i.e. model precision), model confidence, model convergence speed, and gradient forward direction during training.

[0180] Specifically, the training data in the first node can be labeled training data. The first node obtains the prediction result output by the model by inputting the training data into the model whose parameters are sampled values. In this way, the first node can determine whether the prediction result output by the model is accurate by comparing the prediction result output by the model with the label of the training data itself, and thus determine the accuracy of the model.

[0181] For example, in the case of an image classification model, after inputting the images used as training data into the model at the first node, the model outputs a classification prediction result. Then, the first node compares the classification prediction result with the image's original class label to determine whether the model's output classification prediction result is accurate. By inputting a large number of images into the model and determining the model's accuracy in processing these images, the model's accuracy can be obtained by combining the accuracy of the model in processing each image.

[0182] The confidence level of a model refers to the reliability of the prediction results output by the model. The higher the confidence level of the model, the higher the reliability of the prediction results output by the model, that is, the lower the uncertainty; the lower the confidence level of the model, the lower the reliability of the prediction results output by the model, that is, the higher the uncertainty.

[0183] The convergence speed of a model refers to how quickly the model approaches a local optimum (i.e., the converged model) during training. The faster the convergence speed, the closer the model's parameters are to the local optimum, and the better the model's performance; the slower the convergence speed, the further the model's parameters are from the local optimum, and the worse the model's performance.

[0184] The gradient direction during model training refers to the direction in which the function value decreases the most at each point. In the gradient method, the function value moves a certain distance along the gradient direction from the current position, then the gradient is recalculated at the new location, and the process is repeated along the new gradient direction. A better gradient direction during training results in faster model convergence and better model performance; conversely, a worse gradient direction results in slower convergence and worse model performance.

[0185] Step 603: The first node trains to obtain the posterior distribution of the parameters of the first model based on the prior distribution of the parameters of the first model and the training data. The first model is one of multiple models, and the first model is determined among multiple models based on the performance of each model when processing the training data.

[0186] After determining the performance of each of the multiple models when processing the training data, the first node can select the best-performing model from among the multiple models based on the performance of each model. The best performance of the first model when processing the training data means that the prior distribution of the parameters of the first model best matches the distribution of the training data in the first node; therefore, the first node selects the first model for further training.

[0187] Wherein, the prior distribution of the parameters of the first model is the probability distribution of the parameters of the first model, or the probability distribution of the probability distribution of the parameters of the first model. Similarly, when the prior distribution of the parameters of the first model is the probability distribution of the parameters of the first model, the posterior distribution of the parameters of the first model is the probability distribution of the parameters of the first model; when the prior distribution of the parameters of the first model is the probability distribution of the probability distribution of the parameters of the first model, the posterior distribution of the parameters of the first model is the probability distribution of the probability distribution of the parameters of the first model.

[0188] Step 604: The first node sends the posterior distribution of the parameters of the first model to the second node.

[0189] After obtaining the posterior distribution of the parameters of the first model through training, the first node sends the posterior distribution of the parameters of the first model to the second node so that the second node can update the prior distribution of the parameters of the first model based on the posterior distribution of the parameters of the first model.

[0190] The second node can be an aggregation node, used to update the prior distribution of the model's parameters based on the posterior distribution of the parameters of the model uploaded by each client node.

[0191] It is understandable that steps 601-604 above are described using the first node as a client node in the federated learning process. In the actual federated learning process, each client node can obtain the prior distribution of parameters from multiple models, and based on steps 601-604 above, select one model for training, and send the posterior distribution of the trained model's parameters to the corresponding aggregation node. In this way, the aggregation node can obtain the posterior distribution of the model parameters sent by different client nodes. Since different client nodes may select different models for training, the aggregation node updates the prior distribution of the model's parameters based on the posterior distribution of the parameters corresponding to the same model.

[0192] In this scheme, federated learning is achieved by exchanging the prior and posterior distributions of model parameters between nodes. This enables the learning of the data distribution of training data in nodes during model training. Furthermore, when a node obtains multiple models corresponding to different data distributions, it selects the model that best matches the training data distribution from among the multiple models based on the performance of each model in processing training data. This solves the problem of different training data distributions on different nodes and can effectively improve the performance of the trained model.

[0193] Furthermore, machine learning models with parameters following a distribution can predetermine the possible values ​​of the parameters, and these possible values ​​can characterize the merits of various possible improvement directions for the machine learning model. Therefore, federated learning on machine learning models with parameters following a distribution helps participating nodes find better improvement directions for the machine learning model, thereby reducing training time and communication overhead between nodes.

[0194] To facilitate understanding, the following will describe the complete execution flow of federated learning provided in this embodiment when applied in a specific scenario.

[0195] Scenario 1: The aggregation node sends the prior distribution of parameters of multiple models to each client node. The client node selects one of the models to train and uploads the posterior distribution of the trained model's parameters to the aggregation node. The aggregation node updates the prior distribution of the model's parameters based on the posterior distribution of the parameters of the same model uploaded by different client nodes.

[0196] For example, in step 601 above, the first node, acting as a client node, may receive the prior distribution of parameters from multiple models from the second node, acting as an aggregation node. Furthermore, in step 604 above, since the first node selects a first model from among multiple models for training, it sends an indication message to the second node simultaneously with the posterior distribution of the first model's parameters. This indication message indicates that the posterior distribution of the parameters sent by the first node corresponds to the first model. In other words, after selecting the first model from among multiple models, the first node, when uploading the posterior distribution of the first model's parameters, also needs to notify the second node that the model it selected is the first model.

[0197] Since the second node distributes the prior distributions of parameters for multiple models to multiple client nodes, some of these client nodes typically choose to train the first model among the multiple models, while others choose to train other models. Therefore, the second node can receive the posterior distribution of the parameters of the first model uploaded by some of the client nodes. In this way, the second node can update the prior distribution of the first model's parameters based on the posterior distribution of the parameters uploaded by the client nodes.

[0198] After the second node updates the prior distribution of the model parameters based on the posterior distribution of the model parameters uploaded by each client node, the second node can continue to send the prior distribution of the model parameters to each client node so that each client node can cyclically execute the training of the model, thereby completing the federated learning process.

[0199] It should be noted that the second node can be a prior distribution that does not need to send the parameters of multiple models to each client node in every iteration. Alternatively, the second node can send the parameters of multiple models to each client node only after several iterations. After the second node sends the parameters of multiple models to each client node, and the client node reports its selected training model to the second node, the second node will only send the parameters of that selected training model to that client node in subsequent iterations.

[0200] For example, suppose the second node sends the prior distribution of parameters for multiple models to each client node only after three iterations. After the second node sends the prior distribution of parameters for multiple models to each client node in the first iteration, and client node 1 reports back to the second node that it has selected model 1 for training, then in the subsequent second, third, and fourth iterations, the second node only sends the prior distribution of parameters for model 1 to client node 1. In the fifth iteration, the second node then sends the prior distribution of parameters for multiple models to client node 1 again.

[0201] For example, please refer to Figure 7A , Figure 7A This is a schematic diagram illustrating a process for federated learning performed by an aggregation node and multiple client nodes, as provided in an embodiment of this application. Figure 7A As shown, the process of federated learning performed by the aggregation node and multiple client nodes includes the following steps 701-705.

[0202] Step 701: The aggregation node sends the prior distribution of parameters of multiple models to client nodes 1-4.

[0203] The multiple models share the same structure, but the prior distributions of their parameters differ. Furthermore, in this embodiment, the multiple models include Model 1 and Model 2.

[0204] Before federated learning begins, the aggregation node can generate or initialize the prior distributions of the parameters in Model 1 and Model 2 based on the possible distributions of the data in each client node. Alternatively, the aggregation node can also generate the prior distributions of the parameters in Model 1 and Model 2 separately through random initialization; this embodiment does not impose specific limitations on this approach.

[0205] For example, the aggregation node is, for example, the second node in the above embodiment, and the client node 1 is, for example, the first node in the above embodiment.

[0206] Step 702: Client node 1 and client node 2 send the posterior distribution of the parameters of model 1 to the aggregation node, and client node 3 and client node 4 send the posterior distribution of the parameters of model 2 to the aggregation node.

[0207] Specifically, after each client node receives the prior distribution of parameters of multiple models sent by the aggregation node, each client node can select the model to be trained based on the prior distribution of parameters of each model, and further perform model training based on the training data and the prior distribution of parameters of the selected model.

[0208] In this embodiment, client node 1 and client node 2 select to train model 1, that is, the prior distribution of the parameters of model 1 matches the data distribution in client node 1 and client node 2; client node 3 and client node 4 select to train model 2, that is, the prior distribution of the parameters of model 2 matches the data distribution in client node 3 and client node 4.

[0209] Thus, after client nodes 1 and 2 obtain the posterior distribution of the parameters of model 1 through training, they send the posterior distribution of the parameters of model 1 to the aggregation node. Similarly, after client nodes 3 and 4 obtain the posterior distribution of the parameters of model 2 through training, they send the posterior distribution of the parameters of model 2 to the aggregation node.

[0210] It should be noted that client node 1 and client node 2 select the same model for training (i.e., both are model 1), but the posterior distributions of the model 1 parameters obtained by client node 1 and client node 2 are different. Similarly, the posterior distributions of the model 1 parameters obtained by client node 3 and client node 4 are different.

[0211] In addition, client node 1 and client node 2 send instruction information to the aggregation node to indicate that the model they selected for training is model 1; client node 3 and client node 4 also send instruction information to the aggregation node to indicate that the model they selected for training is model 2.

[0212] Step 703: The aggregation node updates the prior distribution of the parameters of model 1 based on the posterior distribution of the parameters of model 1 sent by client node 1 and client node 2, and updates the prior distribution of the parameters of model 2 based on the posterior distribution of the parameters of model 2 sent by client node 3 and client node 4.

[0213] It is understandable that, since client nodes 1 and 2 choose to train model 1, while client nodes 3 and 4 choose to train model 2, when the aggregation node updates the prior distribution of the parameters of model 1, it is based on the posterior distribution of the parameters of model 1 sent by client nodes 1 and 2; when the aggregation node updates the prior distribution of the parameters of model 2, it is based on the posterior distribution of the parameters of model 2 sent by client nodes 3 and 4.

[0214] In this way, the aggregation node can aggregate training data that belongs to or is close to the same distribution of client nodes that have trained models. This makes the prior distribution of the parameters of the model obtained by the aggregation node closer to the data distribution on each client node, thus solving the problem of different training data distributions on different nodes and effectively improving the performance of the trained model.

[0215] Step 704: The aggregation node sends the prior distribution of the parameters of model 1 to client node 1 and client node 2.

[0216] After the aggregation node updates the prior distribution of the parameters of model 1, the aggregation node can send the updated prior distribution of the parameters of model 1 to client node 1 and client node 2 so that client node 1 and client node 2 can continue to execute the training of model 1.

[0217] Step 705: The aggregation node sends the prior distribution of the parameters of model 2 to client node 3 and client node 4.

[0218] After the aggregation node updates the prior distribution of the parameters of model 2, the aggregation node can send the updated prior distribution of the parameters of model 2 to client nodes 3 and 4 so that client nodes 3 and 4 can continue to train model 2.

[0219] In summary, this solution involves the aggregation node pre-creating multiple models corresponding to different data distribution types based on the potential data distribution on each client node. These models are then distributed to each client node, which selects the model that best matches its local data distribution based on its training data. Furthermore, after each client node uploads the posterior distribution of its model parameters, the aggregation node aggregates the corresponding models. This data-distribution-based aggregation ensures that the prior distribution of the final model's parameters more accurately reflects the distribution of training data on the client nodes, thus resolving the issue of inconsistent training data distribution across different nodes and effectively improving the performance of the trained models.

[0220] For example, please refer to Figure 7B 7B is a schematic diagram of an aggregation node update model provided in an embodiment of this application. For example... Figure 7B As shown, in Figure 7B In (a), the aggregation node has the prior distributions of the parameters of model 1, model 2, and model 3, respectively. Furthermore, during the first iteration, the aggregation node sends the prior distributions of the parameters of model 1, model 2, and model 3 to each client node (i.e., client node 1 to client node N).

[0221] exist Figure 7BIn (b), after receiving the prior distributions of the parameters of the three models, each client node selects the model whose distribution best matches the training data based on the training data and the prior distributions of the parameters of the three models, and then sends the posterior distribution of the parameters of the trained model to the aggregation node. Specifically, client node 1 selects to train model 1 and sends the posterior distribution of the parameters of model 1 to the aggregation node; client node 2 selects to train model 2 and sends the posterior distribution of the parameters of model 2 to the aggregation node; client node 3 selects to train model 1 and sends the posterior distribution of the parameters of model 1 to the aggregation node… client node N selects to train model 3 and sends the posterior distribution of the parameters of model 3 to the aggregation node. In this way, the aggregation node can receive the posterior distributions of the parameters of the models uploaded by different client nodes.

[0222] exist Figure 7B In (c), the aggregation node updates the prior distribution of the model's parameters based on the posterior distribution of the parameters for the same model uploaded by different client nodes, thus obtaining the updated prior distribution of the model's parameters. Specifically, for model 1, the aggregation node updates the prior distribution of model 1's parameters based on the posterior distribution of the parameters for model 1 uploaded by different client nodes, obtaining the updated prior distribution of model 1's parameters. For model 2, the aggregation node updates the prior distribution of model 2's parameters based on the posterior distribution of the parameters for model 2 uploaded by different client nodes, obtaining the updated prior distribution of model 2's parameters. For model 3, the aggregation node updates the prior distribution of model 3's parameters based on the posterior distribution of the parameters for model 3 uploaded by different client nodes, obtaining the updated prior distribution of model 3's parameters.

[0223] Then, referring to the models selected and trained by each client node, the aggregation node sends the prior distribution of the updated model parameters to each client node. For example, the aggregation node sends the prior distribution of the updated model 1 parameters to client node 1; the aggregation node sends the prior distribution of the updated model 2 parameters to client node 2; the aggregation node sends the prior distribution of the updated model 1 parameters to client node 3; and the aggregation node sends the prior distribution of the updated model 3 parameters to client node N.

[0224] Scenario 2: The federated learning process is decentralized, that is, the federated learning process includes multiple aggregation nodes, each of which is responsible for aggregating models corresponding to different data distributions.

[0225] During federated learning, each aggregator node distributes the prior distribution of model parameters to each client node. This means each client node receives prior distributions of parameters from multiple models, and these prior distributions for different models originate from different aggregator nodes. Then, each client node selects one model for training and uploads the posterior distribution of the trained model's parameters to the corresponding aggregator node. The aggregator node updates the prior distribution of the model's parameters based on the posterior distributions of the same model uploaded by different client nodes.

[0226] For example, in step 601 above, the first node may receive prior distributions of parameters of different models from multiple nodes to obtain prior distributions of parameters of multiple models, wherein the prior distribution of parameters of the first model is received by the first node from the second node. In this way, after the first node selects to train the first model from multiple models and obtains the posterior distribution of parameters of the first model, the first node sends the posterior distribution of parameters of the first model to the second node.

[0227] In simple terms, compared to Scenario 1 where a single aggregation node is responsible for aggregating multiple models corresponding to different data distributions, Scenario 2 involves multiple aggregation nodes, each responsible for aggregating multiple models corresponding to different data distributions, with each aggregation node corresponding to only one model. This way, when client nodes obtain the posterior distribution of model parameters after training, they only need to send the posterior distribution of the model parameters to the aggregation node corresponding to that model. This avoids all client nodes sending the posterior distribution of model parameters to the same aggregation node, preventing network congestion, reducing the processing burden on aggregation nodes, lowering the risk of a single aggregation node failure, and improving information security.

[0228] For example, please refer to Figure 8A , Figure 8A This is a schematic diagram illustrating a process for federated learning involving multiple aggregation nodes and multiple client nodes, provided as an embodiment of this application. Figure 8A As shown, the process of multiple aggregation nodes and multiple client nodes performing federated learning includes the following steps 801-808.

[0229] Step 801: Aggregator node 1 sends the prior distribution of the parameters of model 1 to client nodes 1-4.

[0230] Before federated learning begins, aggregation node 1 can generate or initialize the prior distribution of parameters in model 1 based on the possible distribution of data in each client node. Alternatively, aggregation node 1 can also generate the prior distribution of parameters in model 1 through random initialization, which is not specifically limited in this embodiment.

[0231] For example, aggregation node 1 is, for example, the second node in the above embodiment, and client node 1 is, for example, the first node in the above embodiment.

[0232] Step 802: Aggregator node 2 sends the prior distribution of the parameters of model 2 to client nodes 1-4.

[0233] Similarly, aggregation node 2 can also generate or initialize the prior distribution of parameters in model 1 based on the possible distribution of data in each client node. Furthermore, aggregation node 2 can also generate the prior distribution of parameters in model 1 through random initialization; this embodiment does not impose specific limitations on this approach.

[0234] It should be noted that in this embodiment, Model 1 and Model 2 have the same structure. However, the prior distribution of the parameters in Model 1 generated by aggregation node 1 is not the same as the prior distribution of the parameters in Model 2 generated by aggregation node 2.

[0235] Step 803: Client node 1 and client node 2 send the posterior distribution of the parameters of model 1 to aggregation node 1.

[0236] Specifically, after each client node receives the prior distribution of parameters of multiple models sent by aggregation node 1 and aggregation node 2, each client node can select the model to be trained based on the prior distribution of parameters of each model, and further perform model training based on the training data and the prior distribution of parameters of the selected model.

[0237] In this embodiment, client node 1 and client node 2 select to train model 1, that is, the prior distribution of the parameters of model 1 matches the data distribution in client node 1 and client node 2.

[0238] In this way, after client node 1 and client node 2 obtain the posterior distribution of the parameters of model 1 through training, they send the obtained posterior distribution of the parameters of model 1 to aggregation node 1.

[0239] Step 804: Client node 3 and client node 4 send the posterior distribution of the parameters of model 2 to aggregation node 2.

[0240] In this embodiment, client nodes 3 and 4 select to train model 2, meaning the prior distribution of the parameters of model 2 matches the data distribution in client nodes 3 and 4. After obtaining the posterior distribution of the parameters of model 2 through training, client nodes 3 and 4 send the obtained posterior distribution of the parameters of model 2 to the aggregation node.

[0241] It should be noted that client node 1 and client node 2 select the same model for training (i.e., both are model 1), but the posterior distributions of the model 1 parameters obtained by client node 1 and client node 2 are different. Similarly, the posterior distributions of the model 1 parameters obtained by client node 3 and client node 4 are different.

[0242] Step 805: Aggregation node 1 updates the prior distribution of the parameters of model 1 based on the posterior distribution of the parameters of model 1 sent by client node 1 and client node 2.

[0243] Step 806: Aggregation node 2 updates the prior distribution of the parameters of model 2 based on the posterior distribution of the parameters of model 2 sent by client node 3 and client node 4.

[0244] Step 807: Aggregator Node 1 sends the prior distribution of the parameters of Model 1 to Client Node 1 and Client Node 2.

[0245] After the aggregation node 1 updates the prior distribution of the parameters of model 1, the aggregation node 1 can send the updated prior distribution of the parameters of model 1 to client node 1 and client node 2 so that client node 1 and client node 2 can continue to execute the training of model 1.

[0246] Step 808: Aggregator node 2 sends the prior distribution of the parameters of model 2 to client node 3 and client node 4.

[0247] After the aggregation node 2 updates the prior distribution of the parameters of model 2, the aggregation node 2 can send the updated prior distribution of the parameters of model 2 to the client nodes 3 and 4 so that the client nodes 3 and 4 can continue to train model 2.

[0248] In summary, this solution involves multiple aggregation nodes pre-building models corresponding to different data distribution types based on the potential data distribution on each client node. These aggregation nodes then distribute these models to their respective client nodes, which select the model that best matches their local data distribution based on their training data. Furthermore, after each client node uploads the posterior distribution of its model's parameters to its corresponding aggregation node, the aggregation node aggregates the models accordingly. This data-distribution-based aggregation ensures that the prior distribution of the final model's parameters more accurately reflects the distribution of training data on the client nodes, thus resolving the issue of inconsistent training data distribution across different nodes and effectively improving the performance of the trained models.

[0249] Meanwhile, when client nodes obtain the posterior distribution of model parameters during training, they only need to send the posterior distribution of model parameters to the aggregation node corresponding to that model. This avoids all client nodes sending the posterior distribution of model parameters to the same aggregation node, thus preventing network congestion, reducing the processing burden on aggregation nodes, lowering the risk of failure of a single aggregation node, and improving information security.

[0250] For example, please refer to Figure 8B . Figure 8B This is a schematic diagram of a multi-aggregate node update model provided in an embodiment of this application. Figure 8B As shown, in Figure 8B In (a), aggregation node 1 has the prior distribution of the parameters of model 1, aggregation node 2 has the prior distribution of the parameters of model 2, and aggregation node 3 has the prior distribution of the parameters of model 3. Furthermore, during the first iteration, aggregation nodes 1-3 each send their own prior distribution of the model parameters to each client node. That is, aggregation node 1 sends the prior distribution of the model 1 parameters to each client node, aggregation node 2 sends the prior distribution of the model 2 parameters to each client node, and aggregation node 3 sends the prior distribution of the model 3 parameters to each client node.

[0251] exist Figure 8B In (b), after receiving the prior distributions of the parameters of the three models, each client node selects the model whose distribution best matches the training data based on the training data and the prior distributions of the parameters of the three models, and then sends the posterior distribution of the parameters of the trained model to the aggregation node. Specifically, client node 1 selects to train model 1 and sends the posterior distribution of the parameters of model 1 to aggregation node 1; client node 2 selects to train model 2 and sends the posterior distribution of the parameters of model 2 to aggregation node 2; client node 3 selects to train model 1 and sends the posterior distribution of the parameters of model 1 to aggregation node 1... client node N selects to train model 3 and sends the posterior distribution of the parameters of model 3 to aggregation node 3. In this way, aggregation nodes 1 through 3 can receive the posterior distributions of the parameters of the models uploaded by different client nodes.

[0252] exist Figure 8BIn step (c), each aggregation node updates the prior distribution of the model's parameters based on the posterior distribution of the parameters for the same model uploaded by different client nodes, thus obtaining the updated prior distribution of the model's parameters. Specifically, for model 1, aggregation node 1 updates the prior distribution of model 1's parameters based on the posterior distribution of the parameters for model 1 uploaded by different client nodes, obtaining the updated prior distribution of model 1's parameters. For model 2, aggregation node 2 updates the prior distribution of model 2's parameters based on the posterior distribution of the parameters for model 2 uploaded by different client nodes, obtaining the updated prior distribution of model 2's parameters. For model 3, aggregation node 3 updates the prior distribution of model 3's parameters based on the posterior distribution of the parameters for model 3 uploaded by different client nodes, obtaining the updated prior distribution of model 3's parameters.

[0253] Then, referring to the models selected and trained by each client node, aggregation nodes 1 through 3 send the prior distribution of the updated model parameters to the corresponding client nodes. For example, aggregation node 1 sends the prior distribution of the updated model 1 parameters to client node 1; aggregation node 2 sends the prior distribution of the updated model 2 parameters to client node 2; aggregation node 1 sends the prior distribution of the updated model 1 parameters to client node 3; and aggregation node 3 sends the prior distribution of the updated model 3 parameters to client node N.

[0254] The above describes the execution flow of federated learning provided in the embodiments of this application. For ease of understanding, the process of obtaining the posterior distribution of the parameters of the first model in step 603 will be described in detail below.

[0255] In step 603 above, the process by which the first node trains the posterior distribution of the parameters of the first model based on the prior distribution of the parameters and the training data is essentially the process of local training the model using the training data. During this local training process, the prior distribution of the model's parameters can be used in various ways. For example, the prior distribution of the model's parameters can be used as a constraint in the optimization objective of the local training. Alternatively, the initial values ​​of the posterior distribution of the model's parameters can be determined based on the prior distribution of the model's parameters. The local training processes corresponding to these two usage methods are described in detail below.

[0256] Method 1: The prior distribution of the model's parameters is used as a constraint in the optimization objective of local training.

[0257] First, the optimization objective for local training can be set as follows: the loss function of the posterior distribution of the model's parameters on the training data should be as small as possible (or the likelihood function should be as large as possible), while the difference between the prior and posterior distributions of the model's parameters should be as small as possible or the penalty should be as small as possible.

[0258] Secondly, before local training begins, initial values ​​can be set for the posterior distribution of the model's parameters. There are several ways to set these initial values. For example, the initial value of the posterior distribution of the model's parameters can be set to the value of the posterior distribution of the model's parameters from the previous iteration (e.g., the previous iteration), or it can be a randomized initial value. In some embodiments, the initial value of the posterior distribution of the model's parameters can be determined based on the prior distribution of the model's parameters. Taking a "point description" of the prior distribution of the model's parameters relative to the posterior distribution as an example, the initial value of the posterior distribution of the model's parameters can be the value of the prior distribution of the model's parameters; taking a "distribution description" of the prior distribution of the model's parameters relative to the posterior distribution as an example, the initial value of the posterior distribution of the model's parameters can be a sampled value of the prior distribution of the model's parameters.

[0259] Next, after determining the initial values ​​of the posterior distribution of the model's parameters and the optimization objective, local training can be performed using a score function or reparameterization until the posterior distribution of the model's parameters converges.

[0260] Method 2: Determine the initial values ​​of the posterior distribution of the model's parameters based on the prior distribution of the model's parameters.

[0261] If the prior distribution of the model's parameters uses a "point description" to describe the posterior distribution, then during local training, the prior distribution of the model's parameters can be used as the initial value of the posterior distribution of the model's parameters. If the prior distribution of the local model's parameters uses a "distribution description" to describe the posterior distribution, then during local training, the initial value of the posterior distribution of the model's parameters can be a sample value from the prior distribution of the model's parameters.

[0262] The optimization objective for local training can be set as follows: during training on the training data, the loss function of the posterior distribution of the model's parameters should be as small as possible, or the likelihood function should be as large as possible.

[0263] Next, after determining the initial values ​​of the posterior distribution of the model's parameters and the optimization objective of local training, training can be performed using a score function or reparameterization until the posterior distribution of the model's parameters converges.

[0264] Optionally, in some embodiments, during the process of training the posterior distribution of the parameters of the first model based on the above-described method one or method two, sparsification parameters can be introduced to sparsify the parameters in the first model (i.e., set some parameters in the first model to 0), thereby reducing the number of parameters in the first model.

[0265] For example, in step 603 above, the first node can be trained to obtain the posterior distribution of the target parameters in the first model based on the prior distribution of the parameters of the first model, the training data, and the selection probability of each parameter in the first model. Here, the selection probability of each parameter indicates the probability of selecting each parameter as the target parameter in the first model. The target parameters are a subset of all parameters in the first model. That is, the target parameters are a subset of all parameters in the first model selected based on the selection probability of each parameter, and are the parameters that need to be retained during the training of the first model. Other parameters besides the target parameters are the parameters that need to be removed during the training of the first model.

[0266] In this way, during the training of the first model, only the posterior distribution of the target parameters in the first model can be obtained, without having to calculate the posterior distribution of other parameters, thereby reducing the amount of computation during the training process.

[0267] After obtaining the posterior distribution of the target parameters in the first model through training, the first node sends the posterior distribution of the target parameters in the first model to the second node, thereby reducing the amount of communication between the first node and the second node.

[0268] Experiments have shown that, in some cases, sparsifying the parameters in a model can effectively reduce the number of parameters without affecting the original model performance, and may even slightly improve it. Therefore, in this embodiment, during the training process on the client node, by introducing sparsification parameters to filter the original parameters in the model, some parameters are removed, effectively reducing the number of parameters, lowering the computational load during training, and reducing the communication load between nodes, thus effectively improving the efficiency of federated learning.

[0269] For example, please refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of a parameter-sparsed model provided in an embodiment of this application. For example... Figure 9 As shown, in the parameter sparsification model, some parameters in each neural network layer are removed, leaving only a portion of the parameters, thus greatly reducing the number of parameters in the entire model.

[0270] Optionally, the selection probability of each parameter in the first model is a probability value that can change dynamically during training. That is, the selection probability of each parameter in the first model can change as training progresses, rather than being a fixed value.

[0271] In this way, during the training process, the first node learns the selection probability of each parameter while learning the posterior distribution of the parameters. This allows the selection probability of each parameter to be automatically adjusted according to the training data, thereby better learning the optimal parameter sparsity result and ensuring the performance of the final trained model.

[0272] For example, to achieve sparsity of parameters in a model, a Bernoulli distribution can be introduced during the initialization of model parameters, thereby making the model parameters sparse. Here, the Bernoulli distribution...

[0273] Specifically, for each parameter in the model, the parameter can be represented by the following formula 1.

[0274] Formula 1

[0275] in, Indicates parameters; Represents Bernoulli variables, i.e. The value of is 1 or 0; ~ indicates that it follows a distribution (Gaussian distribution or Bernoulli distribution). The mean is The variance is Gaussian distribution; Indicates the Bernoulli distribution; This represents the Bernoulli probability.

[0276] During the training process of the model, the optimization objective of the model can be specifically represented by the following formula 2.

[0277] Formula 2

[0278] in, The optimization objective is represented by b; the batch size is represented by a; and the Monte Carlo sample size is represented by a. This represents the likelihood function corresponding to the parameters of the model; This represents the training dataset; KL() represents calculating the KL divergence; The function that measures the difference between the prior and posterior distributions of the model parameters.

[0279] The methods provided in the embodiments of this application have been described in detail above. Next, the device for performing the above methods provided in the embodiments of this application will be described.

[0280] Please see Figure 10 , Figure 10 This is a schematic diagram of a federated learning device provided as an embodiment of this application. Figure 10 As shown, this federated learning device belongs to the first node and includes:

[0281] Module 1001 is used to obtain the prior distribution of parameters of multiple models;

[0282] Processing module 1002 is used to determine the performance of each of the multiple models when processing training data, based on the prior distribution of parameters of multiple models and the training data of the device.

[0283] The processing module 1002 is also used to train the posterior distribution of the parameters of the first model based on the prior distribution of the parameters of the first model and the training data, wherein the first model is one of multiple models, and the first model is determined among multiple models based on the performance of each model when processing the training data.

[0284] The sending module 1003 is used to send the posterior distribution of the parameters of the first model to the second node.

[0285] In one possible implementation, the acquisition device is specifically used to receive the prior distribution of parameters of multiple models from the second node;

[0286] The sending module 1003 is also used to send indication information from the first node to the second node. The indication information is used to indicate that the posterior distribution of the parameters sent by the first node corresponds to the first model.

[0287] In one possible implementation, the acquisition device is specifically used to receive the prior distributions of parameters of different models from multiple nodes respectively, so as to obtain the prior distributions of parameters of multiple models, wherein the prior distribution of parameters of the first model is received by the first node from the second node.

[0288] In one possible implementation, the processing module 1002 is specifically used for:

[0289] Based on the prior distribution of parameters of multiple models, the parameter values ​​of each model in the multiple models are sampled.

[0290] Based on the parameter values ​​and training data of each model, determine the performance of each model when processing the training data.

[0291] In one possible implementation, the processing module 1002 is specifically used to train the posterior distribution of the target parameters in the first model based on the prior distribution of the parameters of the first model, the training data, and the selection probability of each parameter in the first model, wherein the selection probability of each parameter is used to indicate the probability of selecting each parameter as the target parameter in the first model, and the target parameter is a subset of all parameters in the first model.

[0292] The sending module 1003 is specifically used to send the posterior distribution of the target parameters in the first model to the second node.

[0293] Please see Figure 11 , Figure 11 This is a schematic diagram of a federated learning device provided as an embodiment of this application. Figure 11 As shown, this federated learning device belongs to the second node and includes:

[0294] The sending module 1101 is used to send the prior distribution of the parameters of the first model to multiple first nodes. The first model is a machine learning model whose parameters follow a distribution.

[0295] The receiving module 1102 is used to receive the posterior distribution of the parameters of the first model sent by a portion of the multiple first nodes;

[0296] The processing module 1103 is used to update the prior distribution of the parameters of the first model based on the posterior distribution of the parameters of the first model, so as to obtain the updated prior distribution of the parameters of the first model.

[0297] The sending module 1101 is also used to send the prior distribution of the parameters after the first model update to some of the first nodes.

[0298] In one possible implementation, the sending module 1101 is further configured to send the prior distribution of the parameters of multiple models to multiple first nodes, wherein the prior distribution of the parameters of multiple models includes the prior distribution of the parameters of the first model.

[0299] The receiving module 1102 is also used to receive indication information sent by a portion of the first nodes, the indication information being used to indicate that the posterior distribution of the parameters sent by the portion of the first nodes corresponds to the first model.

[0300] In one possible implementation, the second node is one of a plurality of aggregation nodes, each of which is used to send the prior distribution of the model's parameters to a plurality of first nodes, and different aggregation nodes send the prior distribution of the parameters of different models.

[0301] In one possible implementation, the receiving module 1102 is further configured to receive the posterior distribution of some parameters in the first model sent by some of the multiple first nodes;

[0302] The processing module 1103 is also used to update the prior distribution of the parameters of the first model based on the posterior distribution of some parameters in the first model.

[0303] Please see Figure 12 , Figure 12This is a schematic diagram of an execution device provided in an embodiment of this application. The execution device 1200 can specifically be a mobile phone, tablet, laptop, smart wearable device, server, etc., and is not limited thereto. Specifically, the execution device 1200 includes: a receiver 1201, a transmitter 1202, a processor 1203, and a memory 1204 (wherein the execution device 1200 may have one or more processors 1203). Figure 12 (Taking a processor as an example), the processor 1203 may include an application processor 12031 and a communication processor 12032. In some embodiments of this application, the receiver 1201, transmitter 1202, processor 1203, and memory 1204 may be connected via a bus or other means.

[0304] Memory 1204 may include read-only memory and random access memory, and provides instructions and data to processor 1203. A portion of memory 1204 may also include non-volatile random access memory (NVRAM). Memory 1204 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0305] Processor 1203 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together through a bus system, which may include not only the data bus, but also power buses, control buses, and status signal buses. However, for clarity, all buses in the diagram are referred to as the bus system.

[0306] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1203. The processor 1203 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1203 or by instructions in software form. The processor 1203 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1203 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1204. Processor 1203 reads the information in memory 1204 and, in conjunction with its hardware, completes the steps of the above method.

[0307] Receiver 1201 can be used to receive input digital or character information, and to generate signal inputs related to the settings and function control of the execution device. Transmitter 1202 can be used to output digital or character information through the first interface; transmitter 1202 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1202 may also include a display device such as a display screen.

[0308] The electronic device provided in this application embodiment can specifically be a chip, which includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip in the execution device to execute the methods described in the above embodiments. Optionally, the storage unit can be a storage unit within the chip, such as a register or cache. The storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, such as random access memory (RAM).

[0309] For details, please refer to Figure 13 , Figure 13 This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU) 1300. The NPU 1300 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 1303, which is controlled by the controller 1304 to extract matrix data from the memory and perform multiplication operations.

[0310] In some implementations, the arithmetic circuit 1303 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1303 is a two-dimensional pulsating array. The arithmetic circuit 1303 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1303 is a general-purpose matrix processor.

[0311] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1302 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1301 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1308.

[0312] Unified memory 1306 is used to store input and output data. Weight data is directly transferred to weight memory 1302 via Direct Memory Access Controller (DMAC) 1305. Input data is also transferred to unified memory 1306 via DMAC.

[0313] BIU stands for Bus Interface Unit, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 1309.

[0314] The Bus Interface Unit (BIU) 1310 is used by the instruction fetch memory 1309 to fetch instructions from external memory, and also by the memory access controller 1305 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0315] The DMAC is mainly used to move input data from external memory DDR to unified memory 1306, or to weight data to weight memory 1302, or to input data to input memory 1301.

[0316] The vector computation unit 1307 includes multiple arithmetic processing units that, when needed, further process the output of the computation circuit 1303, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0317] In some implementations, the vector computation unit 1307 can store the processed output vector in the unified memory 1306. For example, the vector computation unit 1307 can apply a linear function, or a nonlinear function, to the output of the computation circuit 1303, such as performing linear interpolation on the feature planes extracted by the convolutional layer, or, for example, accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 1307 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as an activation input to the computation circuit 1303, for example, for use in subsequent layers of the neural network.

[0318] The instruction fetch buffer 1309 connected to the controller 1304 is used to store the instructions used by the controller 1304;

[0319] Unified memory 1306, input memory 1301, weighted memory 1302, and instruction fetch memory 1309 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0320] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of the above program.

[0321] You can refer to Figure 14 , Figure 14 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. This application also provides a computer-readable storage medium in some embodiments, wherein the above-described... Figure 6 The disclosed method can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0322] Figure 14 A conceptual partial view of an example computer-readable storage medium arranged according to at least some of the embodiments shown herein is illustrated schematically. The example computer-readable storage medium includes a computer program for executing computer processes on a computing device.

[0323] In one embodiment, the computer-readable storage medium 1400 is provided using a signal bearer medium 1401. The signal bearer medium 1401 may include one or more program instructions 1402, which, when executed by one or more processors, can provide the above-mentioned... Figure 6 The described function or part of the function.

[0324] In some examples, signal carrying medium 1401 may include computer-readable medium 1403, such as, but not limited to, hard disk drive, compact disc (CD), digital video disc (DVD), digital magnetic tape, memory, ROM or RAM, etc.

[0325] In some embodiments, the signal-bearing medium 1401 may include a computer-recordable medium 1404, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal-bearing medium 1401 may include a communication medium 1405, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Therefore, for example, the signal-bearing medium 1401 may be transmitted by a wireless form of communication medium 1405 (e.g., a wireless communication medium conforming to the IEEE 802.X standard or other transmission protocols).

[0326] One or more program instructions 1402 may be, for example, computer-executable instructions or logical implementation instructions. In some examples, the computing device may be configured to provide various operations, functions, or actions in response to one or more program instructions 1402 conveyed to the computing device via a computer-readable medium 1403, a computer-recordable medium 1404, and / or a communication medium 1405.

[0327] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0328] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods of the various embodiments of this application.

[0329] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0330] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A federated learning method, characterized in that, include: The first node obtains the prior distribution of parameters of multiple models, including the prior distribution of parameters of the first model. The first node determines the performance of each of the multiple models when processing the training data based on the prior distribution of the parameters of the multiple models and the training data of the first node. The first node trains to obtain the posterior distribution of the target parameters in the first model based on the prior distribution of the parameters of the first model, the training data, and the selection probability of each parameter in the first model. The selection probability of each parameter is used to indicate the probability of selecting each parameter as the target parameter in the first model. The target parameter is a subset of all parameters of the first model. The first model is determined among the multiple models based on the performance of each model in processing the training data. The first node sends the posterior distribution of the target parameters in the first model to the second node.

2. The method according to claim 1, characterized in that, The first node obtains the prior distribution of parameters from multiple models, including: The first node receives the prior distribution of the parameters of the plurality of models from the second node; The method further includes: The first node sends indication information to the second node, the indication information being used to indicate that the posterior distribution of the parameters sent by the first node corresponds to the first model.

3. The method according to claim 1, characterized in that, The first node obtains the prior distribution of parameters from multiple models, including: The first node receives the prior distributions of parameters of different models from multiple nodes to obtain the prior distributions of parameters of the multiple models, wherein the prior distribution of parameters of the first model is obtained by the first node from the second node.

4. The method according to any one of claims 1-3, characterized in that, The first node determines the performance of each of the multiple models in processing the training data based on the prior distribution of the parameters of the multiple models and the training data of the first node, including: The first node samples the parameter values ​​of each of the multiple models based on the prior distribution of the parameters of the multiple models; The first node determines the performance of each model when processing the training data based on the parameter values ​​of each model and the training data.

5. The method according to any one of claims 1-3, characterized in that, The performance of each model when processing the training data includes one or more of the following: model accuracy, model confidence, model convergence speed, and gradient forward direction during training.

6. The method according to claim 1, characterized in that, In the first model, the probability of selecting each parameter is a probability value that can change dynamically during training.

7. The method according to any one of claims 1-3, characterized in that, The prior distribution of the parameters of the first model is either the probability distribution of the parameters of the first model or the probability distribution of the probability distribution of the parameters of the first model.

8. A federated learning method, characterized in that, include: The second node sends the prior distribution of parameters of multiple models to multiple first nodes, wherein the prior distribution of parameters of multiple models includes the prior distribution of parameters of the first model. The second node receives the posterior distribution of some parameters in the first model sent by some of the multiple first nodes, and receives the posterior distribution of parameters in the second model sent by another part of the multiple first nodes, wherein the second model is one of the multiple models; The second node updates the prior distribution of the parameters of the first model based on the posterior distribution of some parameters in the first model, and obtains the updated prior distribution of the parameters of the first model. The second node sends the prior distribution of the updated parameters of the first model to the first node in the aforementioned part; The second node updates the prior distribution of the parameters of the second model based on the posterior distribution of the parameters of the second model.

9. The method according to claim 8, characterized in that, The method further includes: The second node receives indication information sent by the first node, the indication information being used to indicate that the posterior distribution of the parameters sent by the first node corresponds to the first model.

10. The method according to claim 8, characterized in that, The second node is one of a plurality of aggregation nodes, each of which is used to send the prior distribution of the model's parameters to the plurality of first nodes, and different aggregation nodes send the prior distribution of the parameters of different models.

11. The method according to any one of claims 8-10, characterized in that, The prior distribution of the parameters of the first model is either the probability distribution of the parameters of the first model or the probability distribution of the probability distribution of the parameters of the first model.

12. A federated learning device, characterized in that, The device belongs to the first node and includes: An acquisition module is used to acquire the prior distribution of parameters of multiple models, wherein the prior distribution of parameters of multiple models includes the prior distribution of parameters of a first model; A processing module is configured to determine the performance of each of the plurality of models when processing the training data, based on the prior distribution of the parameters of the plurality of models and the training data of the device. The processing module is further configured to train a posterior distribution of the target parameters in the first model based on the prior distribution of the parameters of the first model, the training data, and the selection probability of each parameter in the first model, wherein the selection probability of each parameter is used to indicate the probability of selecting each parameter as the target parameter in the first model, the target parameter being a subset of all parameters of the first model, wherein the first model is determined among the plurality of models based on the performance of each model in processing the training data; The sending module is used to send the posterior distribution of the target parameters in the first model to the second node.

13. The apparatus according to claim 12, characterized in that, The acquisition module is specifically used to receive the prior distribution of the parameters of the multiple models from the second node; The sending module is further configured to send indication information from the first node to the second node, the indication information being used to indicate that the posterior distribution of the parameters sent by the first node corresponds to the first model.

14. The apparatus according to claim 12, characterized in that, The acquisition module is specifically used to receive the prior distribution of parameters of different models from multiple nodes to obtain the prior distribution of parameters of the multiple models, wherein the prior distribution of parameters of the first model is obtained by the first node from the second node.

15. The apparatus according to any one of claims 12-14, characterized in that, The processing module is specifically used for: Based on the prior distribution of the parameters of the multiple models, the parameter values ​​of each model in the multiple models are sampled. Based on the parameter values ​​of each model and the training data, the performance of each model in processing the training data is determined.

16. A federated learning device, characterized in that, The device belongs to the second node and includes: The sending module is used to send the prior distribution of parameters of multiple models to multiple first nodes, wherein the prior distribution of parameters of multiple models includes the prior distribution of parameters of the first model. The receiving module is configured to receive the posterior distribution of some parameters in the first model sent by a portion of the plurality of first nodes, and to receive the posterior distribution of parameters of the second model sent by another portion of the plurality of first nodes, wherein the second model is one of the plurality of models; The processing module is used to update the prior distribution of the parameters of the first model based on the posterior distribution of some parameters in the first model, so as to obtain the updated prior distribution of the parameters of the first model. The sending module is also used to send the prior distribution of the parameters updated by the first model to the first part of the first nodes; The processing module is further configured to update the prior distribution of the parameters of the second model based on the posterior distribution of the parameters of the second model.

17. The apparatus according to claim 16, characterized in that, The receiving module is further configured to receive indication information sent by the partial first node, the indication information being used to indicate that the posterior distribution of the parameters sent by the partial first node corresponds to the first model.

18. The apparatus according to claim 16, characterized in that, The second node is one of a plurality of aggregation nodes, each of which is used to send the prior distribution of the model's parameters to the plurality of first nodes, and different aggregation nodes send the prior distribution of the parameters of different models.

19. A federated learning device, characterized in that, The device includes a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the device performs the method as described in any one of claims 1 to 11.

20. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed by the computer, cause the computer to perform the method according to any one of claims 1 to 11.

21. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 11.

Images