Methods, devices, systems, media, equipment, and products for implementing federated learning

Abstract

The embodiments of this specification provide a method, device, system, computer-readable storage medium, electronic device, and computer program product for implementing federated learning, which is applied to central servers and clients participating in federated learning tasks. Specifically, the feature vector reflecting the data distribution characteristics of the client is obtained based on the local model of each client, and then the similarity is calculated by calculating the feature vector corresponding to each client to measure the similarity between the data distribution characteristics of different clients. The clients are then grouped according to the calculated similarity to assign clients with high similarity in data distribution characteristics to the same client set. Furthermore, for the same client set, the model data of the local models of each client are aggregated to obtain a personalized model corresponding to the client set.

Description

Technical Field

[0001] This specification relates to the field of machine learning technology, and in particular to a method, device, system, computer-readable storage medium, electronic device, and computer program product for implementing federated learning. Background Art

[0002] The deployment and implementation of AI applications often creates data silos, such as limited training data, poor data quality, and the need to protect data privacy. Federated learning can address these silo issues by exchanging data between different customers in a distributed environment.

[0003] Specifically, a typical method to implement federated learning is Federated Averaging (FedAvg), which generates a global model by averaging the local parameters uploaded by each client.

[0004] However, the federated learning implementation solutions provided by related technologies cannot adapt well to the data distribution of each client, resulting in a low degree of personalization of the trained federated learning model.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this specification, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0006] The purpose of this specification is to provide a method, apparatus, system, computer-readable storage medium, electronic device, and computer program product for implementing federated learning, which can at least improve the personalization of federated learning models to a certain extent without introducing bias or overfitting problems.

[0007] Other features and advantages of the present specification will become apparent from the following detailed description, or may be learned in part from the practice of the present specification.

[0008] According to one aspect of the present specification, a method for implementing federated learning is provided, which is applied to a central server participating in a federated learning task, and the method includes: receiving feature vectors corresponding to N clients of the federated learning task, wherein the i-th feature vector corresponding to the i-th client is obtained from the local model of the i-th client, and the i-th feature vector reflects the data distribution characteristics of the i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; calculating the similarity between the N clients based on the feature vectors corresponding to the N clients; dividing the N clients into M client sets based on the similarity between the N clients, where M is a positive integer less than N; and, for the j-th client set, aggregating the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M.

[0009] According to another aspect of the present specification, a method for implementing federated learning is provided, which is applied to the i-th client participating in the federated learning task, and the method includes: obtaining the i-th feature vector from the local model, the i-th feature vector reflecting the data distribution characteristics of the i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; sending the i-th feature vector to the central server of the federated learning task, so that the central server: calculates the similarity between the N clients based on the feature vectors corresponding to the N clients respectively; and divides the N clients into M client sets based on the similarity between the N clients, where M is a positive integer less than N; and, when the i-th client belongs to the j-th client set, sends the model data of the local model to the central server, so that the central server aggregates the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M.

[0010] According to another aspect of the present specification, a device for implementing federated learning is provided, which is configured on a central server participating in a federated learning task. The device includes: a feature receiving module, a similarity calculation module, a client grouping module, and an aggregation processing module.

[0011] Among them, the above-mentioned feature receiving module is used to receive the feature vectors corresponding to the N clients of the above-mentioned federated learning task, wherein the i-th feature vector corresponding to the i-th client is obtained from the local model of the above-mentioned i-th client, and the above-mentioned i-th feature vector reflects the data distribution characteristics of the above-mentioned i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; the above-mentioned similarity calculation module is used to calculate the similarity between the above-mentioned N clients based on the feature vectors corresponding to the above-mentioned N clients; the above-mentioned client grouping module is used to divide the above-mentioned N clients into M client sets according to the similarity between the above-mentioned N clients, where M is a positive integer less than N; and the above-mentioned aggregation processing module is used to aggregate the model data of the local models corresponding to all clients in the above-mentioned j-th client set for the j-th client set to obtain the personalized model corresponding to the above-mentioned j-th client set, where j is a positive integer not greater than M.

[0012] According to another aspect of the present specification, a device for implementing federated learning is provided, which is configured on the i-th client participating in the federated learning task. The device includes: a feature determination module, a feature sending module, and a model data sending module.

[0013] Among them, the above-mentioned feature determination module is used to obtain the i-th feature vector from the local model, and the above-mentioned i-th feature vector reflects the data distribution characteristics of the above-mentioned i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; the above-mentioned feature sending module is used to send the above-mentioned i-th feature vector to the central server of the above-mentioned federated learning task, so that the above-mentioned central server: calculates the similarity between the above-mentioned N clients according to the feature vectors corresponding to the above-mentioned N clients respectively; and divides the above-mentioned N clients into M client sets according to the similarity between the above-mentioned N clients, where M is a positive integer less than N; and the above-mentioned model data sending module is used to send the model data of the local model to the above-mentioned central server when the above-mentioned i-th client belongs to the j-th client set, so that the above-mentioned central server aggregates the model data of the local models corresponding to all clients in the above-mentioned j-th client set to obtain the personalized model corresponding to the above-mentioned j-th client set, and j is a positive integer not greater than M.

[0014] According to another aspect of the present specification, a system for implementing federated learning is provided, comprising N clients and a central server participating in a federated learning task; wherein:

[0015] The above-mentioned central server executes the implementation method of federated learning as described in the above-mentioned aspect; and the i-th client executes the implementation method of federated learning as described in the above-mentioned aspect, where N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N.

[0016] According to one aspect of this specification, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for implementing federated learning as described in the above embodiment is implemented.

[0017] According to one aspect of this specification, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed on a computer or a processor, the computer or processor executes the implementation method of federated learning as described in the above embodiment.

[0018] According to another aspect of this specification, a computer program product including instructions is provided. When the computer program product is run on a computer or a processor, the computer or processor executes the implementation method of federated learning as described in the above embodiment.

[0019] The implementation methods, devices, systems, computer-readable storage media, electronic devices, and computer program products of federated learning provided in the embodiments of this specification have the following technical effects:

[0020] The solution provided by the exemplary embodiments of this specification is applied to the central server and client participating in the federated learning task. First, the feature vector reflecting the client data distribution characteristics is obtained based on the local model of each client, and then the similarity is calculated by calculating the feature vector corresponding to each client to measure the similarity between the data distribution characteristics of different clients. Finally, the clients are grouped according to the calculated similarity, so that the clients with high similarity in data distribution characteristics are classified into the same set. Finally, for the same client set, the model data of the local model of each client is aggregated to obtain the personalized model corresponding to the set. Therefore, the solution provided by the embodiment of this specification can better adapt to the data distribution of the clients in each client set / each group of clients, which is beneficial to the personalization level of the federated learning model after training. The federated learning model with a higher degree of personalization is beneficial to improving the predictive analysis ability of the model.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are incorporated into and constitute a part of this specification, illustrate embodiments consistent with this specification, and together with the specification, are used to explain the principles of this specification. Obviously, the drawings described below are only some embodiments of this specification, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0023] Figure 1 A schematic diagram of the architecture of the federated learning implementation system provided in the embodiments of this specification.

[0024] Figure 2 A flowchart of a method for implementing federated learning provided in one embodiment of this specification.

[0025] Figure 3 A flowchart of a method for implementing federated learning provided in another embodiment of this specification.

[0026] Figure 4 This is a schematic diagram of information interaction for a method for implementing federated learning provided in one embodiment of this specification.

[0027] Figure 5 A flow chart of a method for determining a feature vector provided in an embodiment of this specification.

[0028] Figure 6 This is a flowchart of a method for determining similarity between clients provided in one embodiment of this specification.

[0029] Figure 7 This is a flowchart of a method for determining similarity between clients provided in another embodiment of this specification.

[0030] Figure 8 This is a schematic diagram of information interaction for a method for implementing personalized federated learning provided in one embodiment of this specification.

[0031] Figure 9 This is a schematic diagram of information interaction for a method for implementing federated learning provided in another embodiment of this specification.

[0032] Figure 10 A schematic diagram of the structure of a device for implementing federated learning provided in one embodiment of this specification.

[0033] Figure 11 A schematic diagram of the structure of a federated learning implementation device provided in another embodiment of this specification.

[0034] Figure 12 A schematic diagram of the structure of a device for implementing federated learning provided in yet another embodiment of this specification.

[0035] Figure 13A schematic diagram of the structure of a device for implementing federated learning provided in yet another embodiment of this specification.

[0036] Figure 14 This is a schematic diagram of the structure of the federated learning implementation system provided in the embodiments of this specification.

[0037] Figure 15 A schematic diagram of the structure of a federated learning implementation system provided in another embodiment of this specification.

[0038] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of this specification more clear, the embodiments of this specification will be further described in detail below with reference to the accompanying drawings.

[0040] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Instead, they are merely examples of devices and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0041] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this specification will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of this specification. However, those skilled in the art will appreciate that the technical solutions of this specification may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this specification.

[0042] In addition, the accompanying drawings are merely schematic illustrations of this specification and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] Since the data distribution of each client has the characteristics of non-independent and identically distributed (Non-IID), making good use of the non-independent and identically distributed data distribution characteristics of the client can help improve the predictive analysis performance of the federated learning model. For example, the machine learning model for predicting the sales of maternal and infant products is determined by the data of clients who prefer to buy maternal and infant products, the machine learning model for predicting the sales of men's clothing is determined by the data of clients who prefer to buy men's clothing, and the machine learning model for predicting the sales of women's clothing is determined by the data of clients who prefer to buy women's clothing. Therefore, it is necessary to improve the personalization level of the obtained federated learning model. In other words, when implementing federated learning based on the federal averaging method, each client corresponds to the same global shared model, which cannot adapt to the data distribution of different clients and is not conducive to improving the training effect of the model.

[0044] To improve the personalization of federated learning models, one related technique first trains a global shared model. Then, based on the data distribution characteristics of each client, the global model is fine-tuned for each client to create a personalized model for that client. However, the personalization process provided by this related technique is limited to a single client, and the data on a single device is very limited, which can introduce bias or overfitting.

[0045] The solution concept provided by the embodiments of this specification is that although each client participating in federated learning corresponds to different data distribution characteristics, by calculating the similarity between the data distribution characteristics of different clients, dividing the clients with higher similarity into a group / a client set, and then performing a personalized federated learning model for any client set, and then determining the personalized model corresponding to each client set, it is possible to solve the above-mentioned bias or overfitting problems. Specifically, the embodiments of this specification provide the following content:

[0046] For example, Figure 1 A schematic diagram of the system architecture for implementing federated learning in the embodiments of this specification.

[0047] like Figure 1 As shown, the system architecture 100 may include a terminal (client) 110 , a network 120 , and a server 130 . The terminal 110 and the server 130 are connected via the network 120 .

[0048] Exemplarily, the terminal 110, as the data holder of federated learning, may be a mobile phone, computer, tablet, etc. with computing capabilities. In the solution provided in the embodiments of this specification, the local model is updated at each terminal, that is, the received global model is iteratively calculated (also called local calculation, local calculation), and the model data (gradient, model parameters) after iterative calculation is sent to the central server (such as server 130); and the feature vector reflecting the feature distribution of the relevant client data is determined according to the local model and sent to the central server.

[0049] Exemplarily, the network 120 may be a communication medium of various connection types capable of providing a communication link between the terminal 110 and the server 130, such as a wired communication link, a wireless communication link, or an optical fiber cable, etc., which is not limited in this specification.

[0050] Exemplarily, server 130 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, as well as big data and artificial intelligence platforms.

[0051] Exemplarily, after receiving model data from multiple clients, the central server 130 can aggregate the model data; further, the global model is updated based on the aggregation results, and the updated model is returned to the data holder / client participating in the federated learning task. In the embodiment provided in the embodiment of this specification, the server 130 can also perform similarity calculations based on the feature vectors sent by each client, thereby measuring the similarity between the data distribution characteristics of different clients. Finally, the clients are grouped according to the calculated similarity, so that clients with high similarity in data distribution characteristics are assigned to the same set. Finally, for the same client set, the model data of the local models of each client are aggregated to obtain a personalized model corresponding to the set. It can be seen that the solution provided by the embodiment of this specification can better adapt to the data distribution of the clients in each client set, which is beneficial to the degree of personalization of the federated learning model after training.

[0052] The following first passes Figures 2 to 9 The implementation method of federated learning provided in this specification is described in detail:

[0053] For example, Figure 2 and Figure 3 The following are flowcharts of the implementation methods of federated learning provided in the embodiments of this specification. Figure 2 The execution subject of the method shown is the central server participating in the federated learning task. Figure 3 The execution subject of the method is any client (i-th client) participating in the federated learning task.

[0054] refer to Figure 2 , the method shown in this embodiment includes:

[0055] S210, receiving the feature vectors corresponding to the N clients of the federated learning task, wherein the i-th feature vector corresponding to the i-th client is obtained from the local model of the i-th client, and the i-th feature vector reflects the data distribution characteristics of the i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; S220, calculating the similarity between the N clients based on the feature vectors corresponding to the N clients; S230, dividing the N clients into M client sets based on the similarity between the N clients, where M is a positive integer less than N; and, S240, for the j-th client set, aggregating the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M.

[0056] refer to Figure 3 , the method shown in this embodiment includes:

[0057] S310, obtain the i-th eigenvector from the local model, the i-th eigenvector reflects the data distribution characteristics of the i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; S320, send the i-th eigenvector to the central server of the federated learning task, so that the central server: calculates the similarity between the N clients based on the eigenvectors corresponding to the N clients respectively; and divides the N clients into M client sets based on the similarity between the N clients, where M is a positive integer less than N; and, S330, when the i-th client belongs to the j-th client set, send the model data of the local model to the central server, so that the central server aggregates the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M.

[0058] The following combination Figure 4 The embodiment shown, for the above Figure 2 and Figure 3 The specific implementation of each step of the embodiment shown is described in detail:

[0059] In an exemplary embodiment, Figure 4 This is a schematic diagram of information interaction for implementing a method of federated learning provided in one embodiment of this specification. Figure 4The embodiment shown in this figure shows the information interaction process between multiple clients (for example, the mth client and the ith client) and the central server 420.

[0060] Exemplary, reference Figure 4 In S40, after the federated learning task performs a preset number of iterations (eg, 10% of the total number of iterations of the task), a global model M is obtained.

[0061] First, in the initial stage of the federated learning task, the central server 420 and each client reach a federated learning agreement, and then the federated learning task is started, and the central server 420 publishes initial parameters to each data holder / client.

[0062] Furthermore, each client performs an iteration, performing local calculations based on its own data. Specifically, after the federated learning modeling task is started and the system parameters are initialized, each client first performs local calculations based on the stored data. After the local calculations are completed, the model data (such as gradients or model parameters) obtained from the local calculations are desensitized and uploaded to the central server for updating the global model.

[0063] Specifically, the central server 420 performs central aggregation processing. After the central server 420 receives the model data (after desensitization) from each client, the central server 420 performs an aggregation operation on these calculated values. It should be noted that in the process of the above-mentioned aggregation processing, many aspects such as efficiency, security, and privacy need to be considered. For example, in order to safely aggregate the parameters, the model data is encrypted using relevant encryption technology. Furthermore, the central server 420 updates the global model based on the data after the central aggregation processing to obtain a global model, such as the global model M.

[0064] Exemplarily, the above-mentioned preset number of iterations is 10% of the total number of iterations of the federated learning task. If there are 100 rounds of iterations in this federated learning task (100 local calculations are performed on each client), the global model M obtained by the 10th update can be performed on the central server. The above-mentioned global model M is sent to each client. For each client, a local iteration is performed on the basis of the same global model M through its own stored data. Then, based on the local model obtained by the iteration, the characteristic vector reflecting the data distribution characteristics of each client can be determined. At the same time, compared with the method in the related art that requires fine-tuning for each client after the iteration of the federated learning task is completed, this solution can save more time and computing resources.

[0065] Continue to refer Figure 4In S42, the central server 420 sends the global model M to multiple clients (eg, the mth client / the ith client). It is understandable that it is not limited to Figure 4 As shown in , in addition to sending the global model M to the mth client / ith client, the central server can also send the global model M to more clients, and when executing S42, the mth client and the i-th client are not yet divided into the j-th client set. The specific client set division process will be introduced in S48-S410. In S44, after the federated learning task performs a preset number of local iterations on the mth client / i-th client, a vector reflecting high-level semantics is obtained from the obtained local model to obtain the mth feature vector / i-th feature vector. And, in S46, the mth feature vector / i-th feature vector is sent to the central server.

[0066] Exemplarily, at each client, the above-mentioned global model is trained (local calculations are performed) based on the data stored by the client, and then the local models corresponding to each client are obtained. Among them, each client performs a model training process based on the above-mentioned global model to convert the data stored on the client into high-level semantic features. For example, in this federated learning task, a neural network is used, and the feature information obtained after the neural network performs several convolutions (feature extractions) on the data stored by the client can be called a high-level semantic feature. The receptive field of high-level semantic features is large, which is conducive to reflecting the data distribution characteristics of the client. Therefore, in the solution provided in the embodiment of this specification, the above-mentioned i-th feature vector is determined based on the high-level semantic features determined by the local model of the i-th client.

[0067] In an exemplary embodiment, Figure 5 This is a flow chart of the method for determining a feature vector provided in the embodiment of this specification. Figure 5 , the method includes: S510-S530.

[0068] In S510 , for the i-th client, a target layer reflecting high-level semantics is determined in the local model.

[0069] As mentioned above, since high-level semantic features are more conducive to reflecting the data distribution characteristics of the client, this embodiment will use high-level semantic features to determine the above feature vectors. In this embodiment, the model layer used to determine high-level semantic features is recorded as the target layer reflecting high-level semantics.

[0070] For example, when the federated learning task is a classification model, the target layer may be one or two layers before the activation function layer. When the federated learning task is a response model, the target layer may be the last layer or two layers of the model.

[0071] In S520, the model weights in the target layer are compared with a third preset threshold; and, in S530, those below the third preset threshold are set to 0 to obtain a sparse vector, and the sparse vector is determined as the i-th eigenvector.

[0072] In this embodiment, a sparse vector is obtained by comparing the model weights in the target layer with a third preset threshold and setting those below the third preset threshold to 0. This sparse vector is then used to determine the client's feature vector. Threshold filtering is used to minimize the feature vector, which not only helps reduce the amount of communication between each client and the central server, but also avoids data privacy issues caused by exchanging raw data.

[0073] Continue to refer Figure 4 , central server 420: in S48, calculates the similarity between the N clients according to the feature vectors corresponding to the N clients respectively; and, in S410, divides the N clients into M client sets according to the similarity between the N clients.

[0074] For example, Figure 6 This is a flow chart of a method for determining similarity between clients provided in one embodiment of this specification. Figure 6 , the embodiment shown in the figure includes:

[0075] In S610, for client a and client b among N clients, similarity is calculated based on the feature vectors corresponding to client a and client b respectively to obtain an N×N similarity matrix, where the values ​​of a and b are any integer between 1 and N, inclusive.

[0076] For example, based on the feature vector (essentially a sparse vector) sent by each client, a method similar to the Euclidean distance is used to calculate the similarity between two clients, and an N×N similarity matrix is ​​obtained as follows:

[0077]

[0078] Among them, it is understandable that S ab With S ba Both represent the similarity calculated based on the a-th feature vector corresponding to the a-th client and the b-th feature vector corresponding to the b-th client, then S ab With S ba It should be noted that, in this embodiment, the similarity between two clients is represented by the similarity matrix. In other embodiments, the similarity between two clients may be represented in other forms.

[0079] Furthermore, in S620 , the N clients are divided into M client sets according to an N×N similarity matrix and a first preset threshold.

[0080] Specifically, the value of each element in the above-mentioned similarity matrix can be compared one by one with the first preset threshold value, or the value of each element in the upper triangular area of ​​the above-mentioned similarity matrix can be compared one by one with the first preset threshold value, or the value of each element in the lower triangular area of ​​the above-mentioned similarity matrix can be compared one by one with the first preset threshold value. It is only necessary to ensure that the similarity between every two clients is compared with the first preset threshold value.

[0081] For example, the first preset threshold may be set to 0.5. Furthermore, the values ​​of the elements in the similarity matrix are compared with 0.5 to ensure that the similarity between any two clients in the same client set is greater than 0.5. This allows all clients to be divided into different groups, and clients in the same group are considered to have similar data distributions.

[0082] In another exemplary embodiment, in order to improve the similarity calculation efficiency, Figure 7 This is a flow chart of a method for determining similarity between clients provided in another embodiment of this specification. Figure 7 , the embodiment shown in the figure includes:

[0083] At step S710, the similarities between the x-th feature vector and the feature vectors corresponding to other clients are calculated respectively to obtain N-1 similarities, where x is any integer between 1 and N, inclusive. At step S720, the N-1 similarities are compared with M preset intervals, and the N-1 similarities are distributed into the M preset intervals to divide the N clients into M client sets.

[0084] For example, a client is randomly sampled from the sparse vectors sent by N clients, and its corresponding feature vector is recorded as the xth feature vector. The similarity between the feature vectors of other clients (represented as: Rep1, Rep2, ..., Repi, ..., RepN) and the xth feature vector (represented as: Repx) is calculated in sequence. Specifically, N-1 similarity values ​​can be obtained: sim(Rep1, Repz), sim(Rep2, Repz), ..., sim(RepN, Repz).

[0085] At the same time, M preset intervals may also be set, such as (0, 0.2], (0.2, 0.4], (0.4, 0.6], (0.6, 0.8], and (0.8, 1).

[0086] Then, the N-1 similarity values ​​are compared with the M preset intervals. If the N-1 similarity values ​​fall within the M preset intervals, the clients can be grouped. For example, if sim(Rep1, Repx) and sim(Rep2, Repx) both fall within the preset interval [0.2, 0.4], then the data distributions of the first and second clients can be considered to be highly similar, and thus can be grouped into the same client set.

[0087] pass Figure 6 or Figure 7 The corresponding embodiment can divide all clients participating in the same federated learning task into different groups, and consider that the clients in the same group are judged to have similar data distribution. For example, all clients participating in the federated learning task are grouped into three client sets. Specifically, the data distribution characteristics corresponding to the first client set are the purchase of clothing products, the data distribution characteristics corresponding to the second client set are the purchase of furniture products, and the data distribution characteristics corresponding to the third client set are fund investments. Furthermore, in the following embodiments, a personalized model can be determined for the same client set (each client has similar data distribution characteristics). Based on the personalized model, the corresponding data distribution characteristics (such as the purchase of furniture products) can be targeted and predicted, which is conducive to improving the prediction accuracy.

[0088] Continue to refer Figure 4 After the clients participating in the federated learning task are grouped, for any client set, the central server aggregates the model data of the local models corresponding to all clients in the same set to obtain the personalized model corresponding to the client set. In S410, for example, Figure 4 The m-th client and the i-th client shown in FIG belong to the j-th client set. Take the j-th client set as an example to illustrate:

[0089] The central server 420 receives the model data of the local model sent by each client in the j-th client set. Figure 4 In S412, the model data of the local model corresponding to the mth client / ith client is received. Then, in S414, the central server 420 aggregates the model data of the local models corresponding to all clients in the jth client set to obtain a personalized model corresponding to the jth client set.

[0090] In an exemplary embodiment, Figure 8 This is a schematic diagram of information interaction of a method for implementing personalized federated learning provided in an embodiment of this specification, specifically providing an embodiment for determining a personalized model corresponding to the j-th client set. Figure 8 , the embodiment shown in the figure includes:

[0091] S810, receiving model data of the local model corresponding to each client in the j-th client set, where the model data is a model weight or a model gradient; and, S820, performing weighted averaging processing on the model data corresponding to the L clients in the j-th client set, and updating the global model with the weights after the weighted averaging processing to obtain a personalized model corresponding to the j-th client set.

[0092] The j-th client set contains L clients, where L is a positive integer no greater than (N-M+1). For example, if N is 100, then the number of clients participating in the federated learning task is 100, and M is 5, then the 100 clients participating in the federated learning task can be divided into 5 client sets, and the number of clients contained in any one of these sets is no greater than 96(N-M+1).

[0093] In the solution provided by this embodiment, a weighted average is performed on the data corresponding to a group of clients with similar data distributions, and the global model is updated using the weights after the weighted average process to obtain a personalized model corresponding to the client set. Compared to the method of fine-tuning for a single client in the related art, each client in the client set has similar data distribution characteristics, so that more sufficient data resources reflecting the data distribution characteristics can be obtained. It can be seen that the embodiments of this specification not only obtain a personalized model suitable for the data distribution characteristics, thereby performing more targeted analysis and prediction of the relevant data distribution characteristics, but also do not introduce bias or over-fitting problems.

[0094] In an exemplary embodiment, the personalized federated learning mode provided in the embodiment of this specification, as described above, can be started after the federated learning task has iterated a preset number of times. Furthermore, in order to adapt to changes in client data, grouping with higher accuracy and further determine the personalized model corresponding to each group, the following is performed in any subsequent iteration process. Figure 4 The steps S44-S414 show that, in each iterative calculation process after the personalized federated learning mode is turned on, the following operations can be performed: obtaining feature vectors (S44), calculating similarity (S48), grouping clients (S410), and determining the personalized model corresponding to any client set (S414).

[0095] Of course, in other embodiments, in order to accelerate the training efficiency of the federated learning model, or if it is known that the data changes of each client are small, the grouping period can be set in advance, that is, the federated learning task can be set to be performed once every 10 iterations. Figure 4 The process shown in S40-S414 can also be set in the entire process of the federated learning task to execute the following Figure 4 For example, this federated learning task has 100 rounds of iterations, and it can be set to execute 5 times in the entire iteration process. Figure 4 Thus, the user can control the process as shown in S40-S414 based on actual needs. Figure 4 The number of personalized federated learning processes shown in S40-S414 ensures that the federated learning model can be personalized while also facilitating the flexibility of implementing the federated learning task.

[0096] In an exemplary embodiment, after the federated learning task is started after a preset number of iterations, the personalized federated learning mode can be turned off based on the client's request, so that each client can turn off the personalized federated learning mode at any time. Figure 9 This is a schematic diagram of information interaction for a method for implementing federated learning provided in another embodiment of this specification.

[0097] refer to Figure 9 At stage 910, the i-th client belongs to the j-th client set 410. In this case, at S92, the i-th client sends a request to the central server 420 to close personalized federated learning. Then, at S94, the central server 420 updates the j-th client set to which the i-th client belongs, obtaining the j'th client set.

[0098] Referring to stage 920, the j'th client set 410' no longer contains the i'th client, and the personalized federated learning model continues between the central server 420 and the j'th client set 410'. Specifically, in S96, the model parameters of the local model corresponding to the m'th client (the model parameters of the local model corresponding to each client in the j'th client set, as updated by the central server 420) are received; and in S98, the received model data for the updated local model corresponding to each client in the j'th client set is aggregated. The global model is then updated to obtain a personalized model corresponding to the j'th client set.

[0099] refer to Figure 9 At the same time, the i-th client and the central server perform a common federated learning mode. For example, in S910: the i-th client and the central server perform data interaction to implement the iterative process of the non-personalized federated learning mode.

[0100] In the federated learning implementation scheme provided in the embodiments of this specification, clients with similar data distribution characteristics are grouped together into the same group / set, and adaptive personalization is then performed on this group / set of clients. This is rather than focusing on personalization for a single local client in the federated learning task. Thus, the solution provided in the embodiments of this specification avoids the drawback of existing personalization processes that are limited to a single device. By uniting clients with similar data distributions and then performing a personalized federated learning model, the model's level of personalization is effectively improved while also avoiding bias or overfitting.

[0101] Among them, for clients participating in federated learning, they can turn on / off personalized federated learning mode at any time during the federated learning process according to their own needs, thereby increasing the flexibility of implementing federated learning.

[0102] It should be noted that the above figures are merely illustrative of the processes included in the methods according to exemplary embodiments of this specification and are not intended to be limiting. It should be understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it should be understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0103] The following are device embodiments of this specification, which can be used to implement the method embodiments of this specification. For details not disclosed in the device embodiments of this specification, please refer to the method embodiments of this specification.

[0104] in, Figure 10 This is a schematic diagram of the structure of the implementation device of federated learning provided in one embodiment of this specification. Figure 10 The implementation device of federated learning shown in the figure can be configured on the above-mentioned central server participating in the federated learning task through software, hardware or a combination of both. It can also be integrated on the server as an independent module, and can also be integrated into an electronic device as an independent module.

[0105] The above-mentioned federated learning implementation device 1000 in the embodiment of this specification includes: a feature receiving module 1010, a similarity calculation module 1020, a client grouping module 1030, and an aggregation processing module 1040.

[0106] The feature receiving module 1010 is configured to receive feature vectors corresponding to the N clients of the federated learning task, wherein the i-th feature vector corresponding to the i-th client is obtained from the local model of the i-th client, and the i-th feature vector reflects the data distribution characteristics of the i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; the similarity calculation module 1020 is configured to calculate the similarity between the N clients based on the feature vectors corresponding to the N clients; the client grouping module 1030 is configured to divide the N clients into M client sets based on the similarity between the N clients, where M is a positive integer less than N; and the aggregation processing module 1040 is configured to aggregate the model data of the local models corresponding to all clients in the j-th client set for the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M.

[0107] In an exemplary embodiment, Figure 11 This is a schematic diagram of the structure of a device for implementing federated learning provided in another embodiment of this specification. Figure 11 :

[0108] In an exemplary embodiment, based on the aforementioned solution, the i-th feature vector is a sparse vector reflecting high-level semantics obtained from the local model of the i-th client.

[0109] In an exemplary embodiment, based on the above solution, the similarity calculation module 1020 is specifically configured to: for the ath client and the bth client among the N clients, calculate similarities based on the feature vectors corresponding to the ath client and the bth client, respectively, to obtain an N×N similarity matrix, where the values ​​of a and b are any integer between 1 and N, inclusive.

[0110] The client grouping module 1030 is specifically configured to group the N clients into M client sets according to the N×N similarity matrix and a first preset threshold.

[0111] In an exemplary embodiment, based on the above solution, the similarity calculation module 1020 is further specifically configured to: calculate the similarity between the x-th feature vector and the feature vectors corresponding to other clients, respectively, to obtain N-1 similarities, where x is any integer between 1 and N, inclusive.

[0112] The client grouping module 1030 is further specifically configured to compare the N-1 similarities with M preset intervals, distribute the N-1 similarities into the M preset intervals, and group the N clients into M client sets.

[0113] In an exemplary embodiment, based on the aforementioned solution, the j-th client set includes L clients, where L is a positive integer not greater than (N-M+1);

[0114] The above-mentioned aggregation processing module 1040 is specifically used to: receive the model data of the local model corresponding to each client in the above-mentioned j-th client set, and the above-mentioned model data is the model weight or model gradient; and perform weighted averaging processing on the model data corresponding to L clients respectively, and update the global model according to the weight after weighted averaging processing to obtain the personalized model corresponding to the above-mentioned j-th client set.

[0115] In an exemplary embodiment, based on the aforementioned solution, the apparatus 1000 for implementing federated learning further includes: an updating module 1050 .

[0116] Among them, the above-mentioned update module 1050 is used to: after obtaining the personalized model corresponding to the above-mentioned j-th client set, when the above-mentioned i-th client belongs to the j-th client set, in response to receiving a close request sent from the above-mentioned i-th client, update the j-th client set where the above-mentioned i-th client is located to obtain the j'-th client set, wherein the above-mentioned j'-th client set does not include the above-mentioned i-th client, and the value of j' is a positive integer not greater than M; and, aggregate the model data of the local model corresponding to each client in the updated j'-th client set to obtain the personalized model corresponding to the above-mentioned j'-th client set.

[0117] In an exemplary embodiment, based on the aforementioned solution, the above-mentioned i-th feature vector is: obtained from the local model obtained after the above-mentioned federated learning task performs a preset number of local iterations on the above-mentioned i-th client.

[0118] It should be noted that the federated learning implementation device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when executing the federated learning implementation method provided in the above embodiment. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0119] in, Figure 12 This is a schematic diagram of the structure of the implementation device of federated learning provided in one embodiment of this specification. Figure 12 The implementation device of federated learning shown in the figure can be configured on any client (i-th client) participating in the above-mentioned federated learning task through software, hardware or a combination of both, and can also be integrated on the server as an independent module, and can also be integrated into an electronic device as an independent module.

[0120] The above-mentioned federated learning implementation device 1200 in the embodiment of this specification includes: a feature determination module 1210, a feature sending module 1220, and a model data sending module 1230.

[0121] Among them, the above-mentioned feature determination module 1210 is used to obtain the i-th feature vector from the local model, and the above-mentioned i-th feature vector reflects the data distribution characteristics of the above-mentioned i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; the above-mentioned feature sending module 1220 is used to send the above-mentioned i-th feature vector to the central server of the above-mentioned federated learning task, so that the above-mentioned central server: calculates the similarity between the above-mentioned N clients according to the feature vectors corresponding to the above-mentioned N clients respectively; and divides the above-mentioned N clients into M client sets according to the similarity between the above-mentioned N clients, where M is a positive integer less than N; and the above-mentioned model data sending module 1230 is used to send the model data of the local model to the above-mentioned central server when the above-mentioned i-th client belongs to the j-th client set, so that the above-mentioned central server aggregates the model data of the local models corresponding to all clients in the above-mentioned j-th client set to obtain the personalized model corresponding to the above-mentioned j-th client set, and j is a positive integer not greater than M.

[0122] In an exemplary embodiment, Figure 13 This is a schematic diagram of the structure of a device for implementing federated learning provided in another embodiment of this specification. Figure 13 :

[0123] In an exemplary embodiment, based on the aforementioned scheme, the feature determination module 1210 is specifically used to: after the above-mentioned federated learning task performs a preset number of local iterations on the above-mentioned i-th client, obtain a vector reflecting high-level semantics from the obtained local model to obtain the above-mentioned i-th feature vector.

[0124] In an exemplary embodiment, based on the aforementioned scheme, the feature determination module 1210 is further specifically used to: determine a target layer reflecting high-level semantics from the local model; compare the model weights in the target layer with a third preset threshold; and set the values ​​below the third preset threshold to 0 to obtain a sparse vector, and determine the sparse vector as the i-th feature vector.

[0125] In an exemplary embodiment, based on the aforementioned solution, the apparatus 1200 for implementing federated learning further includes: a request sending module 1240 .

[0126] The request sending module 1240 is configured to: after obtaining the personalized model corresponding to the j-th client set, if the i-th client belongs to the j-th client set, send a close request to the central server, so that the central server: obtains the j'-th client set based on updating the j-th client set, wherein the j'-th client set does not include the i-th client; and aggregates the model data of the local model corresponding to each client in the updated j'-th client set to obtain the personalized model corresponding to the j'-th client set;

[0127] The above-mentioned model data includes model weights or model gradients.

[0128] It should be noted that when the federated learning implementation device provided in the above embodiment executes the federated learning implementation method provided in the above embodiment, the division of the above-mentioned functional modules is only used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0129] In addition, the implementation device of federated learning provided in the above embodiment and the implementation method embodiment of federated learning belong to the same concept. Therefore, for details not disclosed in the device embodiment of this specification, please refer to the implementation method embodiment of federated learning above in this specification, and no further details will be given here.

[0130] The embodiments of this specification also provide a system for implementing federated learning. Figure 14 This is a schematic diagram of the structure of the federated learning implementation system provided in the embodiments of this specification.

[0131] refer to Figure 14 The federated learning implementation system 1400 provided in this embodiment includes: N clients participating in the federated learning task and a central server 1410.

[0132] In which, the above-mentioned central server 1410 executes an embodiment of the implementation method of federated learning; and, any client among the above-mentioned N clients (the i-th client) executes an embodiment of the implementation method of federated learning, where N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N.

[0133] For example, Figure 15 This is a structural diagram of a federated learning implementation system provided in another embodiment of this specification. Figure 15, wherein each client may further include a feature extractor, wherein the feature extractor is used to obtain a feature vector from the local model of the client, and the feature vector reflects the data distribution characteristics of the client. Exemplarily, the feature extractor may be used to perform the following Figure 3 The embodiments corresponding to S310 and S320 are not described in detail here.

[0134] Continue to refer Figure 15 The central server 1410 includes a similarity calculator and a scheduler. The similarity calculator is used to calculate the similarity between clients based on the feature vectors corresponding to each client. The scheduler can be used to group the clients into multiple client sets based on the similarity between the clients.

[0135] Exemplarily, the similarity calculator can be used to perform the following operations: Figure 2 In the embodiment corresponding to S220, the above scheduler can be used to execute the following Figure 2 The embodiment corresponding to S230 will not be described in detail here.

[0136] The serial numbers of the embodiments in this specification are for description only and do not represent the advantages or disadvantages of the embodiments.

[0137] The embodiments of this specification also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of any of the above-mentioned methods are implemented.

[0138] Figure 16 This is a schematic diagram of the structure of the electronic device provided in the embodiment of this specification. Figure 16 As shown, the electronic device 1600 includes a processor 1601 and a memory 1602 .

[0139] In the embodiments of this specification, the processor 1601 is the control center of the computer system, which can be the processor of a physical machine or the processor of a virtual machine. The processor 1601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1601 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state; the coprocessor is a low-power processor for processing data in the standby state.

[0140] In the embodiment of this specification, when the electronic device is a central server participating in a federated learning task, the processor 1601 belongs to the central server participating in the federated learning task. Then the processor 1601 can be specifically used to:

[0141] Receive the feature vectors corresponding to the N clients of the above-mentioned federated learning task, where the i-th feature vector corresponding to the i-th client is obtained from the local model of the i-th client, and the i-th feature vector reflects the data distribution characteristics of the i-th client, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N; calculate the similarity between the N clients based on the feature vectors corresponding to the N clients; divide the N clients into M client sets based on the similarity between the N clients, where M is a positive integer less than N; and, for the j-th client set, aggregate the model data of the local models corresponding to all clients in the j-th client set to obtain the personalized model corresponding to the j-th client set, where j is a positive integer not greater than M.

[0142] Furthermore, the above-mentioned i-th feature vector is a sparse vector reflecting high-level semantics obtained from the local model of the above-mentioned i-th client.

[0143] Further, calculating the similarity between the N clients based on the feature vectors corresponding to the N clients includes: for the ath client and the bth client among the N clients, calculating the similarity based on the feature vectors corresponding to the ath client and the bth client, respectively, to obtain an N×N similarity matrix, where the values ​​of a and b are any integer between 1 and N, inclusive;

[0144] The dividing the N clients into M client sets according to the similarities between the N clients includes dividing the N clients into M client sets according to the N×N similarity matrix and a first preset threshold.

[0145] Further, calculating the similarities between the N clients based on the feature vectors corresponding to the N clients includes: calculating the similarities between the x-th feature vector and the feature vectors corresponding to other clients, respectively, to obtain N-1 similarities, where x is any integer between 1 and N, inclusive.

[0146] The above-mentioned dividing the N clients into M client sets according to the similarities between the N clients includes: comparing the N-1 similarities with M preset intervals, distributing the N-1 similarities into the M preset intervals, so as to divide the N clients into M client sets.

[0147] Furthermore, the j-th client set includes L clients, and the value of L is a positive integer not greater than (N-M+1);

[0148] The above-mentioned aggregating the model data of the local models corresponding to all clients in the above-mentioned j-th client set to obtain the personalized model corresponding to the above-mentioned j-th client set includes: receiving the model data of the local model corresponding to each client in the above-mentioned j-th client set, where the above-mentioned model data is the model weight or the model gradient; and performing weighted averaging processing on the model data corresponding to the L clients respectively, and updating the global model according to the weights after the weighted averaging processing to obtain the personalized model corresponding to the above-mentioned j-th client set.

[0149] Furthermore, the processor 1601 is also specifically configured to: after obtaining the personalized model corresponding to the j-th client set, when the i-th client belongs to the j-th client set, in response to receiving a close request sent from the i-th client, update the j-th client set where the i-th client is located to obtain the j'-th client set, wherein the j'-th client set does not include the i-th client, and the value of j' is a positive integer not greater than M; and, aggregate the model data of the local model corresponding to each client in the updated j'-th client set to obtain the personalized model corresponding to the j'-th client set.

[0150] Furthermore, the above-mentioned i-th eigenvector is: obtained from the local model obtained after the above-mentioned federated learning task performs local iterations for a preset number of iterations on the above-mentioned i-th client.

[0151] In the embodiment of this specification, if the electronic device is any client participating in the federated learning task, the processor 1601 belongs to any client participating in the federated learning task. Then the processor 1601 can be specifically used to:

[0152] Obtain the i-th eigenvector from the local model, where the i-th eigenvector reflects the data distribution characteristics of the i-th client, where N is an integer greater than 1, and i is any integer between 1 and N, inclusive. Send the i-th eigenvector to the central server of the federated learning task, so that the central server: calculates the similarity between the N clients based on the eigenvectors corresponding to the N clients respectively; and, based on the similarity between the N clients, divides the N clients into M client sets, where M is a positive integer less than N. And, in the case that the i-th client belongs to the j-th client set, send the model data of the local model to the central server, so that the central server aggregates the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M.

[0153] Furthermore, the above-mentioned obtaining of the i-th feature vector from the local model includes: after the above-mentioned federated learning task performs a preset number of local iterations on the above-mentioned i-th client, obtaining a vector reflecting high-level semantics from the obtained local model to obtain the above-mentioned i-th feature vector.

[0154] Furthermore, the above-mentioned obtaining of the i-th eigenvector from the local model includes: determining a target layer reflecting high-level semantics from the above-mentioned local model; comparing the model weights in the above-mentioned target layer with a third preset threshold; and setting the weights below the above-mentioned third preset threshold to 0 to obtain a sparse vector, and determining the above-mentioned sparse vector as the above-mentioned i-th eigenvector.

[0155] Furthermore, the processor 1601 is further specifically configured to: after obtaining the personalized model corresponding to the j-th client set, if the i-th client belongs to the j-th client set, send a close request to the central server, so that the central server: obtains a j'-th client set based on updating the j-th client set, wherein the j'-th client set does not include the i-th client; aggregates the model data of the local model corresponding to each client in the updated j'-th client set to obtain the personalized model corresponding to the j'-th client set;

[0156] The above-mentioned model data includes model weights or model gradients.

[0157] Memory 1602 may include one or more computer-readable storage media, which may be non-transitory. Memory 1602 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices. In some embodiments of this specification, the non-transitory computer-readable storage medium in memory 1602 is used to store at least one instruction, which is executed by processor 1601 to implement the method of the embodiments of this specification.

[0158] In some embodiments, electronic device 1600 further includes a peripheral device interface 1603 and at least one peripheral device. Processor 1601, memory 1602, and peripheral device interface 1603 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1603 via a bus, signal lines, or circuit boards. Specifically, the peripheral device includes at least one of a display screen 1604, a camera 1605, and an audio circuit 1606.

[0159] The peripheral device interface 1603 can be used to connect at least one input / output (I / O)-related peripheral device to the processor 1601 and the memory 1602. In some embodiments of this specification, the processor 1601, the memory 1602, and the peripheral device interface 1603 are integrated on the same chip or circuit board; in some other embodiments of this specification, any one or two of the processor 1601, the memory 1602, and the peripheral device interface 1603 can be implemented on separate chips or circuit boards. This embodiment of this specification is not specifically limited to this.

[0160] The display screen 1604 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1604 is a touch screen display, the display screen 1604 also has the ability to collect touch signals on the surface or above the surface of the display screen 1604. The touch signal can be input as a control signal to the processor 1601 for processing. At this time, the display screen 1604 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments of this specification, the display screen 1604 can be one, set on the front panel of the electronic device 1600; in other embodiments of this specification, the display screen 1604 can be at least two, respectively set on different surfaces of the electronic device 1600 or in a folding design; in still other embodiments of this specification, the display screen 1604 can be a flexible display screen, set on the curved surface or folding surface of the electronic device 1600. Even more, the display screen 1604 can be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1604 can be made of materials such as liquid crystal display (LCD) and organic light-emitting diode (OLED).

[0161] Camera 1605 is used to capture images or videos. Optionally, camera 1605 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the electronic device, and the rear camera is arranged on the back of the electronic device. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and virtual reality (VR) shooting function or other fusion shooting functions. In some embodiments of this specification, camera 1605 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0162] Audio circuit 1606 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, convert the sound waves into electrical signals, and input them into processor 1601 for processing. For the purpose of stereo sound collection or noise reduction, multiple microphones may be provided, respectively, at different locations within electronic device 1600. The microphone may also be an array microphone or an omnidirectional microphone.

[0163] Power supply 1607 is used to power the various components of electronic device 1600. Power supply 1607 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1607 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0164] The electronic device structure block diagram shown in the embodiment of this specification does not constitute a limitation on the electronic device 1600. The electronic device 1600 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0165] In the description of this specification, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. For those skilled in the art, the specific meanings of the above terms in this specification can be understood according to specific circumstances. In addition, in the description of this specification, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0166] The embodiments of this specification also provide a computer-readable storage medium containing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of the aforementioned embodiments. If the various components of the aforementioned federated learning implementation device are implemented as software functional units and sold or used as independent products, they may be stored in the aforementioned computer-readable storage medium.

[0167] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of this specification is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium or transmitted by the above-mentioned computer-readable storage medium. The above-mentioned computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVDs)), or semiconductor media (for example, solid state disks (SSDs)).

[0168] It should be noted that the foregoing description of this specification is based on specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0169] The above description is merely a specific embodiment of this specification, but the scope of protection of this specification is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this specification should be covered by the scope of protection of this specification. Therefore, equivalent variations made according to the claims of this specification are still within the scope of protection of this specification.

Claims

1. A method for implementing federated learning, wherein: Applied to a central server participating in a federated learning task, the method includes: Receive feature vectors corresponding to N clients of the federated learning task, where the i-th feature vector corresponding to the i-th client is obtained from the local model of the i-th client, and the i-th feature vector reflects the data distribution characteristics of the i-th client. N is an integer greater than 1, and i is any integer between 1 and N, inclusive. Calculating similarities between the N clients based on the feature vectors corresponding to the N clients respectively; Divide the N clients into M client sets according to the similarities between the N clients, where M is a positive integer less than N; For the j-th client set, aggregating the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M; Before receiving the feature vectors corresponding to the N clients of the federated learning task, the method further includes: After the federated learning task iterates a preset number of times, a global model is obtained; The global model is sent to the N clients of the federated learning task respectively, so that after the i-th client performs a preset number of local iterations, a sparse vector reflecting high-level semantics is obtained from the obtained local model to obtain the i-th feature vector.

2. The method according to claim 1, wherein Calculating the similarity between the N clients based on the feature vectors corresponding to the N clients includes: For client a and client b among the N clients, similarity is calculated based on the feature vectors corresponding to client a and client b, respectively, to obtain an N×N similarity matrix, where the values ​​of a and b are any integer between 1 and N, inclusive. The dividing the N clients into M client sets according to the similarities between the N clients includes: The N clients are divided into M client sets according to the N×N similarity matrix and a first preset threshold.

3. The method according to claim 1, wherein Calculating the similarity between the N clients based on the feature vectors corresponding to the N clients includes: Calculate the similarity between the x-th feature vector and the feature vectors corresponding to each other client, and obtain N-1 similarities, where x is any integer between 1 and N, inclusive. The dividing the N clients into M client sets according to the similarities between the N clients includes: The N-1 similarities are compared with M preset intervals, and the N-1 similarities are distributed into the M preset intervals, so as to divide the N clients into M client sets.

4. The method according to any one of claims 1 to 3, wherein: The j-th client set includes L clients, where L is a positive integer not greater than (N-M+1); The aggregating model data of local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set includes: Receive model data of a local model corresponding to each client in the j-th client set, where the model data is a model weight or a model gradient; A weighted average process is performed on the model data corresponding to the L clients respectively, and the global model is updated according to the weights obtained after the weighted average process to obtain a personalized model corresponding to the j-th client set.

5. The method according to any one of claims 1 to 3, wherein: After obtaining the personalized model corresponding to the j-th client set, the method further includes: When the i-th client belongs to the j-th client set, in response to receiving a close request sent by the i-th client, updating the j-th client set to which the i-th client belongs to obtain a j'-th client set, wherein the j'-th client set does not include the i-th client, and the value of j' is a positive integer not greater than M; Aggregation processing is performed on the model data of the local model corresponding to each client in the updated j'-th client set to obtain a personalized model corresponding to the j'-th client set.

6. A method for implementing federated learning, wherein: Applied to an i-th client participating in a federated learning task, the method includes: Obtaining an i-th eigenvector from the local model, where the i-th eigenvector reflects a data distribution characteristic of the i-th client, where N is an integer greater than 1, and i is any integer between 1 and N, inclusive. Sending the i-th feature vector to the central server of the federated learning task, so that the central server: calculates the similarity between the N clients based on the feature vectors corresponding to the N clients; and divides the N clients into M client sets based on the similarity between the N clients, where M is a positive integer less than N; When the i-th client belongs to the j-th client set, the model data of the local model is sent to the central server, so that the central server aggregates the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M; Before obtaining the i-th eigenvector from the local model, the method further includes: After the federated learning task has iterated a preset number of times, receiving a global model sent by a central server of the federated learning task; The step of obtaining the i-th eigenvector from the local model includes: After the federated learning task performs a preset number of local iterations on the i-th client, a sparse vector reflecting high-level semantics is obtained from the obtained local model to obtain the i-th feature vector.

7. The method according to claim 6, wherein: The obtaining of the i-th feature vector from the local model includes: Determining a target layer reflecting high-level semantics from the local model; Comparing the model weight in the target layer with a third preset threshold; The values ​​below the third preset threshold are set to 0 to obtain a sparse vector, and the sparse vector is determined as the i-th eigenvector.

8. The method according to claim 6, wherein: After obtaining the personalized model corresponding to the j-th client set, the method further includes: If the i-th client belongs to the j-th client set, a close request is sent to the central server, so that the central server: obtains a j'-th client set according to updating the j-th client set, wherein the j'-th client set does not include the i-th client; and aggregates the model data of the local model corresponding to each client in the updated j'-th client set to obtain a personalized model corresponding to the j'-th client set; The model data includes model weights or model gradients.

9. A federated learning implementation device, configured on a central server participating in a federated learning task, comprising: a feature receiving module, configured to receive feature vectors corresponding to N clients of the federated learning task, wherein the i-th feature vector corresponding to the i-th client is obtained from the local model of the i-th client, and the i-th feature vector reflects the data distribution characteristics of the i-th client, N is an integer greater than 1, and i is any integer between 1 and N, inclusive; A similarity calculation module, configured to calculate the similarity between the N clients based on the feature vectors corresponding to the N clients; a client grouping module, configured to group the N clients into M client sets according to similarities between the N clients, where M is a positive integer less than N; an aggregation processing module, configured to aggregate the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M; Before receiving the feature vectors corresponding to the N clients of the federated learning task, the method further includes: After the federated learning task iterates a preset number of times, a global model is obtained; The global model is sent to the N clients of the federated learning task respectively, so that after the i-th client performs a preset number of local iterations, a sparse vector reflecting high-level semantics is obtained from the obtained local model to obtain the i-th feature vector.

10. A federated learning implementation device, configured on an i-th client participating in a federated learning task, comprising: a feature determination module, configured to obtain an i-th feature vector from a local model, wherein the i-th feature vector reflects a data distribution feature of the i-th client, where N is an integer greater than 1, and i is any integer between 1 and N, inclusive; a feature sending module, configured to send the i-th feature vector to a central server of the federated learning task, so that the central server: calculates similarities between the N clients based on the feature vectors corresponding to the N clients; and divides the N clients into M client sets based on the similarities between the N clients, where M is a positive integer less than N; a model data sending module, configured to send the model data of the local model to the central server when the i-th client belongs to the j-th client set, so that the central server aggregates the model data of the local models corresponding to all clients in the j-th client set to obtain a personalized model corresponding to the j-th client set, where j is a positive integer not greater than M; Before obtaining the i-th eigenvector from the local model, the method further includes: After the federated learning task has iterated a preset number of times, receiving a global model sent by a central server of the federated learning task; The step of obtaining the i-th eigenvector from the local model includes: After the federated learning task performs a preset number of local iterations on the i-th client, a sparse vector reflecting high-level semantics is obtained from the obtained local model to obtain the i-th feature vector.

11. A federated learning implementation system, comprising N clients and a central server participating in a federated learning task, wherein: The central server executes the method for implementing federated learning according to any one of claims 1 to 5; and The i-th client executes the implementation method of federated learning as described in any one of claims 6 to 8, N is an integer greater than 1, and i is any integer between 1 and N, including 1 and N.

12. A computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to execute the method for implementing federated learning as described in any one of claims 1 to 5; and implement the method for implementing federated learning as described in any one of claims 6 to 8.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the method for implementing federated learning as described in any one of claims 1 to 5; and it implements the method for implementing federated learning as described in any one of claims 6 to 8.

14. A computer program product comprising instructions, which, when executed on a computer or processor, causes the computer or processor to execute the method for implementing federated learning as claimed in any one of claims 1 to 5; and implement the method for implementing federated learning as claimed in any one of claims 6 to 8.

Images