Parameter aggregation method, system, device and storage medium based on federated learning

By performing parameter similarity matching and aggregation in federated learning, the problem of data heterogeneity is solved, the model accuracy and iteration speed are improved, and data privacy is protected.

CN118504622BActive Publication Date: 2024-11-29CETC NEW SMART CITY RES INST CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410954483.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-29
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

The existing federated learning methods do not consider data heterogeneity when aggregating parameters, resulting in insufficient accuracy of global models and local models, affecting the effectiveness of the model processing data.

Method used

By optimizing the parameter similarity between the server and the client, select the parameters with the best similarity in each local model for aggregation, update the global model, and conduct local data training on the client to iteratively update the model.

Benefits of technology

It improves the parameter aggregation effect, improves the accuracy of global and local models, adapts to networks of different sizes, speeds up the iteration speed of model, reduces the cost of iteration updates, and protects data privacy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118504622B_ABST
    Figure CN118504622B_ABST
Patent Text Reader

Abstract

The present application is applicable to the field of artificial intelligence technology, and provides a parameter aggregation method, system, device and storage medium based on federated learning, the method comprising: after obtaining multiple local neuron parameters of any local model in a target network layer, solving the optimal matching relationship based on parameter similarity for the multiple local neuron parameters and the multiple global neuron parameters of the global model in the target network layer, and obtaining a target parameter matching method; according to the target parameter matching methods of all local models, performing parameter aggregation on the multiple local neuron parameters of each local model in the target network layer, and updating the target network layer of the global model; sending the updated multiple global neuron parameters of the target network layer to each client for local model updating; the parameter aggregation effect and the accuracy of the model parameters can be effectively improved, thereby improving the model accuracy, and further effectively improving the data processing effect of the local model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a parameter aggregation method, system, device and storage medium based on federated learning. Background Art

[0002] Federated learning is a distributed machine learning framework that allows data to remain on each client device, and only model updates (usually gradients or model parameters) are shared among participants. It can coordinate the computing resources of multiple clients to train high-performance machine learning models while protecting user privacy and ensuring data security. During the federated learning process, the server collects the parameters uploaded by all clients, updates the global model based on the parameters uploaded by the clients, and then distributes the updated global model parameters to each client.

[0003] Currently, when updating the global model based on the parameters uploaded by the client, the corresponding parameters are usually simply averaged and aggregated to obtain the global model parameters. The impact of data heterogeneity of each client is not considered, resulting in poor parameter aggregation effect, insufficient accuracy of the global model and local model, and affecting the effect of the model in processing data. Summary of the invention

[0004] The present invention provides a parameter aggregation method, system, device and storage medium based on federated learning to solve the problem that when the existing model is updated, the parameter aggregation effect is poor, resulting in insufficient accuracy of the global model and the local model, which affects the effect of the model in processing data.

[0005] In a first aspect, an embodiment of the present application provides a parameter aggregation method based on federated learning, including:

[0006] After obtaining multiple local neuron parameters of the local model on any client in the target network layer, the multiple local neuron parameters and the multiple global neuron parameters of the global model on the server in the same target network layer are solved based on parameter similarity to obtain the target parameter matching method of the target network layer in the local model;

[0007] After obtaining the target parameter matching methods of all local models, multiple local neuron parameters of each local model in the target network layer are aggregated according to each target parameter matching method, and the target network layer of the global model is updated;

[0008] The updated multiple global neuron parameters of the target network layer are sent to each client.

[0009] Optionally, a target parameter matching method of a target network layer in a local model is obtained, including:

[0010] Randomly matching each global neuron parameter with each local neuron parameter to obtain multiple parameter matching modes, wherein each global neuron is matched with only one local neuron in the parameter matching mode;

[0011] Based on the maximum a posteriori probability estimation algorithm, the parameter matching method with the largest sum of parameter similarities among multiple parameter matching methods is determined to obtain the target parameter matching method. The similarity of each parameter in the parameter matching method is the posterior probability of the global neuron parameter that matches the local neuron parameter when it is observed.

[0012] Optionally, based on a maximum a posteriori probability estimation algorithm, a parameter matching method with the largest sum of parameter similarities among multiple parameter matching methods is determined to obtain a target parameter matching method, including:

[0013] Determine the posterior probability of each global neuron parameter in each parameter matching mode respectively, and obtain multiple parameter similarities in each parameter matching mode;

[0014] The similarities of multiple parameters in each parameter matching method are summed up respectively to obtain the sum of the parameter similarities of each parameter matching method;

[0015] Based on the parameter matching method with the largest sum of parameter similarities, the target parameter matching method is obtained.

[0016] Optionally, the target parameter matching method is obtained based on the parameter matching method with the maximum sum of parameter similarities, including:

[0017] Determine the parameter matching method with the maximum sum of multi-parameter similarities, and obtain the undetermined parameter matching method;

[0018] When it is determined that there is a bad match in the pending parameter matching method, the bad match in the pending parameter matching method is marked to obtain the target parameter matching method. The bad match refers to a neuron matching group whose parameter similarity in the pending parameter matching method is less than a preset threshold.

[0019] Optionally, performing parameter aggregation on multiple local neuron parameters of each local model in the target network layer to update the target network layer of the global model includes:

[0020] For each global neuron parameter, a plurality of local neuron parameters matching the global neuron parameter are determined according to each target parameter matching method;

[0021] When there is a marked bad match in any target parameter matching mode, the local neuron parameters in the bad match are eliminated from the multiple local neuron parameters that match the global neuron parameters, and multiple local neuron parameters are optimized;

[0022] The optimized multiple local neuron parameters are aggregated to obtain the aggregated neuron parameters of the global neuron parameters, and the aggregated neuron parameters of each global neuron parameter are updated to the target network layer of the global model.

[0023] Optionally, after determining a plurality of local neuron parameters matching the global neuron parameters, the method further comprises:

[0024] When all target parameter matching methods do not have marked bad matches, all local neuron parameters matching the global neuron parameters are directly aggregated to obtain the aggregated neuron parameters of the global neuron parameters;

[0025] The aggregated neuron parameters of each global neuron parameter are updated to the target network layer of the global model.

[0026] In a second aspect, an embodiment of the present application provides a parameter aggregation method based on federated learning, including:

[0027] Receiving multiple global neuron parameters of the target network layer sent by the server, and updating the target network layer in the local model of the client based on the multiple global neuron parameters of the target network layer, where the multiple global neuron parameters are aggregated by the server based on the above-mentioned parameter aggregation method based on federated learning;

[0028] When the updated local model converges, the local model is output as the target model, so that the target model is used to predict the latest local data of the client.

[0029] Optionally, after updating the target network layer in the local model of the client, the method further includes: when the updated local model has not converged, training the updated local model using the local data of the client; sending multiple local neuron parameters of the trained local model in the target network layer to the server for iteration and updating until the updated local model converges.

[0030] In a third aspect, an embodiment of the present application provides a federated learning system, including a server and multiple clients, wherein the server is used to:

[0031] After obtaining multiple local neuron parameters of the local model on any client in the target network layer, the multiple local neuron parameters and the multiple global neuron parameters of the global model on the server in the same target network layer are solved based on parameter similarity to obtain the target parameter matching method of the target network layer in the local model;

[0032] After obtaining the target parameter matching methods of all local models, multiple local neuron parameters of each local model in the target network layer are aggregated according to each target parameter matching method, and the target network layer of the global model is updated;

[0033] The updated multiple global neuron parameters of the target network layer are sent to each client.

[0034] Optionally, each client is used to:

[0035] Receiving multiple global neuron parameters of the target network layer sent by the server, and updating the target network layer in the local model of the client based on the multiple global neuron parameters of the target network layer;

[0036] When the updated local model converges, the current local model is output as the target model, so as to use the target model to predict the latest local data of the client.

[0037] In a fourth aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned parameter aggregation method based on federated learning when executing the computer program.

[0038] In a fifth aspect, an embodiment of the present application provides a readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned parameter aggregation method based on federated learning are implemented.

[0039] In a solution provided by the above-mentioned parameter aggregation method, system, device and storage medium based on federated learning, after obtaining multiple local neuron parameters of the local model on any client in the target network layer, the multiple local neuron parameters and the multiple global neuron parameters of the global model on the server in the same target network layer are solved based on parameter similarity to obtain the target parameter matching method of the target network layer in the local model; after obtaining the target parameter matching method of all local models, the multiple local neuron parameters of each local model in the target network layer are parameter aggregated according to each target parameter matching method, and the target network layer of the global model is updated; the updated multiple global neuron parameters of the target network layer are sent to each client so that each client is updated to the target network layer of the local model; in this embodiment, the local neuron parameters with the best similarity in each local model can be selected for aggregation, which can effectively improve the parameter aggregation effect and the accuracy of the model parameters, thereby improving the accuracy of the global model and the local model, and then effectively improving the data processing effect of the local model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0041] Figure 1 is a structural diagram of a federated learning system in one embodiment of the present invention;

[0042] Figure 2 It is a flow chart of a parameter aggregation method based on federated learning in one embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram of one-parameter aggregation of the coordinate neuron averaging method;

[0044] Figure 4 is a schematic diagram of parameter aggregation based on a target parameter matching method in one embodiment of the present invention;

[0045] Figure 5 yes Figure 2 A schematic diagram of an implementation process of step S10;

[0046] Figure 6 yes Figure 2 A schematic diagram of an implementation process of step S20;

[0047] Figure 7 yes Figure 1 A structural diagram of a server in FIG.

[0048] Figure 8 yes Figure 1 A structural diagram of a client in FIG.

[0049] Fig. 9 It is a structural diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or their collections. It should also be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0052] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0054] It should be understood that the order of execution of the steps in the following embodiments does not imply a precedence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0055] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0056] The parameter aggregation method based on federated learning provided in the embodiment of the present invention can be applied in Figure 1In the federated learning system shown, the federated learning system includes a server and multiple clients, wherein each client communicates with the server through a network. A global model is provided on the server, and a local model is provided on each client. The client sends multiple local neuron parameters of various local models in the target network layer to the server. The server needs to receive the multiple local neuron parameters of the local models in the target network layer sent by each client, and update its own global model, thereby sending the updated multiple global neuron parameters of the global model in the target network layer to each client, so that each client updates the local local model according to the multiple global neuron parameters sent by the server, and obtains a converged local model after multiple rounds of iterations, so that each client can use the converged local model to process local data (privatized data). Among them, Figure 1 of The global model of the server is represented in t Multiple global neuron parameters when sub-parameters are aggregated; Figure 1 In , , , Respectively represent the first, second, and k , k+1 Client No. t Multiple local neuron parameters during subparameter aggregation.

[0057] Specifically, the server is used to: after obtaining multiple local neuron parameters of the local model on any client in the target network layer, solve the optimal matching relationship based on parameter similarity for the multiple local neuron parameters and the multiple global neuron parameters of the global model on the server in the same target network layer, and obtain the target parameter matching method of the target network layer in the local model; after obtaining the target parameter matching method of all local models, perform parameter aggregation on the multiple local neuron parameters of the target network layer in each local model according to each target parameter matching method, and update the target network layer of the global model; send the updated multiple global neuron parameters of the target network layer to each client.

[0058] Specifically, the client is used to: receive multiple global neuron parameters of the target network layer sent by the server, and update the target network layer in the local model of the client based on the multiple global neuron parameters of the target network layer; when the updated local model converges, output the current local model as the target model, so as to use the target model to predict the latest local data of the client.

[0059] In this embodiment, the local neuron parameters with the best similarity in each local model can be selected for aggregation through parameter similarity matching, which can effectively handle the influence of heterogeneous data from different clients and different operating conditions, effectively improve the parameter aggregation effect, and improve the accuracy of model parameters, thereby improving the accuracy of global models and local models, and further effectively improving the data processing effect of local models. During each parameter aggregation, the server can only match and aggregate the neuron parameters of a target network layer to achieve layer-by-layer parameter aggregation of the model network layer, which enables the algorithm to effectively utilize the computing resources of each client and adapt to networks of different sizes, which can speed up the model iteration speed and reduce the model iteration update cost.

[0060] In addition, after updating the target network layer in the client's local model, the client is also used to: train the updated local model using the client's current local data when the updated local model has not converged; send multiple local neuron parameters of the trained local model in the target network layer to the server for iteration and update until the updated local model converges. When the local model has not converged, the edge device (client) is allowed to jointly train the model without sharing local data (private data), which protects data privacy, increases local model performance, and retains the unique feature extractor in the client's local model, reducing information dilution, thereby improving model accuracy and, in turn, improving the accuracy of model processing data.

[0061] The client, also known as the user end, refers to the program that corresponds to the server and provides local services to customers. The client can be installed on, but not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0062] In one embodiment, if Figure 2 As shown in the figure, a parameter aggregation method based on federated learning is provided. Figure 1 The federated learning system in is taken as an example, which includes the following steps:

[0063] S10: After obtaining multiple local neuron parameters of the local model on any client in the target network layer, the server solves the optimal matching relationship based on parameter similarity for the multiple local neuron parameters and the multiple global neuron parameters of the global model on the server in the same target network layer, and obtains the target parameter matching method of the target network layer in the local model.

[0064] When parameter aggregation is required, each client needs to send multiple local neuron parameters of the local local model in the target network layer to the server. After obtaining multiple local neuron parameters of the local model in the target network layer on any client, the server obtains multiple global neuron parameters of the global model in the same target network layer, and then the server will solve the optimal matching relationship based on parameter similarity for the multiple local neuron parameters of the local model in the target network layer and the multiple global neuron parameters of the global model in the same target network layer, and obtain the target parameter matching method of the target network layer in the local model. Among them, the local neuron parameters are the parameters of the local neurons of the local model in the target network layer; the global neuron parameters are the parameters of the global neurons of the global model in the target network layer.

[0065] Specifically, the server will determine the parameter similarity between each global neuron parameter and the local neuron parameter based on a preset parameter similarity function, and then optimize the matching relationship between multiple local neuron parameters and multiple global neuron parameters based on the parameter similarity between each global neuron parameter and the local neuron parameter, so as to obtain the optimal matching relationship between each global neuron parameter and each local neuron parameter, and obtain the target parameter matching method of the target network layer.

[0066] Among them, in this target parameter matching method, each global neuron matches one and only one local neuron (forming a neuron matching group), that is, each local neuron of the local model is matched to exactly one global neuron of the global model, and each global neuron is also matched by only one local neuron in the local model.

[0067] Among them, the target parameter matching method can be solved by the following formula:

[0068] ;

[0069] ;

[0070] ;

[0071] in, Representing a local model j At the target network layer l local neuron parameters; Represents the global model in the target network layer i Global neuron parameters; represents the parameter similarity function, that is and The parameter similarity of , for example, can be defined as Gaussian distribution client model j No.l The posterior probability of a neuron. Representing a local model j At the target network layer l neurons, whether they match the global model in the target network layer i A neuron is a binary variable, which can be either 0 or 1. It means minimizing the mismatch between multiple local neuron parameters and global neuron parameters on the target network layer in different local models, that is, maximizing the sum of the similarities of all matching neuron parameters. Represents the number of neurons in the local model in the target network layer; Represents the number of neurons in the target network layer of the global model. represents a constraint that ensures that each local neuron of the local model is matched to exactly one global neuron of the global model, and that each global neuron is matched by exactly one local neuron in the local model.

[0072] Among them, the parameter similarity function can be a Euclidean distance function or a cosine similarity function, that is, the Euclidean distance algorithm or the cosine similarity algorithm is used to determine the parameter similarity of each global neuron parameter and each local neuron parameter, so as to obtain the optimal matching relationship between the two based on the similarity of each parameter, that is, the target parameter matching method. In other embodiments, other algorithms can also be used to determine the target parameter matching method, which will not be repeated here.

[0073] Each time the server receives multiple local neuron parameters of a local model in the target network layer, it executes the above parameter matching process until the target parameter matching method of the local models of all clients is obtained. Among them, the target network layer is the hidden layer of the global model and the local model. Only the parameters of the hidden layer are aggregated. On the basis of ensuring the accuracy of the model, the amount of calculation in the Diego update process of the model can be reduced, thereby speeding up the model iteration speed and reducing the cost of model iteration update.

[0074] S20: After obtaining the target parameter matching methods of all local models, the server aggregates the parameters of multiple local neuron parameters of each local model in the target network layer according to each target parameter matching method, and updates the target network layer of the global model.

[0075] After obtaining the target parameter matching methods of all local models, the server aggregates the parameters of multiple local neuron parameters of each local model in the target network layer according to each target parameter matching method, and updates the target network layer of the global model.

[0076] For example, for each global neuron parameter, the server will determine multiple local neuron parameters that match the global neuron parameter according to the target parameter matching methods, aggregate the multiple matching local neuron parameters to obtain the aggregated neuron parameter corresponding to the global neuron parameter, and then update the aggregated neuron parameters corresponding to each global neuron parameter to the target network layer of the global model. That is, the server will determine all local neuron parameters that match each global neuron parameter based on the target parameter matching methods corresponding to each local model; then, aggregate (such as average) all local neuron parameters that match the global neuron parameter to obtain the aggregated neuron parameter corresponding to the global neuron parameter; finally, traverse to obtain the aggregated neuron parameters corresponding to all global neuron parameters, and then update the target network layer of the global model with the aggregated neuron parameters corresponding to each global neuron parameter to obtain the updated target network layer.

[0077] It should be understood that the parameter aggregation method based on federated learning of traditional federated learning is usually a coordinate-wise neuron averaging method, that is, after obtaining the neuron parameters of the local model, the neuron parameters with the same or similar coordinate positions (positions in the model) in each local model are averaged, so as to aggregate and update the neuron parameters of the global model. Compared with the traditional coordinate-wise neuron averaging method, this embodiment calculates the parameter similarity between the global model and each local model, so as to select the local neuron parameters with the best similarity in each local model for aggregation, which can effectively handle the influence of heterogeneous data from different clients and different operating conditions, and can effectively improve the parameter aggregation effect.

[0078] like Figure 3 As shown, in the traditional coordinate-wise neuron averaging method, the server receives multiple local neuron parameters of each local model (including model 1, model 2...model J) in the target network layer; wherein the multiple local neuron parameters of each local model include the parameters of three neurons 1, 2, and 3, and the local neuron coordinate positions corresponding to the same numbers are the same (or close); the server averages the parameters of neuron 1 in each local model to obtain the aggregated parameters of neuron 1 in the global model. The parameter aggregation method based on the target parameter matching method in this embodiment has an aggregation process as follows: Figure 4 As shown, the local neuron parameters matching the neuron 1 in the global model are the parameters of neuron 1 in model 1, the parameters of neuron 3 in model 2, ..., and the parameters of neuron 2 in model J; when performing parameter aggregation, the parameters of neuron 1 in model 1, the parameters of neuron 3 in model 2, ..., and the parameters of neuron 2 in model J are aggregated to obtain the aggregated parameters of neuron 1 in the global model.

[0079] S30: The server sends the updated multiple global neuron parameters of the target network layer to each client.

[0080] After obtaining the target network layer for updating the global model, the updated multiple global neuron parameters of the target network layer are sent to each client, so that the client updates to the target network layer of the local model, thereby realizing the update of the local model.

[0081] S40: The client receives multiple global neuron parameters of the target network layer sent by the server, and updates the target network layer in the local model of the client based on the multiple global neuron parameters of the target network layer.

[0082] After the server sends the updated multiple global neuron parameters of the target network layer to the client, each client receives the multiple global neuron parameters of the target network layer sent by the server. Among them, the multiple global neuron parameters of the target network layer are aggregated by the server based on the above-mentioned parameter aggregation method based on federated learning. The specific process is as mentioned above and will not be repeated here.

[0083] Then, each client will update the target network layer in the client's local model based on the multiple global neuron parameters of the target network layer. For example, the multiple global neuron parameters of the target network layer are directly updated to the target network layer in the local model to obtain an updated local model.

[0084] S50: When the updated local model converges, the current local model is output as the target model, so as to use the target model to predict the latest local data of the client.

[0085] After updating the target network layer in the client's local model, the client will determine whether the current local model has converged. For example, when receiving the model convergence notification from the server, or the client determines that the accuracy or number of iterations of the current local model has reached the requirements, the local model is determined to have converged; when not receiving the convergence notification from the server model, and the client determines that the accuracy and number of iterations of the current local model have reached the requirements, the local model is determined to have not converged. When the server determines that the accuracy or number of iterations of the global model has reached the requirements, the server generates a model convergence notification and sends it to each client.

[0086] When it is determined that the local model has not converged, the client will use the current local data (privatized data) to further train the updated local model to refine the model performance, and send the multiple local neuron parameters of the local model obtained after training in the target network layer to the server, so that the server can iteratively update the global model, and the client can update the local model based on the parameters of the updated global model, that is, the server and the client repeat the above steps S10-S40 until it is determined that the updated local model has converged, and output the current local model as the target model, so as to use the target model to predict the latest local data of the client, thereby improving the data processing effect.

[0087] In this embodiment, the local neuron parameters with the best similarity in each local model can be selected for aggregation through parameter similarity matching, which can effectively handle the influence of heterogeneous data from different clients and different operating conditions, effectively improve the parameter aggregation effect, and improve the accuracy of model parameters, thereby improving the accuracy of global models and local models, and thus effectively improving the data processing effect of local models. At the same time, the use of federated learning technology allows edge devices (clients) to jointly train models without sharing local data, which not only protects data privacy; only similar neuron parameters in each local model are averaged, and the dilution of the processing results of the unique feature extractors in each local model in the global model is reduced, and the unique feature extractors in the local models of the client are retained, which reduces the dilution of information, thereby improving the model accuracy, and then improving the accuracy of the model processing data.

[0088] In addition, the server can only match and aggregate the neuron parameters of one target network layer each time the parameters are aggregated, thereby realizing layer-by-layer parameter aggregation of the model network layer. This enables the algorithm to effectively utilize the computing resources of each client and adapt to networks of different sizes, thereby speeding up the model iteration speed and reducing the cost of model iteration updates.

[0089] In one embodiment, the local data on the client may be the operation data of the device, such as the operation data of the device may be cyclic data such as the health status data of the battery, or non-cyclic data such as the remaining service life data of the engine. The client uses the target model to predict the latest local data of the client to achieve online prediction of the health status of the device, which can not only ensure data privacy, but also improve the accuracy of the prediction, thereby effectively monitoring the health status or remaining service life of the device.

[0090] In one embodiment, both the global model and the local model are neural network models of the long short-term memory network architecture. When the client uses the target model to predict the latest local data of the client, the local data needs to be standardized and sequenced to convert the local data into data suitable for the long short-term memory network architecture input.

[0091] In one embodiment, when the neural network structures of the global model and the local model are not completely the same, in order to prevent the global model from over-growing and over-fitting during the iterative update process, it is necessary to limit the size of the global model, that is, when performing the target parameter matching method, it is necessary to limit the local neurons in the target parameter matching method to limit the number of neurons in the global model that is subsequently aggregated. Before step S10, that is, before solving the optimal matching relationship based on parameter similarity for multiple local neuron parameters and multiple global neuron parameters of the global model on the server at the same target network layer, the method also includes the following steps:

[0092] S01: The server determines whether the number of local neurons of the local model in the target network layer is greater than the number of global neurons of the global model in the target network layer.

[0093] Before solving the optimal matching relationship based on parameter similarity for multiple local neuron parameters and multiple global neuron parameters, the server also needs to determine whether the number of local neurons of the local model in the target network layer is greater than the number of global neurons of the global model in the target network layer, so as to execute different optimal matching relationship solving strategies according to different judgment structures.

[0094] S02: If the number of local neurons is less than or equal to the number of global neurons, the server solves the optimal matching relationship based on parameter similarity for multiple local neuron parameters and multiple global neuron parameters of the global model on the server in the same target network layer based on a preset parameter similarity function, and obtains the target parameter matching method of the target network layer.

[0095] Among them, if the number of local neurons is less than or equal to the number of global neurons, the server performs an optimal matching relationship based on parameter similarity for multiple local neuron parameters and multiple global neuron parameters of the global model on the server in the same target network layer based on a preset parameter similarity function (i.e., a first parameter similarity function), thereby obtaining a target parameter matching method for the target network layer. The process of obtaining the target parameter matching method based on the preset parameter similarity function is described above and will not be repeated here.

[0096] S03: If the number of local neurons is greater than the number of global neurons, the server solves the optimal matching relationship based on parameter similarity for multiple local neuron parameters and multiple global neuron parameters of the global model on the server in the same target network layer based on a preset penalty function and a preset parameter similarity function, and obtains the target parameter matching method of the target network layer.

[0097] Among them, if the number of local neurons is greater than the number of global neurons, based on the preset penalty function and the preset parameter similarity function, the optimal matching relationship based on parameter similarity is solved for multiple local neuron parameters and multiple global neuron parameters of the global model on the server in the same target network layer, thereby obtaining the target parameter matching method of the target network layer.

[0098] The parameter matching cost threshold may be determined based on a preset parameter similarity function, and then a new parameter similarity function (ie, a second parameter similarity function) may be updated based on a preset penalty function and the parameter matching cost threshold.

[0099] Among them, the target parameter matching method of the target network layer can be determined by the following method:

[0100] ;

[0101] ;

[0102] ;

[0103] .

[0104] in, Representing a local model j At the target network layer l local neuron parameters; Represents the global model in the target network layer i Global neuron parameters; represents the preset parameter similarity function, that is and The parameter similarity of , for example, can be defined as Gaussian distribution client model j No. l The posterior probability of a neuron. Representation of local model j At the target network layer l neurons, whether they match the global model in the target network layer i A neuron is a binary variable, which can be either 0 or 1. Indicates the parameter matching cost threshold; represents the preset penalty function; Represents a parameter similarity function determined by whether the number of neurons in the target network layer (global number of neurons) of the global model is consistent with the number of local neurons. It means minimizing the mismatch between the parameters of multiple local neurons on the target network layer in different local models and the global neuron parameters, that is, maximizing the sum of the similarities of all matching local neuron parameters. Represents the number of neurons in the local model in the target network layer; Represents the number of neurons in the target network layer of the global model. Represents a constraint that ensures that each local neuron of the local model is matched to at most one global neuron of the global model, and each global neuron is matched by only one local neuron in the local model.

[0105] In one embodiment, if Figure 5 As shown, in step S10 or S01, the optimal matching relationship based on parameter similarity is solved for multiple local neuron parameters and multiple global neuron parameters of the global model on the server at the same target network layer, and the target parameter matching method of the target network layer in the local model is obtained, which specifically includes the following steps:

[0106] S11: The server randomly matches each global neuron parameter with each local neuron parameter to obtain a plurality of parameter matching modes.

[0107] After obtaining multiple local neuron parameters of the local model in the target network layer on any client, the server randomly matches the global neuron parameters of the same target network layer with the local neuron parameters, and obtains multiple parameter matching methods. Among them, in the parameter matching method, each global neuron matches one and only one local neuron, that is, the local neuron of each local model is exactly matched to the global neuron of a global model, and each global neuron is also matched by one and only one local neuron in the local model.

[0108] S12: The server determines a parameter matching method with the largest sum of parameter similarities among multiple parameter matching methods based on a maximum a posteriori probability estimation algorithm, and obtains a target parameter matching method.

[0109] In this embodiment, the preset parameter similarity function can be a posterior probability function in which a local neuron parameter obeys a Gaussian distribution with a global neuron parameter as the mean. Based on the posterior probability function, the server determines the parameter similarity between each global neuron parameter and the local neuron parameter in each parameter matching method. Each parameter similarity is the posterior probability of the global neuron parameter that matches the local neuron parameter when the local neuron parameter is observed; then, based on the maximum posterior probability estimation algorithm, the parameter matching method with the largest sum of parameter similarities among multiple parameter matching methods is determined to obtain the target parameter matching method.

[0110] In this embodiment, the global model is constructed as a Bayesian probability model that obeys the Beta-Bernoulli process. The preset parameter similarity function can be a posterior probability function that a local neuron parameter obeys a Gaussian distribution with the global neuron parameter as the mean, that is, each parameter similarity is the posterior probability that the local neuron parameter obeys a Gaussian distribution with the global neuron parameter as the mean. That is, each parameter similarity in the parameter matching method is the posterior probability of the global neuron parameter that matches the local neuron parameter when the local neuron parameter is observed.

[0111] Among them, the similarity of each parameter in each parameter matching method is first determined, wherein the similarity of each parameter in the parameter matching method is the posterior probability of the global neuron parameter that matches it when the local neuron parameter is observed; then, among all the parameter matching methods, the parameter matching method with the largest sum of parameter similarities is determined as the target parameter matching method.

[0112] In this embodiment, the server randomly matches each global neuron parameter with each local neuron parameter to obtain multiple parameter matching methods, and then determines the parameter matching method with the largest sum of parameter similarities among the multiple parameter matching methods based on the maximum a posteriori probability estimation algorithm to obtain the target parameter matching method, which uses the a posteriori probability as the parameter similarity. By maximizing the sum of the a posteriori probabilities in each parameter matching method, the optimal target parameter matching method can be accurately obtained. This process provides a statistical consistency method to measure and compare the parameters of different local models, quantifies the similarity between different model parameters through probabilistic language, and can take into account the uncertainty and variability of the parameters, which can improve the accuracy of parameter similarity, thereby improving the accuracy of the target parameter matching method, and then improving the subsequent parameter aggregation effect.

[0113] In one embodiment, in step S12, based on the maximum a posteriori probability estimation algorithm, a parameter matching method with the largest sum of parameter similarities among multiple parameter matching methods is determined to obtain a target parameter matching method, which specifically includes the following steps:

[0114] S121: The server determines the posterior probability of each global neuron parameter in each parameter matching method respectively, and obtains multiple parameter similarities in each parameter matching method.

[0115] After obtaining a plurality of parameter matching methods, the server needs to respectively determine the posterior probability of each global neuron parameter in each parameter matching method to obtain a plurality of parameter similarities in each parameter matching method.

[0116] Specifically, for each global neuron parameter in the parameter matching method, the server uses a Bayesian algorithm to analyze the posterior probability of the global neuron parameter when a local neuron parameter matching it (global neuron parameter) is observed based on the set prior distribution and likelihood function, and obtains the parameter similarity corresponding to the global neuron parameter; then, by traversing all parameter matching methods, multiple parameter similarities in each parameter matching method can be determined.

[0117] S122: The server sums up the multiple parameter similarities in each parameter matching method to obtain the sum of the parameter similarities of each parameter matching method.

[0118] After obtaining multiple parameter similarities in each parameter matching method, the server sums the parameter similarities corresponding to all global neuron parameters in the parameter matching method to obtain the sum of parameter similarities of the parameter matching method, and then traverses all parameter matching methods to obtain the sum of parameter similarities of all parameter matching methods.

[0119] S123: The server obtains a target parameter matching method based on the parameter matching method with the largest total parameter similarity.

[0120] Finally, the server obtains the target parameter matching method based on the parameter matching method with the largest sum of parameter similarities. For example, the parameter matching method with the largest sum of parameter similarities can be directly used as the target parameter matching method.

[0121] In this embodiment, the server obtains multiple parameter similarities in each parameter matching method by respectively determining the posterior probability of each global neuron parameter in each parameter matching method, and then respectively sums the multiple parameter similarities in each parameter matching method to obtain the sum of the parameter similarities of each parameter matching method, and then obtains the target parameter matching method based on the parameter matching method with the largest sum of parameter similarities. The specific steps of determining the target parameter matching method based on the maximum a posteriori probability estimation algorithm are clarified. Determining the target parameter matching method in this way can improve the accuracy of parameter similarity, thereby improving the accuracy of the target parameter matching method, and then improving the subsequent parameter aggregation effect.

[0122] In one embodiment, in step S123, i.e., based on the parameter matching method with the largest sum of parameter similarities, a target parameter matching method is obtained, which specifically includes the following steps:

[0123] S1231: The server determines a parameter matching method with the largest sum of multi-parameter similarities, and obtains a pending parameter matching method.

[0124] After obtaining the sum of parameter similarities of all parameter matching methods, the server determines the parameter matching method with the largest sum of parameter similarities, and obtains the pending parameter matching method.

[0125] S1232: When the server determines that there is a bad match in the pending parameter matching method, the server marks the bad match in the pending parameter matching method to obtain the target parameter matching method.

[0126] Then, the server determines whether there is a bad match in the pending parameter matching method, wherein a bad match refers to a neuron matching group whose parameter similarity in the pending parameter matching method is less than a preset threshold. When determining that there is a bad match in the pending parameter matching method, the server needs to mark the bad match in the pending parameter matching method to obtain the target parameter matching method.

[0127] It should be understood that in practice, not all local neuron parameters may be able to match global neuron parameters well. In this case, these poor matches are marked in the process of determining the target parameter matching method. When performing subsequent parameter aggregation, the poor matches in each target parameter matching method are eliminated, that is, the local neuron parameters in the poor matches do not participate in the aggregation, which can further improve the parameter aggregation effect.

[0128] In another embodiment, when the server determines that there is a bad match in the pending parameter matching method, the server does not need to mark the bad match in the pending parameter matching method, but creates a new neuron parameter to replace the local neuron parameter in the bad match, so that the local neuron parameter in the bad match does not participate in aggregation, but the new neuron parameter participates in aggregation, which can further improve the parameter aggregation effect.

[0129] In this embodiment, the server obtains a pending parameter matching method by determining a parameter matching method with the largest sum of multi-parameter similarities, and then when it is determined that there is a bad match in the pending parameter matching method, the server marks the bad match in the pending parameter matching method to obtain a target parameter matching method, so that when parameters are subsequently aggregated based on the target parameter matching method, the bad matches can be screened out, which can improve the parameter aggregation effect and thus improve the model accuracy.

[0130] In one embodiment, if Figure 6 As shown, in step S20, the parameters of multiple local neuron parameters of each local model in the target network layer are aggregated to update the target network layer of the global model, which specifically includes the following steps:

[0131] S21: The server determines, for each global neuron parameter, a plurality of local neuron parameters that match the global neuron parameter according to each target parameter matching method.

[0132] After obtaining the target parameter matching methods of the target network layers of all local models, for each global neuron parameter in the current global model, the server determines multiple local neuron parameters that match the global neuron parameter according to each target parameter matching method. Then, the server needs to determine whether there are marked bad matches in each target parameter matching method. Among them, a bad match refers to a neuron matching group whose parameter similarity in the undetermined parameter matching method is less than a preset threshold.

[0133] S22: When there is a marked bad match in any target parameter matching method, the server removes the local neuron parameters in the bad match from multiple local neuron parameters that match the global neuron parameters, and optimizes to obtain multiple local neuron parameters.

[0134] When there is a marked bad match in any target parameter matching mode, the server removes the local neuron parameters in the bad match from multiple local neuron parameters that match the global neuron parameter, and optimizes to obtain multiple local neuron parameters.

[0135] S23: The server aggregates the optimized multiple local neuron parameters to obtain aggregated neuron parameters of global neuron parameters, and updates the aggregated neuron parameters of each global neuron parameter to the target network layer of the global model.

[0136] Then, the server aggregates the optimized multiple local neuron parameters, averages or weighted averages the optimized multiple local neuron parameters to obtain the aggregated neuron parameters of the global neuron parameters; traverses all global neuron parameters to obtain the aggregated neuron parameters of all global neuron parameters, and updates the aggregated neuron parameters of each global neuron parameter to the target network layer of the global model.

[0137] In this embodiment, for each global neuron parameter, multiple local neuron parameters that match the global neuron parameter are determined according to each target parameter matching method; when there is a marked bad match in any target parameter matching method, it means that there are local neuron parameters with poor matching effect (i.e., small parameter similarity), and the local neuron parameters in the bad match are eliminated from the multiple local neuron parameters that match the global neuron parameter, and multiple local neuron parameters are optimized; the multiple optimized local neuron parameters are aggregated to obtain the aggregated neuron parameters of the global neuron parameters, and the aggregated neuron parameters of each global neuron parameter are updated to the target network layer of the global model, clarifying the update process of the target network layer. When the target network layer of the global model is aggregated and updated based on the target parameter matching method corresponding to each local model, the local neuron parameters matched by each global neuron parameter are screened, and the local neuron parameters with poor matching effect are eliminated, which can improve the parameter aggregation effect and thus improve the model accuracy.

[0138] In one embodiment, after step S21, that is, after determining a plurality of local neuron parameters matching the global neuron parameters, the following steps are specifically included:

[0139] S24: When there is no marked bad match in all target parameter matching methods, the server directly aggregates all local neuron parameters that match the global neuron parameters to obtain an aggregated neuron parameter of the global neuron parameter;

[0140] S25: The server updates the aggregated neuron parameters of each global neuron parameter to the target network layer of the global model.

[0141] After determining multiple local neuron parameters that match the global neuron parameters, when there are no marked bad matches in all target parameter matching methods, it means that the matching effects of all parameters meet the requirements. The server directly aggregates all local neuron parameters that match the global neuron parameters to obtain the aggregated neuron parameters of the global neuron parameters, and updates the aggregated neuron parameters of each global neuron parameter to the target network layer of the global model.

[0142] In this embodiment, after determining multiple local neuron parameters that match the global neuron parameters, when all target parameter matching methods do not have marked bad matches, all local neuron parameters that match the global neuron parameters are directly aggregated to obtain the aggregated neuron parameters of the global neuron parameters, and the aggregated neuron parameters of each global neuron parameter are updated to the target network layer of the global model, which is simple and intuitive.

[0143] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0144] In one embodiment, a server is provided, wherein the parameter aggregation device corresponds one-to-one to the parameter aggregation method based on federated learning in the above embodiment. Figure 7 As shown, the parameter aggregation device includes a matching module 701, an aggregation module 702 and a communication module 703. The detailed description of each functional module is as follows:

[0145] Matching module 701 is used to solve the optimal matching relationship based on parameter similarity between the multiple local neuron parameters and the multiple global neuron parameters of the global model on the server at the same target network layer after obtaining the multiple local neuron parameters of the local model on any client at the target network layer, so as to obtain the target parameter matching mode of the target network layer in the local model;

[0146] Aggregation module 702, for, after obtaining the target parameter matching modes of all local models, performing parameter aggregation on multiple local neuron parameters of each local model in the target network layer according to each target parameter matching mode, and updating the target network layer of the global model;

[0147] The communication module 703 is used to send the updated multiple global neuron parameters of the target network layer to each client.

[0148] In one embodiment, a client is provided, wherein the parameter aggregation device corresponds one-to-one to the parameter aggregation method based on federated learning in the above embodiment. Figure 8 As shown, the parameter aggregation device includes a communication module 801 and a prediction module 802. The detailed description of each functional module is as follows:

[0149] The communication module 801 is used to receive multiple global neuron parameters of the target network layer sent by the server, and update the target network layer in the local model of the client based on the multiple global neuron parameters of the target network layer, where the multiple global neuron parameters are aggregated by the server based on the above-mentioned parameter aggregation method based on federated learning;

[0150] The prediction module 802 is used to output the local model as the target model when the updated local model converges, so as to use the target model to predict the latest local data of the client.

[0151] Optionally, the parameter aggregation device also includes a training module 803. After updating the target network layer in the local model of the client, the training module 803 is used to train the updated local model using the local data of the client when the updated local model has not converged; the communication module 801 is also used to send multiple local neuron parameters of the trained local model in the target network layer to the server for iteration and updating until the updated local model converges.

[0152] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0153] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0154] In one embodiment, a computer device is provided, which may be a server or a client, and includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data used and generated by the above-mentioned parameter aggregation method based on federated learning, such as a global model (or a local model), a target parameter matching method, and related parameters of the updated target network layer. The network interface of the computer device is used to communicate with an external device via a network connection. When the computer program is executed by the processor, a parameter aggregation method based on federated learning is implemented.

[0155] The present application also provides a computer device, such as Fig. 9 As shown, the computer includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above-mentioned method embodiments when executing the computer program, or implements the functions of the modules / units in the above-mentioned device embodiments when executing the computer program.

[0156] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the computer device.

[0157] Those skilled in the art will understand that Fig. 9 These are merely examples of the computer device and do not constitute a limitation of the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0158] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0159] The memory may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The memory may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Furthermore, the memory may also include both an internal storage unit of the computer device and an external storage device.

[0160] An embodiment of the present application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0161] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0162] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0163] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0165] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0166] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A parameter aggregation method based on federated learning, characterized in that: include: After obtaining multiple local neuron parameters of the local model on any client at the target network layer, the multiple local neuron parameters and the multiple global neuron parameters of the global model on the server at the same target network layer are solved based on parameter similarity to obtain the target parameter matching method of the target network layer in the local model, including: Randomly matching the global neuron parameters of the same target network layer with the local neuron parameters respectively to obtain multiple parameter matching methods, wherein in the parameter matching methods, each global neuron matches only one local neuron; Based on the maximum a posteriori probability estimation algorithm, the parameter matching method with the largest sum of parameter similarities among the multiple parameter matching methods is determined to obtain the target parameter matching method, wherein each parameter similarity in the parameter matching method is the posterior probability of the global neuron parameter that matches the local neuron parameter when the local neuron parameter is observed; After obtaining the target parameter matching modes of all the local models, performing parameter aggregation on multiple local neuron parameters of each local model in the target network layer according to each target parameter matching mode, and updating the target network layer of the global model; The updated multiple global neuron parameters of the target network layer are sent to each of the clients.

2. The parameter aggregation method based on federated learning according to claim 1, characterized in that: The method of determining the parameter matching method with the largest sum of parameter similarities among the plurality of parameter matching methods based on the maximum a posteriori probability estimation algorithm to obtain the target parameter matching method includes: Respectively determining the posterior probability of each of the global neuron parameters in each of the parameter matching modes to obtain a plurality of parameter similarities in each of the parameter matching modes; Respectively summing up the plurality of parameter similarities in each of the parameter matching modes to obtain the sum of the parameter similarities of each of the parameter matching modes; The target parameter matching method is obtained based on the parameter matching method with the largest sum of parameter similarities.

3. The parameter aggregation method based on federated learning according to claim 2, characterized in that: The parameter matching method based on the maximum sum of the parameter similarities, obtaining the target parameter matching method, comprises: Determine the parameter matching method with the largest sum of multiple parameter similarities to obtain a pending parameter matching method; When it is determined that there is a bad match in the pending parameter matching method, the bad match in the pending parameter matching method is marked to obtain the target parameter matching method, and the bad match refers to a neuron matching group in which the parameter similarity in the pending parameter matching method is less than a preset threshold.

4. The parameter aggregation method based on federated learning according to any one of claims 1 to 3, characterized in that: The step of aggregating parameters of multiple local neuron parameters of each local model in the target network layer to update the target network layer of the global model includes: For each of the global neuron parameters, determining a plurality of local neuron parameters matching the global neuron parameter according to each of the target parameter matching modes; When there is a marked bad match in any of the target parameter matching modes, the local neuron parameters in the bad match are eliminated from the multiple local neuron parameters that match the global neuron parameters, and the multiple local neuron parameters are optimized; The optimized multiple local neuron parameters are aggregated to obtain the aggregated neuron parameters of the global neuron parameters, and the aggregated neuron parameters of each global neuron parameter are updated to the target network layer of the global model.

5. A parameter aggregation method based on federated learning, characterized in that: include: Receiving multiple global neuron parameters of the target network layer sent by the server, and updating the target network layer in the local model of the client based on the multiple global neuron parameters of the target network layer, wherein the multiple global neuron parameters are aggregated by the server based on the parameter aggregation method based on federated learning according to any one of claims 1 to 4; When the updated local model converges, the current local model is output as the target model, so as to use the target model to predict the latest local data of the client.

6. A federated learning system, characterized in that: The system comprises a server and a plurality of clients, wherein the server is used for: After obtaining a plurality of local neuron parameters of the local model on any of the clients in the target network layer, solving the optimal matching relationship based on parameter similarity between the plurality of local neuron parameters and the plurality of global neuron parameters of the global model on the server in the same target network layer, and obtaining a target parameter matching method for the target network layer in the local model, including: Randomly matching the global neuron parameters of the same target network layer with the local neuron parameters respectively to obtain multiple parameter matching methods, wherein in the parameter matching methods, each global neuron matches only one local neuron; Based on the maximum a posteriori probability estimation algorithm, the parameter matching method with the largest sum of parameter similarities among the multiple parameter matching methods is determined to obtain the target parameter matching method, wherein each parameter similarity in the parameter matching method is the posterior probability of the global neuron parameter that matches the local neuron parameter when the local neuron parameter is observed; After obtaining the target parameter matching modes of all the local models, performing parameter aggregation on multiple local neuron parameters of each local model in the target network layer according to each target parameter matching mode, and updating the target network layer of the global model; The updated multiple global neuron parameters of the target network layer are sent to each of the clients.

7. The federated learning system according to claim 6, wherein: Each of the clients is used for: Receiving a plurality of global neuron parameters of the target network layer sent by the server, and updating the target network layer in the local model of the client based on the plurality of global neuron parameters of the target network layer; When the updated local model converges, the current local model is output as the target model, so as to use the target model to predict the latest local data of the client.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that:When the processor executes the computer program, the steps of the parameter aggregation method based on federated learning as described in any one of claims 1 to 5 are implemented.

9. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the parameter aggregation method based on federated learning as described in any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Edge-based federated learning model cleaning and equipment clustering method, system and equipment and readable storage medium

    CN112181971A

  • Traffic prediction method, device and equipment and computer readable storage medium

    CN114139782A