Federated learning system optimization method, device, equipment and storage medium

By training the full model in federated learning, combining federated models and local models, the problem of resource waste in federated learning is solved, model performance and resource utilization are improved, and better data feature learning and model application are achieved.

CN111222647BActive Publication Date: 2024-11-15WEBANK (CHINA)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010022561.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-01-09
Publication Date
2024-11-15
Estimated Expiration
2040-01-09

AI Technical Summary

Technical Problem

In federated learning, due to the different distribution of the training data characteristics that participants have, the performance of the federated model is not necessarily better than that of the local model alone, resulting in participants choosing to use the local model and abandoning the federated model, resulting in waste of resources and reducing the utilization of computing resources. Rate.

Method used

By using the preset training dataset to train a full model including federated models, local models and fusion models, and perform performance testing, select the final model to make full use of the federated model and improve model performance.

Benefits of technology

Improve model performance, avoid resource waste, enhance computing resource utilization, and enable the entire model to better learn the data characteristics of participating devices and expand the scope of model application.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111222647B_ABST
    Figure CN111222647B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, device, equipment and storage medium for optimizing a federated learning system, the method comprising: participating in the training of a federated learning model to obtain a federated model; using a preset training data set and the federated model training to obtain a local model and a full model, wherein the full model at least comprises three parts: the federated model, the local model and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model; performing performance tests on the federated model, the local model and the full model, and selecting the model to be finally used based on the performance test results. The present invention makes full use of the federated model without wasting the local computing resources of the participating devices, and because the federated model and the local model are combined, the performance of the obtained full model is likely to be better than that of the local model, so that most participating devices can benefit from federated learning and improve model performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a method, device, equipment and storage medium for optimizing a federated learning system. Background Art

[0002] At present, in order to protect the data privacy and security of participants in the joint training model, the concept of federated learning has been proposed. Federated learning refers to the method of building a shared machine learning model by uniting different participants for machine learning or deep learning. In federated learning, participants do not need to expose their own data to other participants and coordinators, so federated learning can well protect user privacy and ensure data security.

[0003] However, in the actual application of federated learning, the data feature distribution of the training data owned by different participants is generally different. For example, one participant A may often use the sentence "I eat at nine o'clock in the evening", while another participant B may often use the sentence "I read at nine o'clock in the evening". When training the next input word prediction model, "I _______ at nine o'clock in the evening", the model obtained by participants A and B through federated learning may not have a good prediction effect, and cannot distinguish whether it is predicting "eating" or "reading". If participant A trains the model alone, it can predict more accurately and always predict "eating". In other words, due to the different data feature distribution of the training data owned by the participants, the model jointly trained by participants A and B through federated learning does not always perform better than the machine learning model trained by participant A or B alone. Then, when the performance of the local model trained by the participant locally is better than the federated model, the participant chooses to use the local model and abandons the federated model, which will cause resource waste, thereby reducing the utilization rate of the participant's local computing resources. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and storage medium for optimizing a federated learning system, aiming to solve the technical problem that when the performance of a local model obtained by local training of a participant is better than that of a federated model, the participant chooses to use the local model and abandons the federated model, which will cause a waste of resources and thus reduce the utilization rate of the participant's local computing resources.

[0005] To achieve the above object, the present invention provides a method for optimizing a federated learning system, which is applied to participating devices participating in federated learning. The method for optimizing a federated learning system includes:

[0006] Participate in the federated learning model training to obtain the federated model;

[0007] A local model and a full model are obtained by training a preset training data set and the federated model, wherein the full model comprises at least three parts: the federated model, the local model and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model;

[0008] The federal model, the local model and the full model are subjected to performance tests, and a model to be finally used is selected based on the performance test results.

[0009] Optionally, the step of using a preset training data set and the federated model to train a local model and a full model includes:

[0010] The local model to be trained is trained using a preset training data set to obtain a local model;

[0011] The data in the preset training data set is used as the input of the federal model and the local model, the output of the federal model and the local model is used as the input of the fusion model to be trained, and the fusion model to be trained is trained to obtain the full model.

[0012] Optionally, the step of using a preset training data set and the federated model to train a local model and a full model includes:

[0013] The data in the preset training data set is used as the input of the federal model and the local model to be trained, and the output of the federal model and the local model to be trained is used as the input of the fusion model to be trained. The local model to be trained and the fusion model to be trained are trained to obtain the local model and the full model.

[0014] Optionally, after the step of performing performance tests on the federated model, the local model and the full model and selecting a model to be finally used based on the performance test results, the step further includes:

[0015] When the performance test result shows that the local model has the best performance, stop participating in federated learning within a preset time period.

[0016] Optionally, after the step of performing performance tests on the federated model, the local model and the full model and selecting a model to be finally used based on the performance test results, the step further includes:

[0017] The performance test result is sent to the coordinating device participating in the federated learning, so that the coordinating device stops sending the federated learning invitation to the participating device within a preset time period when the performance test result shows that the local model has the best performance.

[0018] Optionally, when the performance test result shows that the full model has the best performance and the full model is used to classify the image, after the step of performing performance tests on the federated model, the local model and the full model and selecting the model to be finally used based on the performance test result, the method further includes:

[0019] Inputting image data of the image to be classified into the federal model and the local model in the full model respectively, obtaining a first logit number output by the federal model and a second logit number output by the local model, wherein the first logit number and the second logit number represent the probability that the image to be classified belongs to each image category;

[0020] The first logit number and the second logit number are fused and input into the fusion model in the full model to obtain the classification result of the image to be classified output by the fusion model.

[0021] Optionally, the number of the federal models is at least one, and the number of the local models is at least one.

[0022] In addition, to achieve the above-mentioned purpose, the present invention further provides a federated learning system optimization device, which is deployed on a participating device participating in federated learning, and the federated learning system optimization device includes:

[0023] The federated learning module is used to participate in the federated learning model training to obtain the federated model;

[0024] A training module, used for training a local model and a full model using a preset training data set and the federated model, wherein the full model comprises at least three parts: the federated model, the local model and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model;

[0025] The testing module is used to perform performance testing on the federal model, the local model and the full model, and select the model to be finally used based on the performance testing results.

[0026] In addition, to achieve the above-mentioned purpose, the present invention also provides a federated learning system optimization device, which includes a memory, a processor, and a federated learning system optimization program stored on the memory and executable on the processor, and the federated learning system optimization program, when executed by the processor, implements the steps of the federated learning system optimization method as described above.

[0027] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a federated learning system optimization program is stored. When the federated learning system optimization program is executed by a processor, the steps of the federated learning system optimization method as described above are implemented.

[0028] In the present invention, a federated model is obtained by having participating devices participate in the training of a federated learning model; a local model and a full model including the federated model, the local model and the fusion model are obtained by using a preset training data set and the federated model for training; a performance test is performed on the federated model, the local model and the full model, and the model to be finally used is selected based on the performance test results. That is, the participating devices train the full model by combining the federated model and the local model, so that the federated model can be fully utilized without wasting resources, and because the federated model and the local model are combined, the performance of the obtained full model is likely to be better than that of the local model, so that most participating devices can benefit from federated learning and improve the model performance; and the full model can fully learn the data characteristics of the training data of the participating devices, conduct targeted training on the data owned by the participating devices, and can flexibly select the model structure, so that the trained model has a wider range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;

[0030] Figure 2 This is a flow chart of the first embodiment of the method for optimizing a federated learning system of the present invention;

[0031] Figure 3 A schematic diagram of the structure of a full model involved in an embodiment of the present invention;

[0032] Figure 4 It is a structural schematic diagram of another full model involved in an embodiment of the present invention;

[0033] Figure 5 A schematic diagram of a process of performing federated learning on a participating device according to an embodiment of the present invention;

[0034] Figure 6 This is a functional schematic module diagram of a preferred embodiment of the federated learning system optimization device of the present invention.

[0035] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0037] The present invention provides a federated learning system optimization device, referring to Figure 1 , Figure 1 It is a structural diagram of the hardware operating environment involved in the embodiment of the present invention.

[0038] It should be noted that Figure 1 The schematic diagram of the hardware operating environment of the federated learning system optimization device is shown in FIG. The federated learning system optimization device of the embodiment of the present invention may be a PC, or a terminal device with a display function such as a smart phone, a smart TV, a tablet computer, or a portable computer.

[0039] like Figure 1 As shown, the federated learning system optimization device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0040] Optionally, the federated learning system optimization device may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. Those skilled in the art will appreciate that Figure 1 The structure of the federated learning system optimization device shown in the figure does not constitute a limitation of the federated learning system optimization device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0041] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a federated learning system optimization program.

[0042] exist Figure 1 In the federated learning system optimization device shown, the network interface 1004 is mainly used to connect to other participating devices or coordinating devices participating in the federated learning, and to communicate data with other participating devices or coordinating devices; the user interface 1003 is mainly used to connect to the client (user end) and to communicate data with the client; and the processor 1001 can be used to call the federated learning system optimization program stored in the memory 1005, and perform the following operations:

[0043] Participate in the federated learning model training to obtain the federated model;

[0044] A local model and a full model are obtained by training a preset training data set and the federated model, wherein the full model comprises at least three parts: the federated model, the local model and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model;

[0045] The federal model, the local model and the full model are subjected to performance tests, and a model to be finally used is selected based on the performance test results.

[0046] Furthermore, the step of using the preset training data set and the federated model to train a local model and a full model includes:

[0047] The local model to be trained is trained using a preset training data set to obtain a local model;

[0048] The data in the preset training data set is used as the input of the federal model and the local model, the output of the federal model and the local model is used as the input of the fusion model to be trained, and the fusion model to be trained is trained to obtain the full model.

[0049] Furthermore, the step of using the preset training data set and the federated model to train a local model and a full model includes:

[0050] The data in the preset training data set is used as the input of the federal model and the local model to be trained, and the output of the federal model and the local model to be trained is used as the input of the fusion model to be trained. The local model to be trained and the fusion model to be trained are trained to obtain the local model and the full model.

[0051] Furthermore, after the step of performing performance tests on the federated model, the local model and the full model and selecting the model to be finally used based on the performance test results, the step further includes:

[0052] When the performance test result shows that the local model has the best performance, stop participating in federated learning within a preset time period.

[0053] Furthermore, after the step of performing performance tests on the federated model, the local model and the full model and selecting the model to be finally used based on the performance test results, the step further includes:

[0054] The performance test result is sent to the coordinating device participating in the federated learning, so that the coordinating device stops sending the federated learning invitation to the participating device within a preset time period when the performance test result shows that the local model has the best performance.

[0055] Furthermore, when the performance test result shows that the full model has the best performance and the full model is used to classify the image, after the step of performing performance tests on the federated model, the local model and the full model and selecting the model to be finally used based on the performance test result, the method further includes:

[0056] Inputting image data of the image to be classified into the federal model and the local model in the full model respectively, obtaining a first logit number output by the federal model and a second logit number output by the local model, wherein the first logit number and the second logit number represent the probability that the image to be classified belongs to each image category;

[0057] The first logit number and the second logit number are fused and input into the fusion model in the full model to obtain the classification result of the image to be classified output by the fusion model.

[0058] Furthermore, the number of the federal models is at least one, and the number of the local models is at least one.

[0059] Based on the above hardware structure, various embodiments of the federated learning system optimization method of the present invention are proposed.

[0060] Reference Figure 2 The first embodiment of the method for optimizing a federated learning system of the present invention provides a method for optimizing a federated learning system. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here. The method for optimizing a federated learning system of the present invention is applied to participating devices participating in federated learning, and the participating devices may be terminal devices such as PCs, smart phones, smart TVs, tablet computers, and portable computers. The method for optimizing a federated learning system includes:

[0061] Step S10, participating in the federated learning model training to obtain a federated model;

[0062] Federated learning refers to a method of combining different participants to perform machine learning or deep learning in order to build a shared machine learning model. A global model parameter update in federated learning is divided into two steps: (a) Each participant (also known as the data owner or client) uses only the data they own to train the machine learning model and obtain the model parameter update (model weight update), and sends the model parameter update to a coordinator (also known as a parameter server or aggregation server). For example, the model parameter update can be model weights or gradient information; (b) The coordinator fuses the model parameter updates received from different participants, for example, calculates the weighted average, and distributes the model parameter update obtained after fusion (also known as the global model parameter update) to each participant. In federated learning, participants do not need to expose their own data to other participants and coordinators, so federated learning can well protect user privacy and ensure data security. The core idea of ​​federated learning is "data does not move, machine learning model parameters move". Federated learning can also be done without the participation of a coordinator.

[0063] In this embodiment, based on a learning task, the participating device jointly trains the federated learning model with other participating devices, or with other participating devices in cooperation with the coordinating device, to obtain a federated model. The federated model can be a machine learning model such as a deep neural network or a convolutional neural network. Depending on the learning task, the trained federated model can complete different tasks. For example, when the learning task is to classify an image, the local training data used by the participating device is image data, and the trained federated model is used to classify the image to be classified.

[0064] Step S20, using a preset training data set and the federated model to train to obtain a local model and a full model, wherein the full model at least includes three parts: the federated model, the local model and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model;

[0065] Based on the same learning task as training the federated model, the participating devices use the preset training data set and the trained federated model to train the local model and the full model. Among them, the preset training data set is the data set corresponding to the learning task, and the data set includes multiple sample data, each sample data includes feature data and data labels. For example, when the learning task is to classify images, each sample data may include an image data, and the classification label corresponding to the image data. The classification label is the label that marks which category the image data belongs to. The preset training data set can be the same as the training data set used to train the federated model, or it can be different, but the distribution of data features should be roughly the same. For example, the color features in the image data are mostly "green".

[0066] The local model refers to the model obtained by the participating device training alone using the local preset training data set. That is, unlike the training of the federal model which requires joint training of other participating devices, the local model does not rely on the training data in other participating devices. Since the learning tasks are the same, the local model and the federal model can complete the same tasks, such as both are used to classify images. The local model can also be a machine learning model such as a deep neural network or a convolutional neural network. The structure of the local model may be the same as or different from that of the federal model, and is not limited here.

[0067] The full model refers to a model that includes at least the federated model, the local model, and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model, that is, the fusion model is used to fuse the output data of the federated model and the local model. The fusion model can be any one of a linear regression model (with lower computational complexity), a logistic regression model, a support vector machine model, and a neural network model. Figure 3 This is a schematic diagram of one of the structures of the full model, where M is the federation model and L A is a local model trained with the local training dataset A, Q A It uses the local training data set A combined with M and L A The trained fusion model, N A It's the full model.

[0068] The training process of the local model and the full model can be synchronous or asynchronous, that is, the participating devices can first train separately to obtain the local model, and then train the fusion model part of the full model separately, or they can train the local model and the fusion model part together.

[0069] Step S30, performing performance tests on the federated model, the local model and the full model, and selecting a model to be finally used based on the performance test results.

[0070] The participating devices perform performance tests on the federal model, local model, and full model. Specifically, the participating devices can use local test data sets to perform performance tests on the federal model, local model, and full model respectively. The performance test can be to calculate relevant model performance indicators, such as the accuracy of image classification. The calculation method of the model performance indicators can adopt the existing calculation method, which will not be described in detail here. The participating devices select the model to be finally used based on the performance test results of the three models. For example, the performance test results of the three models can be compared, and the model with the best performance can be selected as the model to be finally used. For example, compare the image classification accuracy of the three models, and use the model with the highest accuracy as the final model to be used, that is, use the model with the highest accuracy to complete the subsequent image classification tasks.

[0071] Furthermore, participating devices can also combine a federated model with multiple local models, that is, participating devices can build multiple local models, then combine them with the federated model, and then train to obtain the full model, such as Figure 4 As shown in the figure, it is a schematic diagram of the full model structure combining a federated model and multiple local models, where M is the federated model, L A1 , L A2 ,…L Ai is a local model trained with the local training dataset A, Q A It uses the local training data set A combined with M and L A1 , L A2 ,…L Ai The trained fusion model, N A is the full model. Similarly, participating devices can also obtain the full model by combining K federated models and S local models, where K and S are integers greater than or equal to one.

[0072] Compared with the participating devices only training local models and federated models, if the performance of the local model is better than that of the federated model, the local model is selected to be used and the federated model is abandoned, resulting in a waste of resources and reducing the computing resource utilization of the participating devices. In this embodiment, by combining the federated model and the local model to build a full model, the participating devices can make full use of the federated model without wasting resources, and because the federated model and the local model are combined, the performance of the obtained full model is likely to be better than that of the local model, so that most participating devices can benefit from federated learning and improve model performance. For example, when the model is used to classify images, the accuracy of the model in image classification is improved, and the computing resource utilization of the participating devices is also improved.

[0073] Furthermore, compared to the scheme in which the participating devices train the federal model and fine-tune the federal model based on local training data to obtain the local model, so that the finally obtained local model is limited by the structure of the federal model and cannot flexibly select the model structure according to the data characteristics of the data set owned by the participating devices, in this embodiment, the full model is trained by combining the federal model and the local model, so that the full model can fully learn the data characteristics of the training data of the participating devices, perform targeted training on the data owned by the participating devices, and can flexibly select the model structure, so that the trained model has a wider range of applications.

[0074] In this embodiment, a federated model is obtained by having participating devices participate in the training of a federated learning model; a local model is obtained by using a preset training data set and the federated model for training, and a full model including the federated model, the local model and the fusion model is obtained; the performance of the federated model, the local model and the full model is tested and compared, and the model to be finally used is selected based on the performance test results. That is, the participating devices train the full model by combining the federated model and the local model, so that the federated model can be fully utilized without wasting resources, and because the federated model and the local model are combined, the performance of the obtained full model is likely to be better than that of the local model, so that most participating devices can benefit from federated learning and improve model performance; and the full model can fully learn the data characteristics of the training data of the participating devices, conduct targeted training on the data owned by the participating devices, and can flexibly select the model structure, so that the trained model has a wider range of applications.

[0075] Further, based on the above first embodiment, the second embodiment of the method for optimizing a federated learning system of the present invention provides a method for optimizing a federated learning system. In this embodiment, step S20 includes:

[0076] Step A10, using a preset training data set to train the local model to be trained to obtain a local model;

[0077] In this embodiment, a method for training a local model and a full model is proposed: the participating devices are first trained separately to obtain a local model, and then the fusion model part of the full model is trained separately.

[0078] Specifically, the participating devices use a preset training data set to train the local model to be trained independently to obtain a trained local model. The local model to be trained refers to a model that has not yet been optimized in parameters, such as a neural network structure that is preset, but the weight parameters in the neural network are initialized parameters; the participating devices use a preset training data set to train the local model to be trained, and specifically, the parameters of the local model to be trained can be iteratively updated so that the classification or prediction results of the local model to be trained are close to the real data labels, and the training is stopped until a certain stopping condition is met, such as when the error between the classification or prediction results and the real data labels (such as the loss function value) is less than a certain threshold, the training can be stopped to obtain the final model parameters, and the trained local model is obtained. The same applies to the fusion model to be trained mentioned later.

[0079] Step A20, using the data in the preset training data set as the input of the federal model and the local model, using the output of the federal model and the local model as the input of the fusion model to be trained, and training the fusion model to be trained to obtain the full model.

[0080] After the local model is trained, the participating devices use the feature data in the preset training data set as the input of the federated model and the local model, and use the output of the federated model and the local model as the input of the fusion model to be trained, and train the fusion model to be trained. During the training process, the federated model and the local model are kept unchanged. Finally, the trained fusion model is obtained, and then the full model including the federated model, the local model and the fusion model is obtained.

[0081] It should be noted that when the learning task is a classification task, such as classifying images, the output results of the federated model and the local model when used separately can be classification labels, that is, labels that represent which category the input data belongs to. When the federated model and the local model are used as part of the full model, the output results of the federated model and the local model can be logits, and the logits output by the two models are fused, and the fusion result is used as the input of the fusion model. Among them, the logits represent the probability that the input data can be classified into each category.

[0082] Furthermore, a flexible approach can be adopted to fuse the outputs of the federated model and the local model, for example, the outputs of the federated model and the local model are concatenated together in the form of vectors, or weighted average and the like.

[0083] Furthermore, the step S20 includes:

[0084] Step B10, using the data in the preset training data set as the input of the federal model and the local model to be trained, using the output of the federal model and the local model to be trained as the input of the fusion model to be trained, training the local model to be trained and the fusion model to be trained to obtain the local model and the full model.

[0085] Furthermore, in one embodiment, another method for training the local model and the full model is proposed: the participating devices simultaneously train the local model and the fusion model part.

[0086] Specifically, the participating devices use the feature data in the preset training data set as the input of the federated model and the local model to be trained, and use the output of the federated model and the local model to be trained as the input of the fusion model to be trained, and train the local model to be trained and the fusion model to be trained simultaneously, that is, in each round of parameter update, the model parameters of the local model to be trained and the model parameters of the fusion model to be trained are updated. After the training is completed, the trained local model and fusion model are obtained, and then the full model including the federated model, the local model and the fusion model is obtained.

[0087] In this embodiment, participating devices train the full model by combining the federal model and the local model, so that the federal model can be fully utilized without wasting resources. Moreover, because the federal model and the local model are combined, the performance of the obtained full model is likely to be better than that of the local model, so that most participating devices can benefit from federated learning and improve model performance; and the full model can fully learn the data characteristics of the training data of the participating devices, perform targeted training on the data owned by the participating devices, and can flexibly select the model structure, so that the trained model has a wider range of applications.

[0088] Further, based on the above-mentioned first and second embodiments, the third embodiment of the method for optimizing a federated learning system of the present invention provides a method for optimizing a federated learning system. In this embodiment, after step S30, the method further includes:

[0089] Step S40: When the performance test result shows that the local model has the best performance, stop participating in federated learning within a preset time period.

[0090] After the participating device obtains the performance test results, if the performance test results show that the performance of the local model is the best, the participating device can choose to stop participating in federated learning within a preset time period. The preset time period can be a time period pre-set by the participating device. If the performance of the federated model obtained by the participating device through participating in federated learning, and the performance of the full model trained by combining the federated model and the local model are not as good as the performance of the local model, it means that federated learning has not brought improvement to the model performance of the participating device. The participating device can choose to stop participating in federated learning within the preset time period to save various local resources including the computing resources of the participating device.

[0091] Furthermore, after step S30, the method further includes:

[0092] Step S50, sending the performance test result to the coordination device participating in the federated learning, so that the coordination device stops sending the federated learning invitation to the participating device within a preset time period when the performance test result shows that the local model has the best performance.

[0093] Furthermore, the participating devices may also send the performance test results to the coordinating devices participating in the federated learning. When the coordinating device detects that the performance test results show that the local model has the best performance, the coordinating device may stop sending federated learning invitations to the participating devices within a preset time period, so that the participating devices do not need to participate in the federated learning within the preset time period, saving various local resources including the computing resources of the participating devices, and also reducing the complexity of the federated learning system. Among them, the federated learning invitation can be a message sent by the coordinating device to the participating devices, which is used to notify the participating devices to participate in the federated learning.

[0094] Reference Figure 5 As shown, it is a schematic diagram of a process of participating devices performing federated learning involved in an embodiment, wherein the participants are the participating devices and the coordinator is the coordinating device.

[0095] Furthermore, when the performance test result shows that the performance of the full model is optimal and the full model is used to classify images, after step S30, the following further includes:

[0096] Step S60, inputting the image data of the image to be classified into the federal model and the local model in the full model respectively, obtaining a first logit number output by the federal model and a second logit number output by the local model, wherein the first logit number and the second logit number represent the probability that the image to be classified belongs to each image category;

[0097] When the participating device detects that the performance test result shows that the performance of the full model is optimal, the participating device can choose to use the full model. When the learning task is to classify images, the trained full model is used to classify images. For images to be classified, the participating device inputs the image data of the images to be classified into the federal model and the local model in the full model respectively. The federal model outputs the first logit and the local model outputs the second logit. The first logit and the second logit represent the probability that the image belongs to each image category. Due to the differences between the federal model and the local model, the first logit and the second logit may be different.

[0098] Step S70, fusing the first logit number and the second logit number and inputting the resultant to the fusion model in the full model to obtain a classification result of the image to be classified output by the fusion model.

[0099] The participating devices fuse the first logit and the second logit in the same way as the output of the federated model and the local model when training the fusion model. The fusion result is input into the fusion model of the whole model to obtain the output of the fusion model, that is, the classification result of the image to be classified. At this time, the classification result can be a category label.

[0100] In this embodiment, the participating devices combine the local model and the federated model for training to obtain a full model. When the performance of the full model is better, the full model is used to classify the image to be classified, thereby improving the accuracy of image classification.

[0101] In addition, the embodiment of the present invention also proposes a federated learning system optimization device, which is deployed on a participating device participating in the federated learning, referring to Figure 6 , the federated learning system optimization device comprises:

[0102] A federated learning module 10 is used to participate in the training of a federated learning model to obtain a federated model;

[0103] A training module 20, configured to obtain a local model and a full model by using a preset training data set and the federated model for training, wherein the full model comprises at least three parts: the federated model, the local model and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model;

[0104] The testing module 30 is used to perform performance testing and comparison on the federated model, the local model and the full model, and select the model with the best performance as the model to be finally used.

[0105] Furthermore, the training module 20 includes:

[0106] A first training unit is used to train a local model to be trained using a preset training data set to obtain a local model;

[0107] The second training unit is used to use the data in the preset training data set as the input of the federal model and the local model, use the output of the federal model and the local model as the input of the fusion model to be trained, and train the fusion model to be trained to obtain the full model.

[0108] Furthermore, the training module 20 includes:

[0109] The third training unit is used to use the data in the preset training data set as the input of the federal model and the local model to be trained, use the output of the federal model and the local model to be trained as the input of the fusion model to be trained, train the local model to be trained and the fusion model to be trained, and obtain the local model and the full model.

[0110] Furthermore, after the step of performing performance testing and comparison on the federated model, the local model and the full model and selecting the model with the best performance as the model to be finally used, the method further includes:

[0111] When the performance test result shows that the local model has the best performance, stop participating in federated learning within a preset time period.

[0112] Furthermore, the federated learning system optimization device also includes:

[0113] The sending module is used to send the performance test results to the coordinating device participating in the federated learning, so that the coordinating device stops sending the federated learning invitation to the participating device within a preset time period when the performance test result shows that the local model has the best performance.

[0114] Furthermore, when the performance test result shows that the performance of the full model is optimal and the full model is used to classify images, the federated learning system optimization device further includes:

[0115] An input module, used to input image data of an image to be classified into the federal model and the local model in the full model respectively, to obtain a first logit number output by the federal model and a second logit number output by the local model, wherein the first logit number and the second logit number represent the probability that the image to be classified belongs to each image category;

[0116] A classification module is used to fuse the first logit number and the second logit number and input the resultant into the fusion model in the full model to obtain a classification result of the image to be classified output by the fusion model.

[0117] Furthermore, the number of the federal models is at least one, and the number of the local models is at least one.

[0118] The expanded content of the specific implementation method of the federated learning system optimization device of the present invention is basically the same as the various embodiments of the above-mentioned federated learning system optimization method, and will not be repeated here.

[0119] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which a federated learning system optimization program is stored. When the federated learning system optimization program is executed by a processor, the steps of the above-mentioned federated learning system optimization method are implemented.

[0120] The expanded contents of the specific implementation methods of the federated learning system optimization device and computer-readable storage medium of the present invention are basically the same as the embodiments of the above-mentioned federated learning system optimization method, and will not be elaborated here.

[0121] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0122] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0124] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for optimizing a federated learning system, characterized in that: Applied to a participating device participating in federated learning, the federated learning system optimization method includes: Participate in the federated learning model training to obtain the federated model; A local model and a full model are obtained by training a preset training data set and the federated model, wherein the full model at least includes the federated model, the local model and a fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model; Performing performance tests on the federated model, the local model, and the full model, and selecting a model to be finally used based on the performance test results; When the performance test result shows that the full model has the best performance and the full model is used to classify images, after the step of performing performance tests on the federated model, the local model and the full model and selecting the model to be finally used based on the performance test result, the step further includes: Inputting the image to be classified into the federal model and the local model in the full model respectively, obtaining a first logit number output by the federal model and a second logit number output by the local model, wherein the first logit number and the second logit number represent the probability that the image to be classified belongs to each image category; The first logit number and the second logit number are input into the fusion model in the full model to obtain a classification result of the image to be classified output by the fusion model.

2. The method for optimizing a federated learning system according to claim 1, wherein: The step of using the preset training data set and the federated model to train a local model and a full model comprises: The local model to be trained is trained using a preset training data set to obtain a local model; The data in the preset training data set is used as the input of the federal model and the local model, the output of the federal model and the local model is used as the input of the fusion model to be trained, and the fusion model to be trained is trained to obtain the full model.

3. The method for optimizing a federated learning system according to claim 1, wherein: The step of using the preset training data set and the federated model to train a local model and a full model comprises: The data in the preset training data set is used as the input of the federal model and the local model to be trained, and the output of the federal model and the local model to be trained is used as the input of the fusion model to be trained. The local model to be trained and the fusion model to be trained are trained to obtain the local model and the full model.

4. The method for optimizing a federated learning system according to claim 1, wherein: After the step of performing performance tests on the federated model, the local model and the full model and selecting a model to be finally used based on the performance test results, the method further includes: When the performance test result shows that the local model has the best performance, stop participating in federated learning within a preset time period.

5. The method for optimizing a federated learning system according to claim 1, wherein: After the step of performing performance tests on the federated model, the local model and the full model and selecting a model to be finally used based on the performance test results, the method further includes: The performance test result is sent to the coordinating device participating in the federated learning, so that the coordinating device stops sending the federated learning invitation to the participating device within a preset time period when the performance test result shows that the local model has the best performance.

6. The method for optimizing a federated learning system according to any one of claims 1 to 5, wherein: The number of the federal models is at least one, and the number of the local models is at least one.

7. A federated learning system optimization device, characterized in that: Deployed on a participating device participating in federated learning, the federated learning system optimization device includes: The federated learning module is used to participate in the federated learning model training to obtain the federated model; A training module, used for training a local model and a full model using a preset training data set and the federated model, wherein the full model comprises at least three parts: the federated model, the local model and the fusion model, and the outputs of the federated model and the local model are connected to the input of the fusion model; A testing module, used to perform performance tests on the federated model, the local model and the full model, and select a model to be finally used based on the performance test results; When the performance test result shows that the performance of the full model is optimal and the full model is used to classify images, the federated learning system optimization device further includes: An input module, used to input image data of an image to be classified into the federal model and the local model in the full model respectively, to obtain a first logit number output by the federal model and a second logit number output by the local model, wherein the first logit number and the second logit number represent the probability that the image to be classified belongs to each image category; A classification module is used to input the first logit number and the second logit number into the fusion model in the full model to obtain a classification result of the image to be classified output by the fusion model.

8. A federated learning system optimization device, characterized in that: The federated learning system optimization device includes a memory, a processor, and a federated learning system optimization program stored in the memory and executable on the processor. When the federated learning system optimization program is executed by the processor, the steps of the federated learning system optimization method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a federated learning system optimization program, which, when executed by a processor, implements the steps of the federated learning system optimization method according to any one of claims 1 to 6.