Federal learning method and device based on assembly line mode, server and client
By adopting pipeline mode in federated learning, using the server to randomly select and update global model parameters on edge devices, the local optimal solution problem caused by limited training data of edge devices is solved, and the accuracy and privacy of the model are improved Protect.
Patent Information
- Application Number
- CN202310609877.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
AI Technical Summary
In federated learning, the training data of edge devices is limited, which makes the learning model of each independent device easily fall into the local optimal solution, which in turn affects the accuracy of the model.
Using a federated learning method based on pipeline mode, a server randomly selects an unselected client from the task set as the target client, and sends the preliminary global model parameters to the client for local training, receives and updates the model Parameters until all training tasks are completed, ensuring that each client is trained independently without affecting the accuracy.
Without affecting the accuracy rate, participants are given more freedom and privacy protection, helping each participant to evade local optimal solutions and explore other values, thereby producing a more accurate model.
Smart Images

Figure CN119026703A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of federated learning, and in particular to a pipeline-based federated learning method, device, server and client. Background Art
[0002] Relying on the rapid development of IoT devices, edge devices have certain computing and storage capabilities, which makes the combination of edge devices and machine learning no longer just a hypothesis. Federated learning means "joint learning", which can jointly predict models through collaborative training of multiple edge devices. The training model can be built on the edge device, and each edge device can independently train the learning model locally, and optimize and merge the global model through the server. During the entire federated learning process, private data always exists on the edge device, and data is not shared with other devices, thereby solving problems such as privacy and data security.
[0003] However, in federated learning for edge devices, since any independent edge device has limited training data, the learning model of each independent device is prone to fall into a local optimal solution when trained alone, resulting in poor accuracy of the final model. Summary of the invention
[0004] The present invention aims to solve the technical problems in the related art to a certain extent.
[0005] To this end, the first purpose of the present invention is to propose a federated learning method based on a pipeline mode, which enables each client to independently train on a local training set and gives participating learners more freedom and privacy protection without affecting the accuracy. It can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby producing a more accurate model.
[0006] The second object of the present invention is to propose another federated learning method based on the pipeline mode.
[0007] The third object of the present invention is to propose a federated learning device based on the pipeline mode.
[0008] The fourth objective of the present invention is to propose another federated learning device based on the pipeline mode.
[0009] A fifth objective of the present invention is to provide a server.
[0010] A sixth objective of the present invention is to provide a client.
[0011] A seventh object of the present invention is to provide a computer-readable storage medium.
[0012] An eighth object of the present invention is to provide a computer program product.
[0013] To achieve the above-mentioned purpose, the first aspect of the embodiment of the present invention proposes a federated learning method based on a pipeline mode, which is applied to a server, including: randomly selecting a client corresponding to a training task that has not been selected from a task set as a target client; wherein the task set is used to represent a set of merged multiple client training tasks; sending preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters; receiving the new local model parameters reported by the target client; using the new local model parameters as new global model parameters, and sending them to the next target client, so that the next target client continues to train until all training tasks in the task set are trained.
[0014] The pipeline-based federated learning method of the embodiment of the present invention is applied to the server, first randomly selecting a client corresponding to a training task that has not been selected from a task set that merges multiple client training tasks as the target client, and then sending the preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters, and finally receives the new local model parameters reported by the target client, and sends the new local model parameters as new global model parameters to the next target client, so that the next target user terminal continues to train until all training tasks in the task set are trained. Therefore, the pipeline-based federated learning method of this method enables each client to train independently on the local training set, and gives participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0015] In addition, the pipeline-based federated learning method proposed in the first aspect of the present invention may also have the following additional technical features:
[0016] According to an embodiment of the present invention, before randomly selecting a client corresponding to a training task that has not been selected from the task set as the target client, the method further includes:
[0017] Randomly select a client corresponding to a training task from the task set as the first target client;
[0018] Receiving preliminary local model parameters sent by the first target client; wherein the preliminary local model parameters are model parameters obtained by the first target client through training the local training set of the first target user;
[0019] The preliminary local model parameters are used as the preliminary global model parameters.
[0020] According to one embodiment of the present invention, after all training tasks in the task set are trained, the method further includes:
[0021] Determine whether the obtained global model meets the set conditions;
[0022] When the global model does not meet the set conditions, the task set is continued to be trained in the next round until a new global model meets the set conditions.
[0023] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present invention proposes another federated learning method based on the pipeline mode, which is applied to the client, including: receiving preliminary global model parameters sent by the server; using the preliminary global model parameters to assign model parameters to the local model of the target client to obtain the local model of the target client; wherein the target client is a client corresponding to a training task that has not been selected by the server randomly selected from the task set; using the local training set of the target client to train the local model of the target client to obtain new local model parameters; uploading the new local model parameters to the server, so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues to train until all training tasks in the task set are trained.
[0024] The pipeline-based federated learning method of the embodiment of the present invention is applied to the client, first receiving the preliminary global model parameters sent by the server, and then using the preliminary global model parameters to assign model parameters to the local model of the target client to obtain the local model of the target client, wherein the target client is a client corresponding to a training task that has not been selected randomly selected by the server from the task set, and finally the local model of the target client is trained using the local training set of the target client to obtain new local model parameters, and the new local model parameters are uploaded to the server, so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues to train until all training tasks in the task set are completed. Therefore, the pipeline-based federated learning method of this method enables each client to train independently on the local training set, and gives participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0025] In addition, the pipeline-based federated learning method proposed in the second aspect of the present invention may also have the following additional technical features:
[0026] According to an embodiment of the present invention, the using the local training set of the target client to train the local model of the target client to obtain new local model parameters includes:
[0027] Calculating a model gradient of the local model of the target client according to the local model parameters of the target client and the local training set;
[0028] According to the model gradient, the local model parameters of the target client are updated to obtain the new local model parameters.
[0029] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a federated learning device based on a pipeline mode, which is applied to a server and includes: a selection module, which is used to randomly select a client corresponding to a training task that has not been selected from a task set as a target client; wherein the task set is used to represent a set of merged multiple client training tasks; a first sending module, which is used to send preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters; a receiving module, which is used to receive the new local model parameters reported by the target client; a second sending module, which is used to use the new local model parameters as new global model parameters, and send them to the next target client, so that the next target client continues to train until all training tasks in the task set are trained.
[0030] The pipeline-based federated learning device of the embodiment of the present invention is applied to the server, and a client corresponding to a training task that has not been selected is randomly selected from a task set that merges multiple client training tasks as a target client through a selection module, and a preliminary global model parameter is sent to the target client through a first sending module, so that the target client uses the preliminary global model parameter to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters, and receives the new local model parameters reported by the target client through a receiving module, and uses the new local model parameters as new global model parameters through a second sending module, and sends them to the next target client, so that the next target user terminal continues to train until all training tasks in the task set are trained. Thus, the device based on the pipeline-based federated learning method enables each client to train independently on the local training set, and gives participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0031] In addition, the pipeline-based federated learning device proposed in the third aspect of the present invention may also have the following additional technical features:
[0032] According to an embodiment of the present invention, the selection module is used to randomly select a client corresponding to a training task that has not been selected from the task set as the target client, and is also used to:
[0033] Randomly select a client corresponding to a training task from the task set as the first target client;
[0034] Receiving preliminary local model parameters sent by the first target client; wherein the preliminary local model parameters are model parameters obtained by the first target client through training the local training set of the first target user;
[0035] The preliminary local model parameters are used as the preliminary global model parameters.
[0036] According to one embodiment of the present invention, the above device further includes:
[0037] A judgment module is used to judge whether the obtained global model meets the set conditions;
[0038] The determination module is used to continue the next round of training for the task set when the global model does not meet the set conditions until the new global model meets the set conditions.
[0039] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes another federated learning device based on the pipeline mode, which is applied to the client and includes: a receiving module for receiving the preliminary global model parameters sent by the server; an assignment module for using the preliminary global model parameters to assign model parameters to the local model of the target client to obtain the local model of the target client; wherein the target client is a client corresponding to a training task that has not been selected and is randomly selected by the server from the task set; a training module for training the local model of the target client using the local training set of the target client to obtain new local model parameters; an upload module for uploading the new local model parameters to the server so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues to train until all training tasks in the task set are trained.
[0040] The pipeline-based federated learning device of the embodiment of the present invention is applied to the client, and the preliminary global model parameters sent by the server are received through the receiving module, and the model parameters are assigned to the local model of the target client through the assignment module using the preliminary global model parameters to obtain the local model of the target client, wherein the target client is a client corresponding to a training task that has not been selected by the server randomly selected from the task set, and the local model of the target client is trained by using the local training set of the target client through the training module to obtain new local model parameters, and the new local model parameters are uploaded to the server through the upload module, so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues to train until all training tasks in the task set are trained. Thus, the device based on the pipeline-based federated learning method enables each client to train independently on the local training set, and gives the participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant to escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0041] In addition, the pipeline-based federated learning device proposed in the fourth aspect of the present invention may also have the following additional technical features:
[0042] According to an embodiment of the present invention, when the training module trains the local model of the target client using the local training set of the target client to obtain new local model parameters, it includes:
[0043] Calculating a model gradient of the local model of the target client according to the local model parameters of the target client and the local training set;
[0044] According to the model gradient, the local model parameters of the target client are updated to obtain the new local model parameters.
[0045] To achieve the above-mentioned purpose, the fifth aspect of the present invention proposes a server, comprising: a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the pipeline mode-based federated learning method of the first aspect.
[0046] The server of the embodiment of the present invention, by executing the above-mentioned pipeline-based federated learning method, can enable each client to independently perform training on a local training set, and give participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0047] To achieve the above-mentioned purpose, the sixth aspect of the present invention proposes a client, comprising: a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the pipeline mode-based federated learning method of the second aspect of the embodiment.
[0048] The client of the embodiment of the present invention, by executing the above-mentioned pipeline-based federated learning method, can enable each client to independently train on a local training set, and give participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0049] To achieve the above-mentioned objectives, the seventh aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the pipeline-based federated learning method of the first aspect, or implements the pipeline-based federated learning method of the second aspect.
[0050] The computer-readable storage medium of an embodiment of the present invention, by executing the pipeline-based federated learning method of the first aspect embodiment or the pipeline-based federated learning method of the second aspect embodiment, can enable each client to independently perform training on a local training set, and give participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0051] To achieve the above-mentioned objectives, the eighth aspect of the present invention proposes a computer program product. When the instruction processor in the computer program product executes, it executes the pipeline-based federated learning method of the first aspect, or executes the pipeline-based federated learning method of the second aspect.
[0052] The computer program product of the embodiment of the present invention executes the pipeline-based federated learning method of the first embodiment, or executes the pipeline-based federated learning method of the second embodiment, which enables each client to independently train on a local training set and gives participating learners more freedom and privacy protection without affecting the accuracy. It can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0053] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0055] Figure 1 is a flow chart of a pipeline-based federated learning method applied to a server according to an embodiment of the present invention;
[0056] Figure 2 is a flow chart of a pipeline-based federated learning method applied to a client according to an embodiment of the present invention;
[0057] Figure 3 is a block diagram of a pipeline-based federated learning device applied to a server according to an embodiment of the present invention;
[0058] Figure 4 4 is a block diagram of a pipeline-based federated learning device applied to a client according to an embodiment of the present invention. DETAILED DESCRIPTION
[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0060] The following describes a pipeline-based federated learning method, device, server, and client according to an embodiment of the present invention with reference to the accompanying drawings.
[0061] Figure 1 4 is a flowchart of a pipeline-based federated learning method applied to a server according to an embodiment of the present invention.
[0062] like Figure 1As shown, the present invention is applied to a pipeline-based federated learning method for a server, comprising:
[0063] S101, randomly selecting a client corresponding to a training task that has not been selected from a task set as a target client; wherein the task set is used to represent a set of merged training tasks for multiple clients.
[0064] Before executing step S101, the server may first combine the training tasks of multiple clients to form a task set. Then, in step S101, the server randomly selects a client corresponding to a training task that has not been selected from the task set as the target client.
[0065] S102, sending the preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters.
[0066] As a process of obtaining preliminary global model parameters, the server can randomly select a client corresponding to a training task from the task set as the first target client before executing step S101, and then receive preliminary local model parameters sent by the first target client, wherein the preliminary local model parameters are model parameters obtained by the first target client through training the local training set of the first target user, and finally use the preliminary local model parameters as preliminary global model parameters.
[0067] As another process of obtaining preliminary global model parameters, the server may first receive local model parameters sent by each client in the task set before executing step S101, and comprehensively determine preliminary global model parameters based on the local model parameters sent by each client.
[0068] In step S102, after obtaining the preliminary global model parameters, the server sends the preliminary global model parameters to the target client, so that the target client calculates the model gradient of the local model of the target client based on the local model parameters of the target client and the local training set, and updates the local model parameters of the target client according to the model gradient to obtain new local model parameters, and uploads them to the server.
[0069] S103: Receive new local model parameters reported by the target client.
[0070] S104: Use the new local model parameters as new global model parameters and send them to the next target client so that the next target client can continue training until all training tasks in the task set are completed.
[0071] In step S104, the server uses the new local model parameters as new global model parameters and sends them to the next target client, so that the next target client returns to execute steps S102-S104 until all training tasks in the task set are trained.
[0072] Therefore, the present invention adopts a pipeline-mode federated learning method, so that the client can train and learn independently. After each round of training and learning, the server is responsible for collecting the model parameters uploaded by the client after the training, updating and maintaining the global model parameters according to the optimization algorithm, and then performing the next round of iteration. During the entire learning process, except for the parameter update between the central analysis server and the client, the data of each client is completely independent to ensure the security of privacy data. In this way, each client can be trained independently on the local training set, and more freedom and privacy protection are given to the participating learners without affecting the accuracy, which can effectively help each participant escape the local optimum and enable them to explore other values, thereby generating a more accurate model.
[0073] According to one embodiment of the present invention, after all training tasks in the task set are trained, it also includes: judging whether the obtained global model meets the set conditions; if the global model does not meet the set conditions, continuing the next round of training on the task set until the new global model meets the set conditions.
[0074] In this embodiment, after a round of training is performed on the task set, it is determined whether the obtained global model meets the set conditions. If the global model does not meet the set conditions, the task set is continued to be trained in steps S101-S104 until the new global model meets the set conditions.
[0075] In summary, the pipeline-based federated learning method of the embodiment of the present invention is applied to the server, first randomly selecting a client corresponding to a training task that has not been selected from the task set that merges multiple client training tasks as the target client, and then sending the preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters, and finally receives the new local model parameters reported by the target client, and sends the new local model parameters as new global model parameters to the next target client, so that the next target user terminal continues to train until all training tasks in the task set are trained. Therefore, the pipeline-based federated learning method of this method enables each client to train independently on the local training set, and gives participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0076] Figure 2 4 is a flowchart of a pipeline-based federated learning method applied to a client according to an embodiment of the present invention.
[0077] like Figure 2 As shown, the present invention is applied to the pipeline mode-based federated learning method of the client, including:
[0078] S201, receiving preliminary global model parameters sent by the server.
[0079] S202, using preliminary global model parameters to assign model parameters to the local model of the target client to obtain the local model of the target client; wherein the target client is a client corresponding to a training task that has not been selected and is randomly selected by the server from the task set.
[0080] S203: Train the local model of the target client using the local training set of the target client to obtain new local model parameters.
[0081] S204, uploading the new local model parameters to the server so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues training until all training tasks in the task set are completed.
[0082] According to one embodiment of the present invention, training a local model of the target client using a local training set of the target client to obtain new local model parameters includes:
[0083] Calculate the model gradient of the local model of the target client according to the local model parameters of the target client and the local training set;
[0084] According to the model gradient, the local model parameters of the target client are updated to obtain new local model parameters.
[0085] It should be noted that for details not disclosed in the pipeline-based federated learning method applied to the client of the present invention, please refer to the details disclosed in the pipeline-based federated learning method applied to the server in the embodiment of the present invention, and the details will not be repeated here.
[0086] In summary, the pipeline-based federated learning method of the embodiment of the present invention is applied to the client, first receiving the preliminary global model parameters sent by the server, and then using the preliminary global model parameters to assign model parameters to the local model of the target client to obtain the local model of the target client, wherein the target client is a client corresponding to a training task that has not been selected randomly selected by the server from the task set, and finally the local model of the target client is trained using the local training set of the target client to obtain new local model parameters, and the new local model parameters are uploaded to the server, so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues to train until all training tasks in the task set are completed. Therefore, the pipeline-based federated learning method of this method enables each client to train independently on the local training set, and gives participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0087] Figure 3 4 is a block diagram of a pipeline-based federated learning device applied to a server according to an embodiment of the present invention.
[0088] like Figure 3 As shown, the present invention is applied to a pipeline-based federated learning device 300 for a server, comprising: a selection module 310 , a first sending module 320 , a receiving module 330 and a second sending module 340 .
[0089] Among them, the selection module 310 is used to randomly select a client corresponding to a training task that has not been selected from the task set as the target client; wherein the task set is used to represent a set of merged multiple client training tasks. The first sending module 320 is used to send preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters. The receiving module 330 is used to receive new local model parameters reported by the target client. The second sending module 340 is used to use the new local model parameters as new global model parameters and send them to the next target client, so that the next target user terminal continues training until all training tasks in the task set are trained.
[0090] According to an embodiment of the present invention, the selection module 310 is used to randomly select a client corresponding to a training task that has not been selected from the task set as the target client, and is also used to:
[0091] Randomly select a client corresponding to a training task from the task set as the first target client;
[0092] Receiving preliminary local model parameters sent by the first target client; wherein the preliminary local model parameters are model parameters obtained by the first target client through training the local training set of the first target user;
[0093] The preliminary local model parameters are used as preliminary global model parameters.
[0094] According to one embodiment of the present invention, the above device further includes:
[0095] A judgment module is used to judge whether the obtained global model meets the set conditions;
[0096] The determination module is used to continue the next round of training on the task set when the global model does not meet the set conditions until the new global model meets the set conditions.
[0097] It should be noted that for details not disclosed in the pipeline-based federated learning device applied to the server of the present invention, please refer to the details disclosed in the pipeline-based federated learning method applied to the server in the embodiment of the present invention, and the details will not be repeated here.
[0098] The pipeline-based federated learning device of the embodiment of the present invention is applied to the server, and a client corresponding to a training task that has not been selected is randomly selected from a task set that merges multiple client training tasks as a target client through a selection module, and a preliminary global model parameter is sent to the target client through a first sending module, so that the target client uses the preliminary global model parameter to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters, and receives the new local model parameters reported by the target client through a receiving module, and uses the new local model parameters as new global model parameters through a second sending module, and sends them to the next target client, so that the next target user terminal continues to train until all training tasks in the task set are trained. Thus, the device based on the pipeline-based federated learning method enables each client to train independently on the local training set, and gives participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0099] Figure 4 4 is a block diagram of a pipeline-based federated learning device applied to a client according to an embodiment of the present invention.
[0100] like Figure 4 As shown, the present invention is applied to a pipeline-based federated learning device 400 on a client, comprising: a receiving module 410 , an assignment module 420 , a training module 430 and an upload module 440 .
[0101] Among them, the receiving module 410 is used to receive the preliminary global model parameters sent by the server. The assignment module 420 is used to use the preliminary global model parameters to assign model parameters to the local model of the target client to obtain the local model of the target client; wherein the target client is a client corresponding to a training task that has not been selected randomly selected by the server from the task set. The training module 430 is used to train the local model of the target client using the local training set of the target client to obtain new local model parameters. The upload module 440 is used to upload the new local model parameters to the server, so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues to train until all training tasks in the task set are trained.
[0102] According to an embodiment of the present invention, when the training module 430 trains the local model of the target client using the local training set of the target client to obtain new local model parameters, it includes:
[0103] Calculate the model gradient of the local model of the target client according to the local model parameters of the target client and the local training set;
[0104] According to the model gradient, the local model parameters of the target client are updated to obtain new local model parameters.
[0105] It should be noted that for details not disclosed in the pipeline-based federated learning device applied to the client in an embodiment of the present invention, please refer to the details disclosed in the pipeline-based federated learning method applied to the client in an embodiment of the present invention, and the details will not be repeated here.
[0106] The pipeline-based federated learning device of the embodiment of the present invention is applied to the client, and the preliminary global model parameters sent by the server are received through the receiving module, and the model parameters are assigned to the local model of the target client by using the preliminary global model parameters through the assignment module to obtain the local model of the target client, and the target client is the client corresponding to a training task that has not been selected randomly selected by the server from the task set, and the local model of the target client is trained by using the local training set of the target client through the training module to obtain new local model parameters, and the new local model parameters are uploaded to the server through the upload module, so that the server uses the new local model parameters as new global model parameters and sends them to the next target client, and the next target client continues to train until all training tasks in the task set are trained. Therefore, the device based on the pipeline-based federated learning method enables each client to train independently on the local training set, and gives more freedom and privacy protection to the participating learners without affecting the accuracy, which can effectively help each participant to escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0107] Based on the above embodiments, the present invention also proposes a server.
[0108] The server of an embodiment of the present invention includes: a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement a pipeline-based federated learning method applied to the server.
[0109] The server of the embodiment of the present invention, by executing the above-mentioned pipeline-based federated learning method applied to the server, can enable each client to independently train on a local training set, and give participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0110] Based on the above embodiments, the present invention also proposes a client.
[0111] The client of an embodiment of the present invention includes: a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement a pipeline-based federated learning method applied to the client.
[0112] The client of the embodiment of the present invention, by executing the above-mentioned pipeline-based federated learning method applied to the client, can enable each client to independently train on a local training set, and give participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0113] Based on the above embodiments, the present invention also proposes a computer-readable storage medium.
[0114] The computer-readable storage medium of an embodiment of the present invention stores a computer program thereon, which, when executed by a processor, implements a federated learning method based on a pipeline mode applied to a server, or implements a federated learning method based on a pipeline mode applied to a client.
[0115] The computer-readable storage medium of an embodiment of the present invention, by executing a pipeline-based federated learning method applied to a server or a pipeline-based federated learning method applied to a client, can enable each client to independently perform training on a local training set, and give participating learners more freedom and privacy protection without affecting the accuracy, which can effectively help each participant escape the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0116] Based on the above embodiments, the present invention also proposes a computer program product.
[0117] When the instruction processor in the computer program product of the embodiment of the present invention is executed, the pipeline-based federated learning method applied to the server is executed, or the pipeline-based federated learning method applied to the client is executed.
[0118] The computer program product of the embodiment of the present invention executes a pipeline-based federated learning method applied to a server, or executes a pipeline-based federated learning method applied to a client, which enables each client to independently train on a local training set and gives participating learners more freedom and privacy protection without affecting the accuracy. It can effectively help each participant escape from the local optimal solution and enable them to explore other values, thereby generating a more accurate model.
[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0120] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0121] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.
[0122] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0123] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0124] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0125] In addition, each functional unit in each embodiment of the present invention may be integrated into one module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0126] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A pipeline-based federated learning method. It is characterized in that Applicable to servers, including: Randomly select a client corresponding to a training task that has not been selected from the task set as the target client; wherein the task set is used to represent a set of merged multiple client training tasks; Sending the preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters; Receiving the new local model parameters reported by the target client; The new local model parameters are used as new global model parameters and sent to the next target client so that the next target client continues to perform training until all training tasks in the task set are completed.
2. The method according to claim 1, It is characterized in that Before randomly selecting a client corresponding to a training task that has not been selected from the task set as the target client, the method further includes: Randomly select a client corresponding to a training task from the task set as the first target client; Receiving preliminary local model parameters sent by the first target client; wherein the preliminary local model parameters are model parameters obtained by the first target client through training the local training set of the first target user; The preliminary local model parameters are used as the preliminary global model parameters.
3. The method according to claim 1, It is characterized in that After all training tasks in the task set are completed, the following steps are also included: Determine whether the obtained global model meets the set conditions; When the global model does not meet the set conditions, the task set is continued to be trained in the next round until a new global model meets the set conditions.
4. A pipeline-based federated learning method. It is characterized in that Applied to the client, including: Receive preliminary global model parameters sent by the server; Using the preliminary global model parameters to assign model parameters to the local model of the target client to obtain the local model of the target client; wherein the target client is a client corresponding to a training task that has not been selected and is randomly selected by the server from the task set; Using the local training set of the target client to train the local model of the target client to obtain new local model parameters; The new local model parameters are uploaded to the server so that the server uses the new local model parameters as new global model parameters and sends them to the next target client. The next target client continues training until all training tasks in the task set are completed.
5. The method according to claim 4, It is characterized in that The using the local training set of the target client to train the local model of the target client to obtain new local model parameters includes: Calculating a model gradient of the local model of the target client according to the local model parameters of the target client and the local training set; According to the model gradient, the local model parameters of the target client are updated to obtain the new local model parameters.
6. A federated learning device based on pipeline mode, It is characterized in that Applicable to servers, including: A selection module, used to randomly select a client corresponding to a training task that has not been selected from a task set as a target client; wherein the task set is used to represent a set of merged client training tasks; A first sending module, used for sending preliminary global model parameters to the target client, so that the target client uses the preliminary global model parameters to assign model parameters to the local model of the target client, and uses the local training set of the target client to train the assigned local model to obtain new local model parameters; A receiving module, configured to receive the new local model parameters reported by the target client; The second sending module is used to use the new local model parameters as new global model parameters and send them to the next target client so that the next target client can continue training until all training tasks in the task set are completed.
7. A federated learning device based on pipeline mode, It is characterized in that Applied to the client, including: A receiving module, used for receiving preliminary global model parameters sent by the server; An assignment module, used to assign model parameters to a local model of a target client using the preliminary global model parameters to obtain a local model of the target client; wherein the target client is a client corresponding to a training task that has not been selected and is randomly selected by the server from a task set; A training module, used to train the local model of the target client using the local training set of the target client to obtain new local model parameters; The uploading module is used to upload the new local model parameters to the server so that the server uses the new local model parameters as new global model parameters and sends them to the next target client. The next target client continues training until all training tasks in the task set are completed.
8. A server, It is characterized in that include: Processor and memory; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the pipeline mode-based federated learning method as described in any one of claims 1 to 3.
9. A client, It is characterized in that include: Processor and memory; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the pipeline mode-based federated learning method as described in claim 4 or 5.
10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, it implements the pipeline-based federated learning method as described in any one of claims 1 to 3, or implements the pipeline-based federated learning method as described in claim 4 or 5.
