An overall interaction method for a federated learning platform

Abstract

The present invention discloses an overall interaction method for a federated learning platform, belonging to the technical field of federated learning platforms, including S1: resource registration; S2: querying the ecological library, where the user queries model library-related information from the basic AI-ecological model library and industry data from the industry AI-database through the industry front-end and back-end; S3: creating a task; S4: querying a task; S5: joining a task; S6: starting an Agent; S7: federated training. Through the industry front-end and back-end, operations such as resource registration, querying the ecological library, creating a task, querying a task, joining a task, and starting an Agent can be performed. All processes of the federated learning platform establish an interactive connection with the user, thereby realizing comprehensive interactive learning as a whole; and continuously improving the federated training model to improve the accuracy of federated learning, thereby improving the efficiency and accuracy of federated learning.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning platforms, and particularly to an overall interaction method for a federated learning platform. Background Art

[0002] Bayesian logic combines generative models and probability theory to calculate the true likelihood of a specific answer given the data. Bottom-up deep learning starts with data rather than theory. It involves labeling large amounts of data ("correct" and "wrong" data) to determine associations and lay the foundation for pattern recognition. It can even learn unsupervised, detect patterns in unlabeled data, and identify clusters (factor analysis). The era of renewed interest in machine learning techniques was followed by the era of deep learning techniques. In recent years, it has led us to the next in-depth study of intelligent and / or neuromorphic computing, namely, federated learning technology.

[0003] Through machine learning, humans can input the input and the desired output. Thereafter, the output comes from the algorithm generated by the algorithm. Instead of directly programming the generated algorithm, the semantics for generating the algorithm are programmed. In this way, humans encode complex behaviors with rules of lower complexity. Although the algorithm does not require programming, these neural networks still need to be trained by humans. The neural networks need to present the input data in a structured manner. Therefore, collecting, cleaning, and labeling data involve a lot of manual assistance work. Efforts are also needed to evaluate the model by guiding the model in the correct direction. However, it is recognized that the beauty of machine learning expressed in this way is that it can even be recursively applied to itself.

[0004] However, the existing federated learning cannot achieve comprehensive interaction as a whole, and the constructed model cannot be adjusted for each process according to user needs. Relying on its own data feedback as a reference for the next run, it is easy to generate large biases in the repeated learning process, and it is impossible to gradually improve the federated learning model, let alone meet the high-precision requirements of the model generated with the depth of federated learning. Summary of the Invention

[0005] The purpose of the present invention is to provide an overall interaction method for a federated learning platform. Through the industry front-end and back-end, operations such as resource registration, querying the ecological library, creating tasks, querying tasks, joining tasks, and starting the Agent can be performed. Each process of the federated learning platform establishes an interactive connection with the user, thereby achieving comprehensive interactive learning as a whole; and continuously improving the joint training model, improving the accuracy of federated learning, and further improving the efficiency and accuracy of federated learning to solve the problems raised in the above background art.

[0006] To achieve the above object, the present invention provides the following technical solution: An overall interaction method for a federated learning platform, comprising the following steps:

[0007] S1: Resource registration, where the user uploads local resources to the computing decision module and receives the registration feedback from the computing decision module;

[0008] S2: Query the ecological library. The user queries the information related to the model library from the basic AI-ecological model library and queries the industry data from the industry AI-database through the industry front-end and back-end;

[0009] S3: Create a task. Perform initial configuration of the task on the industry training configuration through the industry front-end and back-end and submit the task;

[0010] S4: Query the task. Query the federated training task from the basic AI-federated learning engine through the industry front-end and back-end;

[0011] S5: Join the task. Join the federated learning task to the basic AI-federated learning engine through the industry front-end and back-end;

[0012] S6: Start the Agent. After successful deployment, the federated learning engine Agent and the local training agent;

[0013] S7: Federated training.

[0014] Further, the resource registration includes the following steps:

[0015] S101: The user reports the local hardware resources, computing power, and operation parameters to the management console through the industry front-end and back-end;

[0016] S102: The platform resource controller uploads these resource node data to the computing decision module. The computing decision module updates the platform basic resource topology and returns the result to the platform resource controller;

[0017] S103: The platform resource controller further returns the resource registration result to the industry front-end and back-end.

[0018] Further, the query of the ecological library includes the following steps:

[0019] S201: The user sends a request to the basic AI-ecological model library through the industry front-end and back-end to query the historical federated training information, which is used as the basis for whether to participate in the federated training;

[0020] S202: The user sends a request to the industry AI-database through the industry front-end and back-end to query the processed industry data.

[0021] Further, the creation of the task includes the following steps:

[0022] S301: The user configures the initial settings for federated training through the industry front-end background and then submits them to the federated learning engine;

[0023] S302: The federated learning engine requests a long data list from the computing decision module and, based on the long data list, feeds back the verification result and the short data list to the user.

[0024] Further, the query task includes the following steps:

[0025] S401: The user sends a request through the industry front-end background to the federated learning engine to query the federated training task;

[0026] S402: The engine returns the query result to the user, and the user decides whether to join one of the ongoing federated training tasks.

[0027] Further, the joining task includes the following steps:

[0028] S501: The user sends a request through the industry front-end background to the federated learning engine to request to join the federated training;

[0029] S502: The federated learning engine requests a long data list from the computing decision module, then verifies at the algorithm, resource, etc. levels based on the long data list, returns the verification result to the user, and obtains the short data list;

[0030] S503: After passing the verification, the federated learning engine sends an orchestration and deployment request to the computing decision module, and then the platform resource controller performs the deployment and returns the deployment result to the computing decision module and the federated learning engine.

[0031] Further, the federated training includes the following steps:

[0032] S701: Request data. After the Agent starts, it first calls the industry backend api interface according to the parameters to obtain a certain training data set. After receiving the request, the industry backend queries the corresponding database or local server to find the corresponding training data and returns the training data to the Agent;

[0033] S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information. The Agent determines whether to perform the current federated iteration process based on the monitoring information;

[0034] S703: This round of iterative training. When the monitored information meets the training criteria, the Agent performs this round of iterative training, uploads the model of this round of training to the fl-engine, and downloads the aggregated model from the fl-engine;

[0035] S704: Interaction of monitoring information. The Agent feeds back the monitoring information obtained from the platform resource controller before this iteration, the heartbeat information obtained after the iteration, and the information on the error and accuracy of the model to the industry backend.

[0036] S705: Training termination. When the normal collaborative training ends, the Agent uploads the final model to the fl-engine. The fl-engine saves the model to the model library, and the Agent calls the collaborative termination interface of the fl-engine to end the training.

[0037] Furthermore, the collaborative training includes the following steps:

[0038] S701: Request data. The Agent directly obtains data from the library or locally according to the metadata information provided by the industry user and integrates it into a training dataset.

[0039] S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information. The Agent determines whether to perform the current collaborative iteration process based on the monitoring information.

[0040] S703: This round of iterative training. When the monitored information meets the training criteria, the Agent performs this round of iterative training, uploads the model of this round of training to the fl-engine, and downloads the aggregated model from the fl-engine.

[0041] S704: Interaction of monitoring information. The Agent feeds back the monitoring information obtained from the platform resource controller before this iteration, the heartbeat information obtained after the iteration, and the information on the error and accuracy of the model to the industry backend.

[0042] S705: Training termination. When the normal collaborative training ends, the Agent uploads the final model to the fl-engine. The fl-engine saves the model to the model library, and the Agent calls the collaborative termination interface of the fl-engine to end the training.

[0043] Furthermore, the collaborative training includes the following steps:

[0044] S701: Request data. After the Agent starts, it first calls the industry backend api interface according to the parameters to obtain a certain training dataset. After receiving the request, the industry backend queries the corresponding database or local server to find the corresponding training data and returns the training data to the Agent.

[0045] S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information. The Agent determines whether to perform the current collaborative iteration process based on the monitoring information.

[0046] S703: In this round of iterative training, when the monitored information meets the training criteria, the Agent executes this round of iterative training, uploads the model trained in this round to the fl-engine, and downloads the aggregated model from the fl-engine.

[0047] S704: Interaction of monitored information. The Agent feeds back the monitored information obtained from the platform resource controller before this iteration, the heartbeat information obtained after the iteration, and the information on the error and accuracy of the model to the industry backend.

[0048] S705: Training termination. The industry backend calls the Agent's training termination interface, and the user notifies the Agent to actively exit the training information based on the obtained monitored information.

[0049] Further, the collaborative training includes the following steps:

[0050] S701: Request data. The Agent directly obtains data from the library or locally according to the metadata information provided by the industry user and integrates it into a training dataset.

[0051] S702: Obtain monitored information. Send a request to the platform resource controller to obtain monitored information. The Agent determines whether to perform this round of collaborative iteration process based on the monitored information.

[0052] S703: In this round of iterative training, when the monitored information meets the training criteria, the Agent executes this round of iterative training, uploads the model trained in this round to the fl-engine, and downloads the aggregated model from the fl-engine.

[0053] S704: Interaction of monitored information. The Agent feeds back the monitored information obtained from the platform resource controller before this iteration, the heartbeat information obtained after the iteration, and the information on the error and accuracy of the model to the industry backend.

[0054] S705: Training termination. The industry backend calls the Agent's training termination interface, and the user notifies the Agent to actively exit the training information based on the obtained monitored information.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows: An overall interaction method of a federated learning platform proposed by the present invention enables operations such as resource registration, querying the ecological library, creating tasks, querying tasks, joining tasks, and starting the Agent through the industry front-end and back-end. All processes of the federated learning platform establish interactive connections with users, thereby achieving comprehensive interactive learning as a whole; obtaining monitoring information from the platform resource controller, delivering training data and monitoring information to the Agent. When the monitored information meets the training criteria, the Agent performs iterative training. The Agent feeds back the monitoring information obtained from the platform resource controller before this iteration, the heartbeat information obtained after iteration, the model error, and the accuracy information to the industry back-end, continuously improving the federated training model, enhancing the accuracy of federated learning, and further improving the efficiency and accuracy of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is the overall flowchart of the overall interaction method of the federated learning platform of the present invention;

[0057] Figure 2 It is the schematic diagram of resource registration of the overall interaction method of the federated learning platform of the present invention;

[0058] Figure 3 It is the flowchart of querying the ecological library of the overall interaction method of the federated learning platform of the present invention;

[0059] Figure 4 It is the flowchart of creating a task of the overall interaction method of the federated learning platform of the present invention;

[0060] Figure 5 It is the flowchart of querying a task of the overall interaction method of the federated learning platform of the present invention;

[0061] Figure 6 It is the flowchart of joining a task of the overall interaction method of the federated learning platform of the present invention;

[0062] Figure 7 It is the flowchart of federated training of the overall interaction method of the federated learning platform in Embodiment 1 of the present invention;

[0063] Figure 8 It is the flowchart of federated training of the overall interaction method of the federated learning platform in Embodiment 2 of the present invention;

[0064] Figure 9 It is the flowchart of federated training of the overall interaction method of the federated learning platform in Embodiment 3 of the present invention;

[0065] Figure 10 It is the flowchart of federated training of the overall interaction method of the federated learning platform in Embodiment 4 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0066] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0067] Agent technology is a computer system packaged in a certain environment. In order to achieve the design purpose, it can move flexibly and autonomously in this environment. In 1995, Wooldrige gave two definitions of Agent: (weak definition) Agent is used to generally describe a software and hardware system, which has the following characteristics: autonomy, sociality, reactivity, and initiative; (strong definition) In addition to all the characteristics in the weak definition, Agent should also have some characteristics that only humans have, such as knowledge, belief, obligation, intention, etc.

[0068] Embodiment 1

[0069] Refer to Figure 1 , an overall interaction method of a collaborative learning platform, including the following steps:

[0070] S1: Resource registration, the user uploads local resources to the computing decision module and receives the registration feedback from the computing decision module;

[0071] S2: Query the ecological library, the user queries the information related to the model library from the basic AI-ecological model library and queries the industry data from the industry AI-database through the industry front-end background;

[0072] S3: Create a task, perform initial configuration of the task on the industry training configuration through the industry front-end background and submit the task;

[0073] S4: Query the task, query the collaborative training task from the basic AI-collaborative learning engine through the industry front-end background;

[0074] S5: Join the task, join the collaborative learning task to the basic AI-collaborative learning engine through the industry front-end background;

[0075] S6: Start the Agent. After successful deployment, the collaborative learning engine Agent and the local training agent;

[0076] S7: Collaborative training.

[0077] Refer to Figure 2 , the resource registration includes the following steps:

[0078] S101: The user reports local hardware resources, computing power, and operation parameters to the management console through the industry front-end background;

[0079] S102: The platform resource controller uploads these resource node data to the computing decision-making module. The computing decision-making module updates the platform basic resource topology and returns the result to the platform resource controller;

[0080] S103: The platform resource controller further returns the resource registration result (success or failure) to the industry front-end background.

[0081] Refer to Figure 3 , the steps for querying the ecological library are as follows:

[0082] S201: The user sends a request to the basic AI-ecological model library through the industry front-end background to query historical joint training information, including information such as participants in joint training, models in joint training, accuracies corresponding to the models, and model download times, etc., which is used as a basis for whether to participate in joint training;

[0083] S202: The user sends a request to the industry AI-database through the industry front-end background to query processed industry data, including their own training data and others' metadata.

[0084] Refer to Figure 4 , the steps for creating a task are as follows:

[0085] S301: The user configures the initial configuration of joint training (algorithm selection, data selection, metadata information, computing power requirements, etc.) through the industry front-end background and then submits it to the joint learning engine;

[0086] S302: The joint learning engine requests a long data list from the computing decision-making module and, based on the long data list, feeds back the verification result and a short data list to the user.

[0087] Refer to Figure 5 , the steps for querying a task are as follows:

[0088] S401: The user sends a request to the joint learning engine through the industry front-end background to query joint training tasks (in progress, terminated);

[0089] S402: The engine returns the query result to the user, and the user decides whether to join one of the ongoing joint training tasks.

[0090] Refer to Figure 6 , the steps for joining a task are as follows:

[0091] S501: The user sends a request to the joint learning engine through the industry front-end background to request to join joint training;

[0092] S502: The federated learning engine requests a long data list from the computing decision-making module, then performs verification at levels such as algorithms and resources based on the long data list, returns the verification result to the user (passed or not passed), and obtains a short data list.

[0093] S503: After passing, the federated learning engine sends an orchestration and deployment request to the computing decision-making module. Then the platform resource controller executes the deployment and returns the deployment result to the computing decision-making module and the federated learning engine.

[0094] Refer to Figure 7 , the federated training includes the following steps:

[0095] S701: Request data. After the Agent starts, it first calls the industry backend api interface according to parameters such as task name, algorithm name, and data type to obtain a certain training data set. After receiving the request, the industry backend queries the corresponding database or local server to find the corresponding training data and returns the training data to the Agent; it is required that the industry participant perform a K-V mapping on the data, where K is the interface input parameter for obtaining the corresponding V training data, and the Agent does not interfere with the data storage method.

[0096] S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information such as cpu usage rate, memory usage rate, and remaining disk space. The Agent determines whether to perform the current federated iteration process based on the monitoring information.

[0097] S703: This round of iterative training. When the monitored information meets the training criteria, the Agent performs this round of iterative training, uploads the model of this round of training to the fl-engine, and downloads the aggregated model from the fl-engine; at this time, the fl-engine will save the aggregated intermediate model.

[0098] S704: Interaction of monitoring information. The Agent feeds back the monitoring information (usage rate, network) obtained from the platform resource controller before this iteration, the heartbeat information (whether online) obtained after the iteration, and the information of the model error and accuracy to the industry backend.

[0099] S705: Training termination. When the normal federated training ends and meets the end conditions (epoch, accuracy, etc.), the Agent uploads the final model to the fl-engine, the fl-engine saves the model to the model library, and the Agent calls the federated termination interface of the fl-engine to end the training.

[0100] The industry front-end and back-end, industry training configuration, basic AI - local training agent, and platform resource controller are all located locally, and the industry front-end and back-end are directly connected to users; the basic AI - federated learning engine, basic AI - ecological model library, industry AI - database, and computing decision-making module are all located on the cloud platform.

[0101] Embodiment 2

[0102] Refer to Figure 8 , the federated training includes the following steps:

[0103] S701: Request data. The Agent directly obtains data from the library or locally according to the metadata information provided by the industry user and integrates it into a training data set;

[0104] S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information. The Agent determines whether to perform the current federated iteration process according to the monitoring information;

[0105] S703: This round of iterative training. When the monitored information meets the training criteria, the Agent performs this round of iterative training, uploads the model trained in this round to the fl-engine, and downloads the aggregated model from the fl-engine;

[0106] S704: Interaction of monitoring information. The Agent feeds back the monitoring information obtained from the platform resource controller before this iteration, the heartbeat information obtained after the iteration, the error and accuracy information of the model to the industry back-end;

[0107] S705: Training termination. When the normal federated training ends, the Agent uploads the final model to the fl-engine, the fl-engine saves the model to the model library, and the Agent calls the federated termination interface of the fl-engine to end the training.

[0108] Embodiment 3

[0109] Refer to Figure 9 , the federated training includes the following steps:

[0110] S701: Request data. After the Agent starts, it first calls the industry back-end api interface according to the parameters to obtain a certain training data set. After receiving the request, the industry back-end queries the corresponding database or local server to find the corresponding training data and returns the training data to the Agent;

[0111] S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information. The Agent determines whether to perform the current federated iteration process according to the monitoring information;

[0112] S703: In this round of iterative training, when the monitored information meets the training criteria, the Agent executes this round of iterative training, uploads the model trained in this round to the fl-engine, and downloads the aggregated model from the fl-engine;

[0113] S704: Interaction of monitoring information. The Agent feeds back the monitoring information obtained from the platform resource controller before this iteration, the heartbeat information obtained after the iteration, and the information on the error and accuracy of the model to the industry backend;

[0114] S705: Training termination. The industry backend calls the Agent's training termination interface, and the user notifies the Agent to actively exit the training information based on the obtained monitoring information.

[0115] Embodiment 4

[0116] Refer to Figure 10 , the joint training includes the following steps:

[0117] S701: Request data. The Agent directly obtains data from the library or locally according to the metadata information such as the server address, database name, and table name provided by the industry user, and integrates it into a training dataset. (PS Note: This method requires the Agent to obtain server address permissions and database permissions, which may sensibly reduce the user's trust in the Agent);

[0118] S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information such as CPU usage rate, memory usage rate, and remaining disk space. The Agent determines whether to perform this joint iteration process based on the monitoring information;

[0119] S703: In this round of iterative training, when the monitored information meets the training criteria, the Agent executes this round of iterative training, uploads the model trained in this round to the fl-engine, and downloads the aggregated model from the fl-engine; at this time, the fl-engine will save the aggregated intermediate model;

[0120] S704: Interaction of monitoring information. The Agent feeds back the monitoring information (usage rate, network) obtained from the platform resource controller before this iteration, the heartbeat information (whether online) obtained after the iteration, and the information on the error and accuracy of the model to the industry backend;

[0121] S705: Training is terminated. The industry backend calls the Agent termination training interface. The user notifies the Agent to actively exit the training information based on the acquired monitoring information, such as when a certain accuracy is reached. In this case, the industry backend calls the Agent termination training interface (the Agent termination training interface also calls the fl-engine exit joint training interface to remove the participant from the joint list), thereby ending the Agent training.

[0122] In summary, the overall interactive method of the joint learning platform proposed in the present invention can register resources, query the ecological library, create tasks, query tasks, join tasks and start Agent operations through the industry front-end and back-end. Each process of the joint learning platform establishes an interactive connection with the user, thereby realizing comprehensive interactive learning as a whole; obtaining monitoring information from the platform resource controller, transmitting training data and monitoring information to the Agent, when the monitored information meets the training standards, the Agent performs iterative training, and the Agent feeds back the monitoring information obtained from the platform resource controller before this iteration, the heartbeat information obtained after the iteration, and the model error and accuracy information to the industry back-end, continuously improving the joint training model, improving the accuracy of joint learning, and thereby improving the efficiency and accuracy of joint learning.

[0123] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An overall interaction method for a federated learning platform, characterized in that, it includes the following steps: S1: Resource registration, where the user uploads local resources to the computing decision module and receives the registration feedback from the computing decision module; S2: Query the ecological library. The user queries model library-related information from the basic AI-ecological model library and industry data from the industry AI-database through the industry front-end background; S3: Create a task. Through the industry front-end background, perform initial configuration of the task for industry training configuration and submit the task; S4: Query the task. Through the industry front-end background, query the federated training task from the basic AI-federated learning engine; S5: Join the task. Through the industry front-end background, join the federated learning task to the basic AI-federated learning engine; S6: Start the Agent. After successful deployment, the federated learning engine Agent, local training agent; S7: Federated training; the federated training includes the following steps: S701: Request data. The Agent directly obtains data from the library or locally according to the metadata information provided by the industry user and integrates it into a training data set; S702: Obtain monitoring information. Send a request to the platform resource controller to obtain monitoring information. The Agent judges whether to perform the current federated iteration process according to the monitoring information; the monitoring information includes obtaining cpu usage, memory usage, disk remaining, network; S703: This round of iterative training. When the monitored information meets the training standard, the Agent executes this round of iterative training, uploads the model trained in this round to the fl-engine, and downloads the aggregated model from the fl-engine; S704: Interaction of monitoring information. The Agent feeds back the monitoring information obtained from the platform resource controller before this iteration, the heartbeat information obtained after iteration, and the information of the error and accuracy of the model to the industry backend; S705: Training termination. The industry backend calls the Agent to terminate the training interface, and the user notifies the Agent to actively exit the training information according to the obtained monitoring information.

2. A method for the overall interaction of a federated learning platform as described in claim 1, characterized in that, the resource registration includes the following steps: S101: The user reports local hardware resources, computing power, and operation parameters to the management console through the industry front-end background; S102: The platform resource controller uploads these resource node data to the computing decision module. The computing decision module updates the platform basic resource topology and returns the result to the platform resource controller; S103: The platform resource controller further returns the resource registration result to the industry front-end background.

3. A method for the overall interaction of a federated learning platform as described in claim 1, characterized in that, the query of the ecological library includes the following steps: S201: The user sends a request to the basic AI-ecological model library through the industry front-end background to query historical federated training information, which is used as a basis for whether to participate in federated training; S202: The user sends a request to the industry AI-database through the industry front-end background to query the processed industry data.

4. A method for the overall interaction of a federated learning platform as described in claim 1, It is characterized in that the query task includes the following steps: S401: The user sends a request to the federated learning engine through the industry front-end background to query the federated training task; S402: The engine returns the query result to the user, and the user decides whether to join one of the ongoing federated training tasks.

5. A method for the overall interaction of a federated learning platform as described in claim 1, It is characterized in that in the steps included in the federated training, step S701 is replaced with: S701: Request data. After the Agent is started, it first calls the industry background api interface according to the parameters to obtain the training data set. After receiving the request, the industry backend queries the corresponding database or local server to find the corresponding training data and returns the training data to the Agent.

6. A method for the overall interaction of a federated learning platform as described in claim 1, It is characterized in that in the steps included in the federated training, step S705 is replaced with: S705: Training termination. When the normal federated training ends, the Agent uploads the final model to the fl-engine, the fl-engine saves the model to the model library, and the Agent calls the federated termination interface of the fl-engine to end the training.

Images