An enhanced horizontal federated learning method and device

Abstract

The present application discloses an enhanced horizontal federated learning method, including: an application applies to a user for a federated learning data access right for accessing non-essential user data; under the federated learning data access right, non-essential user data is isolated from the application, the application calls a unified federated learning service (UnifiedFED), UnifiedFED obtains non-essential user data, and receives necessary user data from the application, and provides the non-essential user data and necessary user data to a model for horizontal federated learning training. The present application also discloses a corresponding enhanced horizontal federated learning device. By applying the technical solution disclosed in the present application, it is possible to achieve strict separation of application programs and user data.

Description

Technical Field

[0001] The present application relates to the field of privacy protection technology, and in particular to an enhanced horizontal federated learning method and device. Background Art

[0002] With the development of the Internet and the popularization of mobile phones, IOT and other terminal devices, people enjoy the convenience of modern life, while user data is collected by various applications, and privacy leakage is becoming increasingly serious. In the era of big data, many applications rely on a large amount of various user data to train their AI models and optimize their services. Some of this user data is necessary for applications to provide services, while some is unnecessary. Sometimes, because of user privacy, the user data that is necessary for applications can also be processed locally without uploading to cloud servers.

[0003] In order to curb and solve the problem of privacy leakage, the government has formulated more stringent privacy protection regulations (such as GPDR); people are also studying new methods to achieve the goal of using user data to train and use AI models while protecting user privacy. Federated Learning was born in this context.

[0004] In 2016, Google Research first proposed the concept of federated learning to solve the problem of keyboard input method optimization for individual users. The key points of federated learning technology include:

[0005] 1. Global model;

[0006] 2. Store user data securely locally.

[0007] For example, the content entered by the user on the keyboard does not need to be uploaded to the server, and the model training and prediction can be completed locally through distributed services. Since Google proposed the concept of federated learning, this technology field has developed rapidly, and Google launched the Tensorflow Federated Framework. Tencent WeBank open-sourced the FATE federated learning system. In 2019, Apple iOS 13 used federated learning technology in QuickType Keyboard, Siri personalized automatic speech recognition and other functions. In 2020, IEEE approved the first international standard for federated learning, IEEE 3652.1-2020.

[0008] Federated learning is also known as joint learning or alliance learning. Federated learning is a machine learning framework that can effectively help multiple institutions use data and conduct machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations. In federated learning, the parties do not directly exchange data, which can protect user privacy to a large extent.

[0009] Federated learning can be divided into: horizontal federated learning, vertical federated learning, and federated transfer learning. Figure 1 This is a classification diagram of federated learning technology, see below Figure 1 The above three federated learning technologies are explained respectively.

[0010] 1. Horizontal Federated Learning

[0011] 1) Learning process

[0012] a. Each participant downloads the latest model from server A;

[0013] b. Each participant uses local data to train the model, encrypts the gradient and uploads it to server A. Server A aggregates the user's gradient and updates the model parameters.

[0014] c. Server A returns the updated model to each participant;

[0015] d. Each participant updates their own model separately.

[0016] 2) Prediction process

[0017] Each machine has the same complete model, and each machine can make independent predictions during prediction.

[0018] 2. Vertical Federated Learning

[0019] Figure 2 This is a schematic diagram of the principle of vertical federated learning. Figure 2 Provide explanation.

[0020] 1) Learning process

[0021] a. Encrypted sample alignment; this operation is performed at the system level, so non-crossing users are not exposed at the enterprise-aware level (for example, enterprise A and enterprise B as shown in the figure);

[0022] b. A third party (collaborator C as shown in the figure) distributes public keys to A and B to encrypt the data to be transmitted;

[0023] cA and B respectively calculate the intermediate results of their own features and encrypt the interactions to obtain their respective gradients and losses;

[0024] dA and B calculate their own encrypted gradients respectively, add masks and send them to collaborator C respectively. At the same time, A and B calculate the encrypted loss and send it to C.

[0025] eC decrypts the gradient and loss and passes them back to A and B. A and B remove the mask and update the model prediction process.

[0026] 2) Prediction process

[0027] Since each participant can only obtain the model parameters related to itself, the prediction requires collaboration between both parties.

[0028] The inventor of this application analyzes the prior art and finds that the prior art has the following problems:

[0029] 1. Currently, federated learning on mobile phones / IOT devices has not completely separated applications and data, as shown in the following aspects:

[0030] 1) Applications can still access user data by applying for corresponding data permissions.

[0031] 2) Applications rely on self-regulation and are subject to privacy regulations, and do not upload user privacy data but process it locally.

[0032] 3) How applications use data is not transparent. Users know that applications have data, but they do not know how applications use the data, which objectively hinders applications from obtaining certain user information. For example, if an e-commerce application applies to read a user's address book, it is likely to be rejected by the user.

[0033] 2. In the era of big data and AI algorithms, applications have an unlimited desire for user data, but privacy regulations and user acceptance do not allow applications to obtain unnecessary data without restriction. There is an objective contradiction between the two. Therefore, it is necessary to find a solution so that applications can obtain good AI models without obtaining user data.

[0034] 3. Federated learning tasks are computationally intensive, consume a lot of CPU and GPU resources, and are also very power-intensive. In addition, considering that there may be multiple federated learning tasks running simultaneously on the device side in the future, the device side needs a unified system-level service to optimize and manage all tasks. Summary of the invention

[0035] The present application provides an enhanced horizontal federated learning method and apparatus to achieve strict separation of application and user data.

[0036] This application discloses an enhanced horizontal federated learning method, including:

[0037] The application requests the user to access the federated learning data permission for accessing non-essential user data;

[0038] Under the federated learning data access rights, non-essential user data is isolated from the application, and the application calls a unified federated learning service (UnifiedFED). The UnifiedFED obtains non-essential user data and receives necessary user data from the application, and provides the non-essential user data and necessary user data to the model for horizontal federated learning training; wherein, the UnifiedFED is a trusted service for horizontal federated learning.

[0039] Preferably, when an application calls UnifiedFED, UnifiedFED performs permission management on the application, and the permission management specifically includes:

[0040] Review the data access permissions of the application;

[0041] Determine whether the application has the access right to the federated learning data. If so, end the permission management process and continue to execute the UnifiedFED to obtain non-essential user data; otherwise, apply to the user for the access right to the federated learning data;

[0042] If the user chooses to allow only this time, the permission management process ends, and the UnifiedFED continues to execute to obtain non-essential user data;

[0043] If the user chooses to always allow, the application is granted access to the federated learning data, and a corresponding record is added to the permission management, and the UnifiedFED continues to execute to obtain non-essential user data;

[0044] If the user chooses not to allow, authorization is denied.

[0045] Preferably, when the UnifiedFED obtains non-essential user data, it also includes obtaining the application's own data and model, and preprocessing the obtained data according to the user settings and parameters passed when the application calls UnifiedFED.

[0046] Preferably, before UnifiedFED provides data to the model for horizontal federated learning training, the process further includes: UnifiedFED scheduling the training task, wherein the scheduling specifically includes:

[0047] UnifiedFED determines whether to execute the training task immediately or asynchronously;

[0048] If the training task is to be executed immediately, when the current device status meets the conditions, the application is notified that model training can be performed, and the application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance; when the current device status does not meet the conditions, the application is notified that the request is rejected, and the application notifies the cloud server to select other devices for model training;

[0049] If the training task is executed asynchronously, the current device status is checked periodically, and when the current device status meets the conditions, the application is notified that model training can be performed. The application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance.

[0050] Preferably, the UnifiedFED determines whether the current device status meets the condition based on at least one of the following factors:

[0051] Task type, model size, number of parameters to be trained, device CPU / GPU / battery / network usage, amount of data to be trained, user settings;

[0052] Among them, the task types include: prediction tasks and training tasks. The priority of prediction tasks is higher than that of training tasks.

[0053] The size of the model, the number of parameters that need to be trained, and the amount of data that needs to be trained are used to estimate the time required for training.

[0054] The present application also provides an enhanced horizontal federated learning device, which is based on a public federated learning framework and includes at least: an application programming interface module, a permission management module, a task scheduler, and a federated learning framework layer interface module, wherein:

[0055] The application programming interface module is used to interact with the application program;

[0056] The permission management module is used to determine whether the application has the federated learning data access permission to access non-essential user data when receiving the scheduling of the application, and notify the task scheduler when the application has the federated learning data access permission to access non-essential user data;

[0057] The task scheduler is used to obtain non-essential user data according to the notification of the permission management module, and receive necessary user data from the application program, and provide the non-essential user data and necessary user data to the model through the federated learning framework layer interface module for horizontal federated learning training;

[0058] Under the federated learning data access rights, non-essential user data is isolated from applications.

[0059] Preferably, the rights management module is specifically used for:

[0060] Review the data access permissions of the application;

[0061] Determine whether the application has the access right to the federated learning data, and if so, notify the task scheduler that the application has the access right to the federated learning data; otherwise, notify the application to apply to the user for the access right to the federated learning data;

[0062] If the user chooses to allow only this time, notifying the task scheduler application that it has access rights to the federated learning data;

[0063] If the user selects "Always Allow", the application is granted the access right to the federated learning data, and a corresponding record is added to the permission management to notify the task scheduler that the application has the access right to the federated learning data;

[0064] If the user chooses not to allow, authorization is denied.

[0065] Preferably, the device further comprises a data preprocessing module;

[0066] The data preprocessing module is used to preprocess the user data according to the user settings and the parameters passed when the application calls UnifiedFED; the user data includes the obtained non-essential user data and the application's own data.

[0067] Preferably, the task scheduler is specifically used for:

[0068] Determine whether to execute the training task immediately or asynchronously;

[0069] If the training task is to be executed immediately, when the current device status meets the conditions, the application is notified that model training can be performed, and the application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance; when the current device status does not meet the conditions, the application is notified that the request is rejected, and the application notifies the cloud server to select other devices for model training;

[0070] If the training task is executed asynchronously, the current device status is checked periodically, and when the current device status meets the conditions, the application is notified that model training can be performed. The application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance.

[0071] The device refers to a user device that applies or includes the apparatus.

[0072] Preferably, the task scheduler determines whether the current device status meets the condition based on at least one of the following factors:

[0073] Task type, model size, number of parameters to be trained, device CPU / GPU / battery / network usage, amount of data to be trained, user settings;

[0074] Among them, the task types include: prediction tasks and training tasks. The priority of prediction tasks is higher than that of training tasks.

[0075] The size of the model, the number of parameters that need to be trained, and the amount of data that needs to be trained are used to estimate the time required for training.

[0076] It can be seen from the above technical solution that the enhanced horizontal federated learning method and device provided by this application introduces a new data access permission dedicated to federated learning: federated learning data access permission, and under the federated learning data access permission, data access and interaction with the model are all completed by UnifiedFED, and the application cannot know the data content, thereby achieving strict isolation between application and user data and improving the security of the use of user privacy data.

[0077] Secondly, through the federated learning data access rights, this application can introduce new user features (i.e., user privacy data) for training on the basis of horizontal federated learning, so that the trained model not only has the advantages of horizontal federated learning, but also has the advantages of feature combination in vertical federated learning.

[0078] In addition, this application can be more intelligent and power-saving by scheduling and managing application applications, scheduling applications of multiple different applications according to user settings and system CPU / GPU / battery / network usage conditions.

[0079] Based on the above scheme, this application can support data preprocessing for model training or prediction by defining a common data preprocessing function. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A classification diagram of federated learning technologies;

[0081] Figure 2 This is a schematic diagram of the principle of vertical federated learning;

[0082] Figure 3 This is a schematic diagram of the principle of the enhanced horizontal federated learning technology of this application;

[0083] Figure 4 Schematic diagram of the module composition of a preferred enhanced horizontal federated learning device for this application

[0084] Figure 5 A schematic diagram of the process of performing rights management by a preferred rights management module for this application;

[0085] Figure 6 A schematic diagram of a process of data preprocessing performed by a preferred data preprocessing module of the present application;

[0086] Figure 7 A schematic diagram of a preferred task scheduler for the present application performing task scheduling;

[0087] Figure 8 A schematic diagram of the process of applying for federated learning data access rights for an application in Embodiment 1 of the present application;

[0088] Fig. 9 This is a schematic diagram of the model training process of the second embodiment of the present application;

[0089] Fig.10 This is a schematic diagram of the model prediction process of Example 3 of the present application;

[0090] Fig.11 It is a comparative schematic diagram before and after the application is adopted;

[0091] Fig.12 It is a schematic diagram of a typical application scenario of the present application. DETAILED DESCRIPTION

[0092] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0093] In order to solve the problems existing in the prior art, this application proposes a new technical solution based on the existing horizontal federated learning technology, which is called Unified Federated Learning Service (UnifiedFED) in the subsequent description process. The technical points are as follows:

[0094] 1. Set up a new type of data access permission specifically for federated learning: federated learning data access permission.

[0095] Federated learning data access permissions are different from ordinary data access permissions. Applications can apply for federated learning data access permissions. Under ordinary data access permissions, data will be provided directly to applications; under federated learning data access permissions, data access and interaction with the model are completed by UnifiedFED, and applications cannot know the data content. This method can isolate applications and user data. Therefore, for user privacy data, users can authorize applications to grant this permission, thereby ensuring isolation between applications and user privacy data and improving the security of user privacy data.

[0096] 2. Based on the access rights of federated learning data, realize "Enhanced Horizontal Federated Learning".

[0097] Through the federated learning data access rights, UnifiedFED can obtain user privacy data, so that new user features (i.e., user privacy data) can be introduced for training on the basis of horizontal federated learning, thereby realizing an enhanced horizontal federated learning technology, which not only has the advantages of horizontal federated learning, but also has the advantages of feature combination in vertical federated learning.

[0098] 3. By scheduling and managing application applications, the application of multiple different applications can be scheduled according to user settings and the system CPU / GPU / battery / network usage, making it smarter and more power-saving.

[0099] 4. Since the application cannot directly access and modify the data, this application defines some common data preprocessing functions to support data preprocessing for model training or prediction.

[0100] 5. Introduce an incentive mechanism to encourage users to participate in model training, reasonably evaluate the contribution of each participant, and provide some form of incentive.

[0101] Feasibility demonstration of the above technical solution of this application: If horizontal federated learning and vertical federated learning can ensure the security of user data and do not violate privacy regulations, then the enhanced horizontal federated learning of this application also meets the conditions of ensuring the security of user data and not violating privacy regulations in most cases. Specifically:

[0102] 1) This application's enhanced horizontal federated learning uses the same framework as horizontal federated learning at the Common Federated Learning Framework layer to ensure data security.

[0103] 2) If the user data obtained by an enterprise can be used to jointly train AI models with other enterprises through vertical federated learning without violating user privacy regulations, then UnifiedFED, which directly applies to users for "federated learning data access rights", will not violate privacy regulations.

[0104] 3) The difference between this application and ordinary horizontal federated learning is that it introduces user privacy data into the training. It is only necessary to ensure the security of this data. This application strictly isolates the application and user privacy data through UnifiedFED, and AI model data feeding, training and prediction are all completed by UnifiedFED.

[0105] The above is a description of the technical points and feasibility analysis of the technical solution of the present application. The specific implementation of the technical solution of the present application is further described in detail below with reference to the accompanying drawings.

[0106] Figure 3 This is a schematic diagram of the principle of the enhanced horizontal federated learning technology for this application, see Figure 3 :

[0107] The code logic of the application (e.g., App A in the figure) applies to the user for general data access permissions for accessing necessary user data, and for federated learning data access permissions for accessing non-essential user data; where non-essential user data corresponds to user privacy data;

[0108] Necessary user data is directly accessed by the application;

[0109] Non-essential user data (as shown in the figure, also called: non-essential data used only for federated learning) is isolated from the application. The application calls UnifiedFED, and the non-essential user data is directly transmitted to UnifiedFED without passing through the application. In addition, UnifiedFED can also receive necessary user data actively sent by the application, and provide the obtained non-essential user data and / or necessary user data to the model for horizontal federated learning training.

[0110] UnifiedFED can schedule multiple federated learning tasks based on device status and user settings, and can predefine some data preprocessing functions to support data preprocessing and introduce user incentive mechanisms.

[0111] Figure 4 This is a schematic diagram of the module composition of a preferred enhanced horizontal federated learning device of the present application, which corresponds to the UnifiedFED as described above, and is based on the Common Federated Learning Framework layer, and can include the following modules: Application Programming Interface (APP Interface) module, Status Tracker (Status Monitor), User Setting (User Setting) module, Security Check (Security Check) module, Rights Management (Rights Management) module, Task Scheduler (Task Scheduler), Data Preprocessing (DataProcessing) module, Incentive Mechanism (Incentive Mechanism) module and Federated Learning Framework Interface (Federated Learning Framework Interface) module. The following is a description of each module in conjunction with the attached drawings.

[0112] Figure 5 This is a schematic diagram of the process of performing permission management in a preferred permission management module for this application, see Figure 5 , including the following steps:

[0113] Step 1: The application calls UnifiedFED to apply for specific non-essential permissions for use only in federated learning data, i.e., non-essential user data / user privacy data as mentioned above.

[0114] Step 2: Review the data access permissions of the app.

[0115] Step 3: Determine whether the application has access rights to federated learning data. If so, skip to step 7; otherwise, continue to step 4.

[0116] Step 4: Apply to users for access rights to federated learning data.

[0117] Step 5: If the user selects a) Allow only this time, skip to step 7;

[0118] If the user selects b) Always allow, proceed to step 6;

[0119] If the user selects c) Do not allow, authorization is denied and this process ends.

[0120] Step 6: Grant the application specified access rights to federated learning data and add corresponding records in the permission management module.

[0121] Step 7: Continue with the subsequent processing flow.

[0122] Figure 6 This is a schematic diagram of the process of data preprocessing performed by a preferred data preprocessing module of the present application, see Figure 6 , including the following steps:

[0123] Step 1: After the permission management module checks the application's federated learning data access rights, proceed to step 2.

[0124] Step 2: UnifiedFED obtains data that is not necessary for federated learning, that is, non-essential user data.

[0125] Step 3: UnifiedFED obtains the application's own data and models.

[0126] Step 4: Perform data preprocessing according to the user settings and the parameters passed when the application calls UnifiedFED.

[0127] Step 5: The preprocessed data is put into the model for training or prediction, that is, feeding data to the model (that is, providing data to the model).

[0128] In the above data preprocessing process, user privacy data is strictly independent of the application. The access to this user data and the interaction with the model are completed by UnifiedFED. The application does not know the content of the data, which is more secure. In addition, users can also specify the granularity of certain privacy data exposure through user settings. For example, for home address information, by default, it is only provided to the community level and not accurate to the house number.

[0129] The AI ​​models and data owned by the application can be transmitted to UnifiedFED through shared memory or other means, and UnifiedFED will provide data to the model in a unified manner.

[0130] Figure 7 This is a schematic diagram of a preferred task scheduler for this application for task scheduling, see Figure 7 , including the following steps:

[0131] Step 1: The application registers with UnifiedFED to request model training.

[0132] Each application registers a federated learning task (i.e., training model) with UnifiedFED. These federated learning tasks exist in the form of "tasks".

[0133] Step 2: Determine whether to execute the task immediately or asynchronously. If it is executed immediately, continue to step 3.a; if it is executed asynchronously, continue to step 3.b.

[0134] Step 3.a: The task scheduler checks the current device status.

[0135] Step 4.a: Determine whether the current device status meets the conditions. If so, jump to step 4.b; otherwise, continue to step 5.a.

[0136] The task scheduler is responsible for scheduling the running of all federated learning tasks. When scheduling, the task scheduler needs to determine whether the current device status meets the conditions. Factors to consider may include: task type, model size, number of parameters to be trained, device CPU / GPU / battery / network usage, amount of data to be trained, user settings, etc. Among them:

[0137] There are two types of tasks: prediction and training. The priority of prediction tasks is higher than that of training tasks.

[0138] The time required for training can be estimated based on the size of the model, the number of parameters, the equipment status, etc.

[0139] Model training is very resource-intensive, and users can set some rules through user settings. For example, to reduce network traffic costs, you can set: only train when connected to WIFI; to reduce power consumption, only participate in model training when the power is greater than the set threshold, or when it is charging, or in a specified time period; in order not to affect other applications, you can specify that model training is only performed when the device is idle.

[0140] Applications can provide material incentives to encourage users to participate in model training. The task scheduler can also ensure that the contributions, data quality, and benefits of different users are fair through a dedicated incentive mechanism module.

[0141] Step 5.a: The application is notified that the request is rejected, and then the application notifies the server to select another device for model training, ending this process.

[0142] Step 3.b: The task scheduler periodically checks the current device status. When the current device status conditions are met, it notifies the registered application that the device is ready and can perform model training.

[0143] Step 4.b: The application notifies the cloud server that the device can perform model training. If the cloud server selects itself, the model is downloaded from the cloud to the local device and training begins. The operation of downloading the model can also be performed in steps 1, 2, and 3 in advance.

[0144] Security Check Module:

[0145] Security is of paramount importance for federated learning. In the technical solution provided by this application, the Common Federated Learning Framework, which is independent of UnifiedFED, is used to ensure security. The Common Federated Learning Framework can resist common security attacks, including:

[0146] a. Model extraction attack: The attacker attempts to steal the parameters and hyperparameters of the model and destroy the confidentiality of the model;

[0147] b. Model reverse attack: The attacker attempts to obtain information about the training data set from the trained model, thereby obtaining the user's private information;

[0148] c. Poisoning attack, confrontation attack, etc.

[0149] On this basis, the security check module in UnifiedFED is enhanced in terms of security, mainly including: security checks on APKs listed in the extended app store and local APKs, security checks on applications applying for access rights to federated learning data, and security checks on AI models and codes used for training and prediction. The industry has already conducted some research and practice on APK security check methods.

[0150] A brief description of other modules:

[0151] Incentive mechanism module: Design a relatively fair benefit distribution plan, evaluate the contribution of each node in model training, and ensure that the contribution, data quality and benefits of different users are fair. There have been some relevant studies in the industry on the specific incentive mechanism design plan, which will not be elaborated here.

[0152] Status Tracker: Monitors and manages the tasks running in UnifiedFED, and can manually stop certain tasks.

[0153] Application Programming Interface: API interface provided for application programs to call.

[0154] Federated learning framework layer interface module: used to interact with the public federated learning framework.

[0155] The implementation of the technical solution of the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0156] Embodiment 1:

[0157] This embodiment describes the process of an application applying for access rights to federated learning data. Figure 8 As shown, where:

[0158] Step 1: The application applies for specific federated learning data access permissions, and the permission management module in UnifiedFED accepts the application.

[0159] Step 2: The permission management module in UnifiedFED pops up a UI interface to request the user to access the federated learning data. The user can choose not to authorize, authorize only once, or always allow.

[0160] Step 3: The user makes an authorization choice, and the result of the user's choice is saved in the permission management module.

[0161] Permission requests usually occur when an application is installed in the form of APK, etc., or when the application is running and interacting with the user.

[0162] The actual training process of federated learning generally occurs when the system is idle, and the user is likely not looking at the phone at this time, so he will not respond to the application's permission request. Therefore, usually, the application will not wait until training to apply for permission.

[0163] Embodiment 2:

[0164] Fig. 9 This is a schematic diagram of the model training process of Example 2 of this application, see Fig. 9 , the model training process includes:

[0165] Step 1: The cloud server sends a request to the application to perform model training.

[0166] Step 2: The application sends a request to the UnifiedFED of this device, asking if it is ready for training.

[0167] Step 3: After checking the application’s federated learning data access permissions and device status, the task scheduler notifies UnifiedFED that it is ready for training or not.

[0168] Step 4: The application notifies the cloud server that it is ready for training.

[0169] Step 5: The cloud server selects and notifies the device in the candidate training device list to participate in the training.

[0170] Step 6: The application calls the UnifiedFED API, trains the model, and passes the parameters.

[0171] Step 7.1: The cloud server shares the model to the user’s device.

[0172] Step 7.2: UnifiedFED obtains the authorized federated learning user data.

[0173] Step 7.3: Application-owned data is passed to UnifiedFED.

[0174] The above steps 7.1 to 7.3 can be performed simultaneously or in any other order, regardless of the order.

[0175] Step 8: The training task is scheduled for execution.

[0176] Step 9: UnifiedFED sends the gradient to the cloud server.

[0177] Step 10: The cloud server securely aggregates the training results sent back by each device.

[0178] Step 11: The cloud server sends the updated model back to UnifiedFED.

[0179] Fig. 9 In the process shown, before model training, the cloud server has a candidate selection process. For example, the server may send a training request to 10,000 devices, and 5,000 devices agree, but in the end the server may only select 3,000 of them to participate in model training.

[0180] Model training may be completed in one round or multiple rounds, so steps 9, 10, and 11 may be repeated for multiple rounds.

[0181] Embodiment three:

[0182] Fig.10 This is a schematic diagram of the model prediction process of Example 3 of this application, see Fig.10 , including the following steps:

[0183] Step 1: The application calls the UnifiedFED API to obtain the prediction results of a specific model.

[0184] Step 2.1: If the local model needs to be updated, UnifiedFED downloads the latest model from the cloud server. Considering that the model download time is relatively long, the model download process in this step can be completed in advance when the application is running using the preloading mechanism, without having to download it when it is used.

[0185] Step 2.2: The application shares its own data with UnifiedFED.

[0186] Step 2.3: UnifiedFED obtains authorized federated learning user data.

[0187] The above steps 2.1 to 2.3 can be performed simultaneously or in any other order, regardless of the order.

[0188] Step 3: The application's model prediction task is scheduled for execution and the result is returned after execution.

[0189] Step 4: UnifiedFED returns the prediction result to the application.

[0190] Since model prediction requires immediate results, the model prediction task has the highest priority. If other models are being trained and occupy resources at this time, the other models will be paused or terminated.

[0191] Examples of application scenarios of the technical solution of this application:

[0192] Application scenario 1:

[0193] Fig.11 This is a comparison diagram before and after the adoption of this application. By adopting this application, the user allows application A to use the federated learning data access rights of certain user data that are not necessary for the application to run. Application A combines its own data and non-essential user data to train a better AI model.

[0194] Application scenario 2:

[0195] Fig.12 is a schematic diagram of a typical application scenario of this application. Fig.12 ,include:

[0196] Step 1: UnifiedFED reads the user settings.

[0197] For example, the user settings are as follows:

[0198] The network condition is set to: run only when WIFI is connected;

[0199] The battery condition is set to: when charging or battery level is greater than 80%;

[0200] The allowed time range for training is set to: 23:00-06:00.

[0201] Step 2: UnifiedFED monitors system CPU / GPU / battery / network usage.

[0202] Step 3: UnifiedFED schedules all federated learning tasks based on user settings and system CPU / GPU / battery / network usage to ensure that the federated learning model training tasks are completed without interfering with the user's normal use of the device.

[0203] Step 4: When the device completes training, the contribution is evaluated and appropriate incentives are given.

[0204] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An enhanced horizontal federated learning method, characterized in that: The method is to train a model for an application program to optimize the service of the application program, and to construct a unified federated learning service (UnifiedFED) based on a general federated learning framework layer, which at least includes an application programming interface provided to the application program; the method includes: The application applies to the user for a federated learning data access permission for accessing non-essential user data; Under the federated learning data access rights, non-essential user data is isolated from the application, and the application calls the unified federated learning service (UnifiedFED) through the application programming interface. The non-essential user data is directly transmitted to the UnifiedFED without passing through the application. The UnifiedFED obtains the non-essential user data and receives the necessary user data from the application, and provides the non-essential user data and the necessary user data to the model for horizontal federated learning training. The horizontal federated learning training is trained using local data; wherein, the UnifiedFED is a trusted service for horizontal federated learning.

2. The method according to claim 1, characterized in that: When an application calls UnifiedFED, UnifiedFED performs permission management on the application. The permission management specifically includes: Review the data access permissions of the application; Determine whether the application has the access right to the federated learning data. If so, end the permission management process and continue to execute the UnifiedFED to obtain non-essential user data; otherwise, apply to the user for the access right to the federated learning data; If the user chooses to allow only this time, the permission management process ends, and the UnifiedFED continues to execute to obtain non-essential user data; If the user chooses to always allow, the application is granted access to the federated learning data, and a corresponding record is added to the permission management, and the UnifiedFED continues to execute to obtain non-essential user data; If the user chooses not to allow, authorization is denied.

3. The method according to claim 2, characterized in that: When the UnifiedFED acquires non-essential user data, it also includes acquiring the application's own data and model, and preprocessing the acquired data according to the user settings and the parameters passed when the application calls the UnifiedFED.

4. The method according to claim 3, characterized in that Before UnifiedFED provides data to the model for horizontal federated learning training, UnifiedFED also schedules the training task, and the scheduling specifically includes: UnifiedFED determines whether to execute the training task immediately or asynchronously; If the training task is to be executed immediately, when the current device status meets the conditions, the application is notified that model training can be performed, and the application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance; when the current device status does not meet the conditions, the application is notified that the request is rejected, and the application notifies the cloud server to select other devices for model training; If the training task is executed asynchronously, the current device status is checked periodically, and when the current device status meets the conditions, the application is notified that model training can be performed. The application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance.

5. The method according to claim 4, characterized in that The UnifiedFED determines whether the current device status meets the condition based on at least one of the following factors: Task type, model size, number of parameters to be trained, device CPU / GPU / battery / network usage, amount of data to be trained, user settings; Among them, the task types include: prediction tasks and training tasks. The priority of prediction tasks is higher than that of training tasks. The size of the model, the number of parameters that need to be trained, and the amount of data that needs to be trained are used to estimate the time required for training.

6. An enhanced horizontal federated learning device, characterized in that: The device architecture is based on a public federated learning framework, trains a model for an application to optimize the service of the application, and includes at least: an application programming interface module, a rights management module, a task scheduler, and a federated learning framework layer interface module, wherein: The application programming interface module is used to interact with the application program; The permission management module is used to determine whether the application has the federated learning data access permission to access non-essential user data when receiving the scheduling of the application, and notify the task scheduler when the application has the federated learning data access permission to access non-essential user data; The task scheduler is used to obtain non-essential user data according to the notification of the permission management module, the non-essential user data is directly transmitted to UnifiedFED without passing through the application, and receives necessary user data from the application, and provides the non-essential user data and necessary user data to the model through the federated learning framework layer interface module for horizontal federated learning training, wherein the horizontal federated learning training is training using local data; Under the federated learning data access rights, non-essential user data is isolated from applications.

7. The device according to claim 6, characterized in that The rights management module is specifically used for: Review the data access permissions of the application; Determine whether the application has access rights to the federated learning data, and if so, notify the task scheduler that the application has access rights to the federated learning data; Otherwise, notify the application to apply for the federated learning data access permission from the user; If the user chooses to allow only this time, notifying the task scheduler application that it has access rights to the federated learning data; If the user selects "Always Allow", the application is granted the access right to the federated learning data, and a corresponding record is added to the permission management to notify the task scheduler that the application has the access right to the federated learning data; If the user chooses not to allow, authorization is denied.

8. The device according to claim 7, characterized in that The device also includes a data preprocessing module; The data preprocessing module is used to preprocess the user data according to the user settings and the parameters passed when the application calls UnifiedFED; the user data includes the obtained non-essential user data and the application's own data.

9. The device according to claim 8, characterized in that The task scheduler is specifically used for: Determine whether to execute the training task immediately or asynchronously; If the training task is to be executed immediately, when the current device status meets the conditions, the application is notified that model training can be performed, and the application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance; when the current device status does not meet the conditions, the application is notified that the request is rejected, and the application notifies the cloud server to select other devices for model training; If the training task is executed asynchronously, the current device status is checked periodically, and when the current device status meets the conditions, the application is notified that model training can be performed. The application notifies the cloud server that the device can be trained, and downloads the model when the device is selected by the cloud server, or downloads the model in advance. The device refers to a user device that applies or includes the apparatus.

10. The device according to claim 9, characterized in that The task scheduler determines whether the current device status meets the condition based on at least one of the following factors: Task type, model size, number of parameters to be trained, device CPU / GPU / battery / network usage, amount of data to be trained, user settings; Among them, the task types include: prediction tasks and training tasks. The priority of prediction tasks is higher than that of training tasks. The size of the model, the number of parameters that need to be trained, and the amount of data that needs to be trained are used to estimate the time required for training.

Images