Privacy data processing method, device, equipment and medium based on federated learning

Abstract

The present invention discloses a method, device, equipment and medium for processing privacy data based on federated learning, including: the participants use statistical analysis to remove anomalies and preprocess local basic data to obtain local standard data; the participants perform local model training based on local standard data to obtain an updated local model, and upload the model parameters of the updated local model to the collaborating parties; the collaborating parties receive the model parameters uploaded by each participant, and screen out the participants participating in the aggregation; the collaborating parties aggregate the model parameters uploaded by the participants participating in the aggregation to obtain updated global parameters, and send them to each participant for the next round of training until the global model of the collaborating parties converges to obtain the target global model; the collaborating parties send the model parameters of the target global model to each participant for privacy data processing. The present invention improves the efficiency and security of privacy data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, device, equipment and medium for processing privacy data based on federated learning. Background Art

[0002] Federated learning is a distributed machine learning technology with privacy protection and secure encryption technology. It aims to allow decentralized participants to collaborate on model building and training without exposing their sensitive data. The core idea is to build and train distributed models between participants with local data sources, and to build a global model based on virtual fusion data by exchanging the parameters of their training models or intermediate calculation results without exchanging their own individual or sample data, so as to achieve a balance between data privacy protection and data sharing calculation, so as to achieve the effect of "data available but invisible" and "data unchanged but model moving". Horizontal federated learning is also called federated learning divided by samples, but as the amount of data increases, the time required for processing privacy data in horizontal federated learning will gradually increase.

[0003] In the current technical solution, the following methods are mainly used to improve the training efficiency of horizontal federated learning:

[0004] 1. Optimize the communication overhead during the training of the federated learning model. Each client only needs to upload the updated part of the parameters to the central server, without uploading all the parameters, such as the patent document of patent application number CN202310484383.3;

[0005] 2. Alleviate the problem of stragglers in the synchronous aggregation algorithm by selecting the client, such as using a greedy protocol mechanism to select the client with the highest score for aggregation through the reward value; or minimize the total learning time of online federated learning while ensuring the convergence accuracy of the model, and finally obtain the optimal strategy for adaptive participant sampling, such as the patent document of patent application number CN202310262721.9;

[0006] 3. Reduce waiting time and improve the training efficiency of federated learning through asynchronous aggregation algorithm. The so-called asynchronous aggregation algorithm means that after each client uploads the local model, the central server will immediately aggregate according to the client's parameters without waiting for other clients to complete the execution, such as the patent document with patent application number CN202211014211.1.

[0007] In the process of realizing the present invention, the inventors realized that the prior art has at least the following technical problems: Due to the heterogeneity of clients and unstable network environment in federated learning, the use of the above method for federated learning training will lead to a decrease in the accuracy of federated learning and uncertainty in training efficiency, and cannot effectively improve the training efficiency of horizontal federated learning. At the same time, in the process of improving the training efficiency of federated learning by selecting clients, the existing technical solutions cannot filter out participants with data problems and cannot prevent data attacks, which will lead to a decrease in the accuracy of the global model and threaten the security of private data. Summary of the invention

[0008] The embodiments of the present invention provide a method, apparatus, computer device and storage medium for processing privacy data based on federated learning to improve the security of privacy data processing.

[0009] In order to solve the above technical problems, the embodiment of the present application provides a method for processing private data based on federated learning, which is applied to a horizontal federated learning system. The horizontal federated learning system is a CS architecture, S is a collaborator, and C is a participant. The method for processing private data based on federated learning includes:

[0010] Participants use statistical analysis to eliminate anomalies and pre-process local basic data to obtain local standard data;

[0011] The participant trains the local model based on the local standard data to obtain an updated local model, and uploads the model parameters of the updated local model to the collaborating party;

[0012] The collaborator receives the model parameters uploaded by each participant, and based on the received model parameters, screens the participants participating in each round of aggregation through a supervised integration method to determine the participants participating in the aggregation;

[0013] The collaborating party aggregates the model parameters uploaded by the participating parties to obtain updated global parameters, and sends the updated global parameters to each participating party for the next round of training until the global model of the collaborating party converges and the target global model is obtained;

[0014] The collaborating party sends the model parameters of the target global model to each participating party for privacy data processing.

[0015] Optionally, the participant uses statistical analysis to remove anomalies and preprocess the local basic data to obtain local standard data including:

[0016] Collect local data to be tested as local basic data;

[0017] Perform data preprocessing on local basic data to obtain preprocessed data;

[0018] Using the interquartile range method, anomaly identification is performed on the preprocessed data, and the data detected to have anomalies are marked as abnormal data;

[0019] Abnormal data in the preprocessed data is eliminated to obtain the local standard data.

[0020] Optionally, the participant performs local model training based on local standard data, and obtains an updated local model including:

[0021] Normalize the local standard data to obtain training data;

[0022] Copy the global model parameters issued by the collaborator and extract training tasks from the task pool, where each training task contains a set of input-output pairs;

[0023] Perform local model training based on the global model parameters and the training task.

[0024] Optionally, based on the received model parameters, the participants participating in each round of aggregation are screened in a supervised integration manner, and the participants participating in the aggregation are determined to include:

[0025] Determine the evaluation indicators;

[0026] Obtaining parameter information associated with the evaluation index for each participant;

[0027] For each participant, based on the evaluation index and the parameter information associated with the evaluation index, calculate the evaluation score of the participant;

[0028] The participants whose evaluation scores exceed the preset threshold are regarded as candidates;

[0029] All candidates are clustered by cluster analysis to obtain several categories;

[0030] Using the preset screening method, the participants involved in the aggregation are screened out from each category.

[0031] Optionally, the collaborating party aggregates the model parameters uploaded by the participants participating in the aggregation, and the updated global parameters obtained include:

[0032] The following formula is used to aggregate the model parameters uploaded by the participants to obtain the updated global parameters:

[0033] ;

[0034] in, is the updated global parameter, is the global parameter of the previous round, is the model gradient of the kth participant in the aggregation, is the total number of participants involved in the aggregation, is the data volume of the kth participant in the aggregation, is the learning rate of the global model, is the total amount of data involved in the aggregation.

[0035] In order to solve the above technical problems, the embodiment of the present application further provides a privacy data processing device based on federated learning, including:

[0036] The data standardization module is used to eliminate anomalies and preprocess local basic data by means of statistical analysis to obtain local standard data;

[0037] A local training module is used to perform local model training based on local standard data to obtain an updated local model, and upload model parameters of the updated local model to the collaborating party;

[0038] The participant screening module is used to receive the model parameters uploaded by each participant, and based on the received model parameters, screen the participants participating in each round of aggregation through a supervised integration method to determine the participants participating in the aggregation;

[0039] The aggregation update module is used to aggregate the model parameters uploaded by the participants to obtain updated global parameters, and send the updated global parameters to each participant for the next round of training until the global model of the collaborating party converges and the target global model is obtained;

[0040] The data processing module is used to send the model parameters of the target global model to each participant for privacy data processing.

[0041] Optionally, the data standardization module includes:

[0042] A collection unit, used to collect local data to be detected as local basic data;

[0043] A preprocessing unit, used for preprocessing local basic data to obtain preprocessed data;

[0044] An anomaly identification unit is used to identify anomalies of the preprocessed data by using an interquartile range method, and mark the data detected to have anomalies as abnormal data;

[0045] The abnormality elimination unit is used to eliminate the abnormal data in the preprocessed data to obtain the local standard data.

[0046] Optionally, the local training module includes:

[0047] A normalization unit, used for normalizing the local standard data to obtain training data;

[0048] The task acquisition unit is used to copy the global model parameters issued by the collaborator and extract training tasks from the task pool, where each training task contains a set of input-output pairs;

[0049] A local training unit is used to perform local model training based on the global model parameters and the training task.

[0050] Optionally, the participant screening module includes:

[0051] An indicator determination unit, used for determining evaluation indicators;

[0052] An information acquisition unit, used to acquire parameter information associated with the evaluation index of each participant;

[0053] A score calculation unit, configured to calculate, for each participant, an evaluation score of the participant based on the evaluation index and the parameter information associated with the evaluation index;

[0054] A candidate party determination unit, configured to select the parties whose evaluation scores exceed a preset threshold as candidate parties;

[0055] A clustering unit, used to cluster all candidate parties into several categories by cluster analysis;

[0056] The screening unit is used to screen out the participants participating in the aggregation from each category by using a preset screening method.

[0057] Optionally, the aggregation update module includes:

[0058] The calculation unit is used to aggregate the model parameters uploaded by the participants participating in the aggregation using the following formula to obtain the updated global parameters:

[0059] ;

[0060] in, is the updated global parameter, is the global parameter of the previous round, is the model gradient of the kth participant in the aggregation, is the total number of participants involved in the aggregation, is the data volume of the kth participant in the aggregation, is the learning rate of the global model, is the total amount of data involved in the aggregation.

[0061] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned privacy data processing method based on federated learning when executing the computer program.

[0062] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned privacy data processing method based on federated learning are implemented.

[0063] The method, device, computer equipment and storage medium for processing private data based on federated learning provided by the embodiment of the present invention are as follows: the participants use statistical analysis to remove abnormalities and preprocess local basic data to obtain local standard data; the participants perform local model training based on local standard data to obtain an updated local model, and upload the model parameters of the updated local model to the collaborating parties; the collaborating parties receive the model parameters uploaded by each participant, and based on the received model parameters, screen the participants participating in each round of aggregation in a supervised integration manner to determine the participants participating in the aggregation; the collaborating parties aggregate the model parameters uploaded by the participants participating in the aggregation to obtain updated global parameters, and send the updated global parameters to each participant for the next round of training until the global model of the collaborating parties converges to obtain the target global model; the collaborating parties send the model parameters of the target global model to each participant for private data processing. The local data of each participant is marked with abnormal values ​​by statistical analysis, so as to remove the abnormal data of each participant and ensure the data quality of each participant in the subsequent horizontal federated learning process. Using a supervised integration approach to screen participants can ensure fairness and diversity among all participants in federated learning. This helps prevent individual participants from monopolizing or over-relying on certain data sources, thereby increasing the robustness and reliability of the global model. At the same time, by selecting only a portion of the participants to participate in the aggregation, the communication and computing overhead in the federated learning process can be reduced. This can reduce the demand for network bandwidth and the computing load of the collaborators, improve time and resource efficiency, and also improve the performance of the overall federated learning model. This will improve the efficiency and security of privacy data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0065] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0066] Figure 2 is a flowchart of an embodiment of a method for processing private data based on federated learning of the present application;

[0067] Figure 3 is a structural diagram of an embodiment of a privacy data processing device based on federated learning according to the present application;

[0068] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0070] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] See also Figure 1,like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0073] Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0074] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0075] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0076] It should be noted that the privacy data processing method based on federated learning provided in the embodiment of the present application is executed by a server, and accordingly, the privacy data processing device based on federated learning is arranged in the server.

[0077] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. The terminal devices 101, 102, 103 in the embodiment of the present application may specifically correspond to application systems in actual production.

[0078] See also Figure 2 , Figure 2 A method for processing private data based on federated learning provided by an embodiment of the present invention is shown, which is applied to a horizontal federated learning system. The horizontal federated learning system is a CS architecture, S is a collaborator, and C is a participant. The method is applied in Figure 1 The server in is used as an example to illustrate the details as follows:

[0079] S201: The participants use statistical analysis to eliminate anomalies and pre-process local basic data to obtain local standard data.

[0080] In a specific optional implementation, in step S201, the participants use statistical analysis to remove anomalies and preprocess the local basic data, and the local standard data obtained includes:

[0081] Collect local data to be tested as local basic data;

[0082] Perform data preprocessing on local basic data to obtain preprocessed data;

[0083] The interquartile range method is used to identify anomalies in the preprocessed data, and the data detected to have anomalies are marked as abnormal data;

[0084] The abnormal data in the preprocessed data is eliminated to obtain local standard data.

[0085] Among them, the interquartile range (IQR), also known as the four-point difference, is a method in descriptive statistics to determine the difference between the third quartile and the first quartile.

[0086] It should be noted that, by adopting the method provided in this embodiment, abnormal data of each participant can be eliminated, thereby ensuring the data quality of each participant.

[0087] Specifically, the preprocessing of local data of each participant includes: filling missing values ​​(using different filling methods according to the type of data, filling with 'unkown' values ​​for discrete variables and filling with 0 or median for continuous variables) to ensure data integrity; filtering duplicate data (deleting completely duplicate data and keeping only one copy); deleting features with high null value rates (by counting the null value rates of features, deleting features with null value rates exceeding 70%).

[0088] Furthermore, the anomaly identification of the preprocessed data is performed by using the interquartile range method, including:

[0089] Calculate the first quartile (Q1) and the third quartile (Q3);

[0090] Calculate the IQR (the difference between the third quartile and the first quartile, representing the middle 50% of the data set) value: ;

[0091] Find outliers, which are usually defined as values ​​smaller than Q1-1.5IQR or larger than Q3+1.5IQR.

[0092] This is because under the assumption of a normal distribution, approximately 99.7% of the data should fall within this range.

[0093] S202: The participant performs local model training based on local standard data to obtain an updated local model, and uploads model parameters of the updated local model to the collaborating party.

[0094] In a specific optional implementation, in step S202, the participant performs local model training based on local standard data, and the updated local model includes:

[0095] Normalize the local standard data to obtain training data;

[0096] Copy the global model parameters issued by the collaborator and extract training tasks from the task pool, where each training task contains a set of input-output pairs;

[0097] Perform local model training based on global model parameters and training tasks.

[0098] Specifically, the participants perform model training based on the local data set after removing abnormal data. In this embodiment, the inner loop update of meta-learning is adopted. The updates are all performed based on the local data set of the participants, and there is no need to communicate with the collaborating parties (meta-learning is an algorithm of learning algorithms, and its goal is to quickly adapt to new tasks, even if the amount of data for the new tasks is small. In meta-learning, there are two loops: an outer loop and an inner loop. The inner loop is mainly responsible for the model update of each task, while the outer loop is responsible for updating the meta parameters), including the following steps:

[0099] Step 1: Each participant needs to preprocess the sample data after removing abnormal data. Here, normalization is mainly performed to limit the data of each participant to a certain range to avoid data deviation. This ensures that the data of different participants have similar distribution.

[0100] Step 2: Copy the global model parameters, because each participant only performs local updates based on local data instead of changing the global model parameters. The global model parameters here are the initial model parameters sent by the collaboration party to each participant.

[0101] Step 3, task sampling: For each participant, one or more tasks are randomly selected from the local data set, and each task is divided into a training set and a test set.

[0102] Step 4, inner loop update: The participants use the training set combined with the global model parameters to perform gradient descent updates on each piece of data. Each update calculates the model loss based on the participant's local training data set and performs gradient descent based on this loss.

[0103] Step 5, evaluate the model effect: use the test set data to test the updated model effect and calculate the loss function, which will be used to update the global model parameters in the outer loop.

[0104] Step 6: Repeat steps 3 to 5 above until the termination conditions of local optimization are met, such as the number of iterations reaches the set value, the model performance is not significantly improved, etc., then the local meta-update can be completed (all calculations and updates in this step are performed locally on each participant, and no communication with the collaborating party is required).

[0105] Furthermore, in step 2, task sampling is to randomly select one or more tasks from the local data set for each participant, and each task is divided into a training set and a test set. Task sampling is a key step in the meta-learning process, which involves selecting one or more tasks from the local data set of each participant for training. The following are the specific implementation steps:

[0106] Defining the task: A task is usually defined by a specific class distribution, that is, input-output pairs (such as images and labels) drawn from a specific distribution.

[0107] Create a task pool: Each participant divides their local dataset into multiple tasks, each task contains a set of input-output pairs. For example, in an image classification problem, each task may consist of a set of images belonging to the same category.

[0108] Random sampling: Randomly select one or more tasks from each participant's task pool for training. This sampling is usually uniform, that is, each task has an equal probability of being selected.

[0109] Split into support set and query set: For each extracted task, further split its data into a support set (for training) and a query set (for testing).

[0110] Repeated sampling: During the entire meta-learning process, new random sampling of tasks is required each time the inner loop is updated. This ensures that the meta-learning algorithm can face diverse tasks, thereby enhancing its generalization ability.

[0111] Inner loop update: Participants use the training set combined with the global model parameters to perform gradient descent updates on each task. Each update is based on the training data set in each task of the participant to calculate the model loss and perform gradient descent based on this loss. For regression tasks, the loss function is expressed as follows:

[0112] ;

[0113] in, is the sample size, is the i-th input data, is the corresponding true value, The model is The predicted value of .

[0114] For classification tasks, the loss function is expressed as follows:

[0115] ;

[0116] in, The model is Input The predicted probability of belonging to the positive class.

[0117] In this embodiment, it is assumed that there are a series of participants in the entire federated learning model training process. , where each participant Has its own data distribution , for each participant , the parameters of the model First, it goes through the inner loop update, which usually involves multiple steps of gradient descent. Specifically, let Is on the participating side The loss function on the training set, the inner loop update is to find new parameters , so that exist The formula is as follows:

[0118] ;

[0119] in, is the learning rate, is the gradient of the instantaneous velocity function.

[0120] S203: The collaborating party receives the model parameters uploaded by each participant, and based on the received model parameters, screens the participants participating in each round of aggregation in a supervised integration manner to determine the participants participating in the aggregation.

[0121] In a specific optional implementation, in step S203, based on the received model parameters, the participants participating in each round of aggregation are screened in a supervised integration manner, and the participants participating in the aggregation are determined to include:

[0122] Determine the evaluation indicators;

[0123] Obtain parameter information associated with the evaluation indicators for each participant;

[0124] For each participant, based on the evaluation index and parameter information associated with the evaluation index, the evaluation score of the participant is calculated;

[0125] Participants whose evaluation scores exceed the preset threshold are considered as candidates;

[0126] All candidates are clustered by cluster analysis to obtain several categories;

[0127] Using the preset screening method, the participants involved in the aggregation are screened out from each category.

[0128] The evaluation indicators are determined based on the goals and requirements of federated learning, and are used to select participants. These indicators may include model performance indicators (such as accuracy, loss function value, etc.), data contribution, communication overhead, etc.

[0129] Collecting participant information refers to communicating with participants in federated learning and collecting relevant information of each participant, including model parameters or gradient updates, performance indicators, data volume, etc.

[0130] Participant evaluation refers to evaluating each participant's model and calculating its performance indicators. The performance of the participant's model can be evaluated using training data or validation data sets. Based on the selected indicators, a relative score or ranking is given.

[0131] S204: The collaborating party aggregates the model parameters uploaded by the participating parties to obtain updated global parameters, and sends the updated global parameters to each participating party for the next round of training until the global model of the collaborating party converges to obtain the target global model.

[0132] In a specific optional implementation, in step S204, the collaborating party aggregates the model parameters uploaded by the participants participating in the aggregation, and the updated global parameters include:

[0133] The following formula is used to aggregate the model parameters uploaded by the participants to obtain the updated global parameters:

[0134] ;

[0135] in, is the updated global parameter, is the global parameter of the previous round, is the model gradient of the kth participant in the aggregation, is the total number of participants involved in the aggregation, is the data volume of the kth participant in the aggregation, is the learning rate of the global model, is the total amount of data involved in the aggregation.

[0136] Specifically, the above formula is used to calculate whether the global communication round has been reached or the predetermined model accuracy has been achieved; if reached, the training is completed; otherwise, the collaborating party sends the model parameters aggregated by the federated learning to each participant, and goes to step three to continue the next round of training.

[0137] S205: The collaborating party sends the model parameters of the target global model to each participating party for privacy data processing.

[0138] Specifically, the collaborating party broadcasts the converged model to all participating parties for actual application reasoning.

[0139] In this embodiment, the participants use statistical analysis to remove abnormalities and preprocess local basic data to obtain local standard data; the participants perform local model training based on local standard data to obtain an updated local model, and upload the model parameters of the updated local model to the collaborating party; the collaborating party receives the model parameters uploaded by each participant, and based on the received model parameters, screens the participants participating in each round of aggregation through supervised integration to determine the participants participating in aggregation; the collaborating party aggregates the model parameters uploaded by the participants participating in aggregation to obtain updated global parameters, and sends the updated global parameters to each participant for the next round of training until the global model of the collaborating party converges to obtain the target global model; the collaborating party sends the model parameters of the target global model to each participant for privacy data processing. The local data of each participant is marked with abnormal values ​​by statistical analysis, so as to eliminate the abnormal data of each participant and ensure the data quality of each participant in the subsequent horizontal federated learning process. The use of supervised integration to screen the participants can ensure the fairness and diversity of the participants in federated learning. This helps prevent individual participants from monopolizing or over-relying on certain data sources, thereby increasing the robustness and reliability of the global model. At the same time, by selecting only a portion of the participants to participate in the aggregation, the communication and computing overhead in the federated learning process can be reduced. This can reduce the demand for network bandwidth and the computing load of the collaborators, improve time and resource efficiency, and also improve the performance of the overall federated learning model. This will improve the efficiency and security of privacy data processing.

[0140] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0141] Figure 3 The principle block diagram of the privacy data processing device based on federated learning is shown in one-to-one correspondence with the privacy data processing method based on federated learning in the above embodiment. Figure 3As shown, the privacy data processing device based on federated learning includes a data standardization module 31, a local training module 32, a participant screening module 33, an aggregation update module 34 and a data processing module 35. The functional modules are described in detail as follows:

[0142] The data standardization module 31 is used to remove abnormalities and preprocess the local basic data by means of statistical analysis to obtain local standard data;

[0143] A local training module 32, configured to perform local model training based on local standard data, obtain an updated local model, and upload model parameters of the updated local model to the collaborating party;

[0144] The participant screening module 33 is used to receive the model parameters uploaded by each participant, and based on the received model parameters, screen the participants participating in each round of aggregation in a supervised integration manner to determine the participants participating in the aggregation;

[0145] The aggregation update module 34 is used to aggregate the model parameters uploaded by the participants participating in the aggregation to obtain updated global parameters, and send the updated global parameters to each participant for the next round of training until the global model of the collaborating party converges to obtain the target global model;

[0146] The data processing module 35 is used to send the model parameters of the target global model to each participant for privacy data processing.

[0147] Optionally, the data standardization module 31 includes:

[0148] A collection unit, used to collect local data to be detected as local basic data;

[0149] A preprocessing unit, used for preprocessing local basic data to obtain preprocessed data;

[0150] An anomaly identification unit is used to identify anomalies of the preprocessed data by using an interquartile range method, and mark the data detected to have anomalies as abnormal data;

[0151] The abnormality elimination unit is used to eliminate abnormal data in the preprocessed data to obtain local standard data.

[0152] Optionally, the local training module 32 includes:

[0153] A normalization unit, used for normalizing the local standard data to obtain training data;

[0154] The task acquisition unit is used to copy the global model parameters issued by the collaborator and extract training tasks from the task pool, where each training task contains a set of input-output pairs;

[0155] The local training unit is used to perform local model training based on the global model parameters and training tasks.

[0156] Optionally, the participant screening module 33 includes:

[0157] An indicator determination unit, used for determining evaluation indicators;

[0158] An information acquisition unit, used to acquire parameter information associated with the evaluation index of each participant;

[0159] A score calculation unit, used to calculate the evaluation score of each participant based on the evaluation index and parameter information associated with the evaluation index;

[0160] A candidate party determination unit, used to select a participant whose evaluation score exceeds a preset threshold as a candidate party;

[0161] A clustering unit, used to cluster all candidate parties into several categories by cluster analysis;

[0162] The screening unit is used to screen out the participants participating in the aggregation from each category by using a preset screening method.

[0163] Optionally, the aggregation update module 34 includes:

[0164] The calculation unit is used to aggregate the model parameters uploaded by the participants participating in the aggregation using the following formula to obtain the updated global parameters:

[0165] ;

[0166] in, is the updated global parameter, is the global parameter of the previous round, is the model gradient of the kth participant in the aggregation, is the total number of participants involved in the aggregation, is the data volume of the kth participant in the aggregation, is the learning rate of the global model, is the total amount of data involved in the aggregation.

[0167] For the specific definition of the privacy data processing device based on federated learning, please refer to the definition of the privacy data processing method based on federated learning above, which will not be repeated here. Each module in the above-mentioned privacy data processing device based on federated learning can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0168] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0169] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components connected to the memory 41, the processor 42, and the network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0170] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0171] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as the program code of the privacy data processing method based on federated learning, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0172] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the program code stored in the memory 41 or process data, such as running the program code of the privacy data processing method based on federated learning.

[0173] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0174] The present application also provides another implementation, namely, providing a computer-readable storage medium, which stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned privacy data processing method based on federated learning.

[0175] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0176] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for processing private data based on federated learning, characterized in that: Applied to a horizontal federated learning system, the horizontal federated learning system is a CS architecture, S is a collaborator, C is a participant, and the privacy data processing method based on federated learning includes: Participants use statistical analysis to eliminate anomalies and pre-process local basic data to obtain local standard data; The participant trains the local model based on the local standard data to obtain an updated local model, and uploads the model parameters of the updated local model to the collaborating party; The collaborator receives the model parameters uploaded by each participant, and based on the received model parameters, screens the participants participating in each round of aggregation through a supervised integration method to determine the participants participating in the aggregation; The collaborating party aggregates the model parameters uploaded by the participating parties to obtain updated global parameters, and sends the updated global parameters to each participating party for the next round of training until the global model of the collaborating party converges and the target global model is obtained; The collaborating party sends the model parameters of the target global model to each participating party for privacy data processing; The participants use statistical analysis to remove anomalies and preprocess local basic data, and obtain local standard data including: Collect local data to be tested as local basic data; Perform data preprocessing on local basic data to obtain preprocessed data; Using the interquartile range method, anomaly identification is performed on the preprocessed data, and the data detected to have anomalies are marked as abnormal data; Eliminate abnormal data in the preprocessed data to obtain the local standard data; Wherein, based on the received model parameters, the participants participating in each round of aggregation are screened by means of supervised integration, and the participants participating in the aggregation are determined to include: Determine the evaluation indicators; Obtaining parameter information associated with the evaluation index for each participant; For each participant, based on the evaluation index and the parameter information associated with the evaluation index, calculate the evaluation score of the participant; The participants whose evaluation scores exceed the preset threshold are regarded as candidates; All candidates are clustered by cluster analysis to obtain several categories; Using the preset screening method, the participants involved in the aggregation are screened out from each category.

2. The method for processing private data based on federated learning according to claim 1, characterized in that: The participant performs local model training based on local standard data, and obtains an updated local model including: Normalize the local standard data to obtain training data; Copy the global model parameters issued by the collaborator and extract training tasks from the task pool, where each training task contains a set of input-output pairs; Perform local model training based on the global model parameters and the training task.

3. The method for processing private data based on federated learning according to claim 1, characterized in that: The collaborating party aggregates the model parameters uploaded by the participating parties, and obtains updated global parameters including: The following formula is used to aggregate the model parameters uploaded by the participants to obtain the updated global parameters: in, is the updated global parameter, is the global parameter of the previous round, is the model gradient of the kth participant in the aggregation, K is the total number of participants in the aggregation, is the data volume of the kth participant in the aggregation, is the learning rate of the global model, and n is the total amount of data involved in aggregation.

4. A private data processing device based on federated learning, executing the private data processing method based on federated learning according to any one of claims 1 to 3, characterized in that: include: The data standardization module is used to eliminate anomalies and preprocess local basic data by means of statistical analysis to obtain local standard data; A local training module is used to perform local model training based on local standard data to obtain an updated local model, and upload model parameters of the updated local model to the collaborating party; The participant screening module is used to receive the model parameters uploaded by each participant, and based on the received model parameters, screen the participants participating in each round of aggregation through a supervised integration method to determine the participants participating in the aggregation; The aggregation update module is used to aggregate the model parameters uploaded by the participants to obtain updated global parameters, and send the updated global parameters to each participant for the next round of training until the global model of the collaborating party converges and the target global model is obtained; The data processing module is used to send the model parameters of the target global model to each participant for privacy data processing.

5. The privacy data processing device based on federated learning according to claim 4, characterized in that: The data standardization module includes: A collection unit, used to collect local data to be detected as local basic data; A preprocessing unit, used for preprocessing local basic data to obtain preprocessed data; An anomaly identification unit is used to identify anomalies of the preprocessed data by using an interquartile range method, and mark the data detected to have anomalies as abnormal data; The abnormality elimination unit is used to eliminate the abnormal data in the preprocessed data to obtain the local standard data.

6. The privacy data processing device based on federated learning according to claim 4, characterized in that: The local training module includes: A normalization unit, used for normalizing the local standard data to obtain training data; The task acquisition unit is used to copy the global model parameters issued by the collaborator and extract training tasks from the task pool, where each training task contains a set of input-output pairs; A local training unit is used to perform local model training based on the global model parameters and the training task.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the privacy data processing method based on federated learning as described in any one of claims 1 to 3 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for processing privacy data based on federated learning as described in any one of claims 1 to 3 is implemented.

Images