A method and system for partitioning data sets in a federated learning scenario
By judging the consistency of data distribution in federated learning scenarios and using federated classification models to divide the data sets, the problems of low model accuracy and inaccurate performance evaluation caused by uneven data distribution are solved, and the training and evaluation effect of the model is improved.
Patent Information
- Application Number
- CN202011455586.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-10
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-12-10
AI Technical Summary
In the federated learning scenario, due to the uneven distribution of data provided by various institutions, when using these data directly for federated learning, the model accuracy is usually not high, and the data distribution of the verification data set and the test data set are inconsistent, resulting in the inaccurate evaluation of model performance. .
By judging the consistency of the data distribution provided by each federated learning participant, and using a federated classification model to divide the data into a validation set and a training set, the data distribution of the validation set is as consistent as possible with the test data set, and the federated classification models M1 and M2 are used. Data attribution prediction and label assignment are carried out, and data samples are selected as the verification set according to the prediction probability from high to low, and the remaining ones are used as the training set.
The consistency judgment and reasonable division of the data distribution of federated learning participants is realized, and the accuracy of performance evaluation and model performance of federated learning models are improved.
Smart Images

Figure CN112686388B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a data partitioning method, and in particular to a data set partitioning method and system in a federated learning scenario. Background Art
[0002] Federated machine learning, also known as federated learning, joint learning, and alliance learning, is a machine learning framework that can effectively help multiple institutions use data and conduct machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations. In the federated learning scenario, the data provided by various institutions are often unevenly distributed, and it is common that the data do not meet the same distribution conditions. If the data provided by various institutions is used directly for federated learning without any processing, the accuracy of the learned model is usually not high. Therefore, in federated learning, it is particularly necessary to ensure the consistency of the data distribution of the data provided by all parties.
[0003] When training a deep learning model, the dataset is pre-divided into a training set for model training and a validation set for verifying model performance. However, the evaluation of the actual performance of the model depends on the test dataset. Ideally, we hope that the data distribution of the validation dataset and the test dataset are consistent, so that the model performance can be better evaluated using the validation dataset during the model training process, and the evaluation result is similar to the evaluation result of the actual performance of the model using the test dataset. However, if the data distribution of the validation dataset and the test dataset is significantly different, the model performance of the same model verified by the validation dataset and the test dataset will get different results, and the model performance cannot be accurately evaluated. Therefore, in model training, how to divide the dataset so that the data distribution of the divided validation dataset is as consistent as possible with the data distribution of the test dataset becomes the key to ensuring the effect of model training. Summary of the invention
[0004] The present invention aims to find data samples with the most similar data distribution to the test data set in the data provided by each participant in federated learning as a verification set for model training, and provides a data set partitioning method and system in a federated learning scenario.
[0005] To achieve this object, the present invention adopts the following technical solutions:
[0006] A method for partitioning a data set in a federated learning scenario is provided, comprising the following steps:
[0007] Step S1, determining whether the data distribution of the original data provided by each federated learning participant is consistent;
[0008] Step S2, using the original data with consistent data distribution provided by each federated learning participant, as well as model test data training, and using the validation set to optimize to obtain the optimal federated classification model M1;
[0009] Step S3, inputting the original data belonging to each of the federated learning participants into the federated classification model M1, and the federated classification model M1 outputs the probability that the model input data is the model test data;
[0010] Step S4, selecting a specified number of the model input data from high to low according to the prediction probability and dividing them into a verification set provided by the federated learning participant to which the data belongs for verifying the model performance, and dividing the remaining model input data into a training set provided by the federated learning participant to which the data belongs for training the model.
[0011] Preferably, in step S1, the method for determining whether the data distribution of the original data provided by each of the federated learning participants is consistent specifically includes:
[0012] Step S11, dividing the original data provided by the federated learning participant into a training set, a validation set, and a test set consistent with the data distribution of the original data;
[0013] Step S12, assigning corresponding data labels to the divided training sets and validation sets belonging to each of the federated learning participants;
[0014] Step S13, using the training set with data labels belonging to each of the federated learning participants for training, and using the validation set for optimization to obtain the optimal federated classification model M2;
[0015] Step S14, inputting the test sets belonging to each of the federated learning participants into the federated classification model M2, and obtaining a number of local performance evaluation indicators of the federated classification model M2 for distinguishing the input data of each party;
[0016] Step S15, aggregate calculation is performed on the values of the local performance evaluation indicators obtained by the federal classification model M2 to distinguish the input data ownership, obtain a global evaluation indicator value, and judge whether the data distribution of the original data provided by each of the federal learning participants is consistent based on the global evaluation indicator value.
[0017] The present invention also provides a data set partitioning system in a federated learning scenario, the data set partitioning system comprising:
[0018] The data distribution consistency judgment module is used to judge whether the data distribution of the original data provided by each federated learning participant is consistent;
[0019] A data label assignment module, used to assign corresponding data labels to the original data provided by each federated learning participant with consistent data distribution and store them, and to assign corresponding data labels to model test data and store them;
[0020] A data acquisition module, connected to the data label assignment module, used to acquire the original data after label assignment as a model training sample, and acquire the model test data as a model verification sample;
[0021] M1 federated classification model training module, connected to the data acquisition module, used to use the original data and the model test data provided by each federated learning participant to train and use the validation set to optimize and obtain the optimal federated classification model M1;
[0022] An M1 model performance test module, connected to the data acquisition module and the M1 federated classification model training module, respectively, for inputting the acquired original data belonging to each of the federated learning participants into the federated classification model M1, and the federated classification model M1 outputs the probability that the model input data is the model test data;
[0023] A validation set selection module is connected to the M1 model performance test module and the data acquisition module, and is used to select a specified number of the model input data from high to low according to the prediction probability as a validation set provided by the federated learning participant to which the data belongs for verifying the model performance, and the remaining model input data is used as a training set provided by the federated learning participant to which the data belongs for training the model.
[0024] Preferably, the data distribution consistency judgment module specifically includes:
[0025] A data partitioning unit, used to partition the original data provided by each of the federated learning participants into a training set, a validation set, and a test set consistent with the data distribution of the original data;
[0026] A data label assignment unit, connected to the data division unit, for assigning corresponding data labels to the divided training sets and validation sets belonging to each of the federated learning participants, and assigning corresponding data labels to the model test data;
[0027] An M2 federated classification model training unit, connected to the data labeling unit, used to train the training set with data labels belonging to each of the federated learning participants and to optimize the validation set to obtain the optimal federated classification model M2;
[0028] An M2 model performance testing unit, connected to the data partitioning unit and the M2 federated classification model training unit, respectively, for inputting the test sets belonging to each of the federated learning participants into the federated classification model M2, and obtaining a number of local performance evaluation indicators of the federated classification model M2 for distinguishing the input data of each belonging party;
[0029] A numerical aggregation calculation unit, connected to the M2 model performance test unit, is used to perform aggregation calculation on the values of the local performance evaluation indicators obtained by the federated classification model M2 to distinguish the input data ownership party, and obtain a global evaluation indicator value;
[0030] A data distribution consistency judgment unit is connected to the numerical aggregation calculation unit and is used to judge whether the data distribution of the original data provided by each of the federated learning participants is consistent according to the global evaluation index value.
[0031] The beneficial effects of the present invention are:
[0032] 1. It realizes the effective judgment on whether the data distribution of the original data provided by each federated learning participant is consistent;
[0033] 2. The original data provided by each federated learning participant can be reasonably divided into a training set and a validation set. The data distribution of the validation set is the same or similar to that of the test set, which is conducive to improving the model performance of the federated learning model. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0035] Figure 1 is a step diagram of a method for partitioning a data set in a federated learning scenario provided by an embodiment of the present invention;
[0036] Figure 2 It is a principle diagram of dividing a data set according to a data set dividing method in a federated learning scenario provided by an embodiment of the present invention;
[0037] Figure 3 is a step diagram of a method for determining whether the data distribution of the original data provided by each federated learning participant is consistent according to the present invention;
[0038] Figure 4 It is a schematic diagram of the present invention for judging whether the data distribution of the original data provided by each federated learning participant is consistent;
[0039] Figure 5 It is a system structure diagram of a data set partitioning system in a federated learning scenario provided by an embodiment of the present invention;
[0040] Figure 6 It is a schematic diagram of the internal structure of the data distribution consistency judgment module in the data set partitioning system. DETAILED DESCRIPTION
[0041] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.
[0042] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on this patent; in order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.
[0043] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "inner", "outer", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as a limitation on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0044] In the description of the present invention, unless otherwise clearly specified and limited, if the term "connection" or the like appears to indicate the connection relationship between components, the term should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two components or the interaction relationship between two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0045] An embodiment of the present invention provides a method for partitioning a data set in a federated learning scenario, such as Figure 1 As shown, the following steps are included:
[0046] Step S1, determining whether the data distribution of the original data provided by each federated learning participant is consistent;
[0047] Step S2, use the original data with consistent data distribution provided by each federated learning participant and the model test data for training, and use the validation set to optimize to obtain the optimal federated classification model M1; in order to distinguish the owner of the original data and the original data and the model test data, before training the federated classification model M1, the present invention first assigns corresponding data labels to the original data provided by each federated learning participant, and assigns corresponding data labels to the model test data. For example, the original data provided by federated learning participant A is recorded as A-data, and the original data provided by federated learning participant B is recorded as B-data. A-data and B-data can be assigned the same data label 0, recorded as A-data|0 and B-data|0 respectively; the model test data is recorded as test, and the test data can be assigned a label 1, recorded as test|1.
[0048] Step S3, input the original data belonging to each federated learning participant into the federated classification model M1, and the federated classification model M1 outputs the probability that the model input data is the model test data; for example, input the original data A-data provided by federated learning participant A into the model M1, and the model M1 will output the probability that A-data is the model test data;
[0049] Step S4, select a specified number of model input data according to the predicted probability from high to low and divide it into a verification set provided by the federated learning participant to which the data belongs for verifying the model performance, and the remaining model input data is divided into a training set provided by the federated learning participant to which the data belongs for training the model. For example, when the data input into the model M1 in step S3 is A-data, select a specified number of data from the original data A-data provided by the federated learning participant A according to the predicted probability from high to low and divide it into a verification set provided by the federated learning participant A for verifying the model performance (for example, select 20% of A-data as the verification set according to the predicted probability from high to low), and the remaining 80% of the model input data is divided into a training set provided by the federated learning participant A for training the model.
[0050] The following example takes the two federated learning participants A and B as examples, and combines Figure 2 , the data set partitioning method provided by the present invention is specifically described:
[0051] like Figure 2As shown, the original data provided by federated learning participant A is recorded as A-data, and the original data provided by federated learning participant B is recorded as B-data. First, the data label 0 is added to the original data provided by federated learning participants A and B, recorded as A-data|0 and B-data|0, and the label 1 is added to the model test data, recorded as test-1. Then, A-data|0, B-data|0 and test|1 are used for federated learning to train and optimize to form the optimal federated learning classification model M1. Model M1 is used to distinguish whether the input data is model test data or whether it can be used as model test data.
[0052] After the training of model M1 is completed, data A-data and B-data are input into model M1 for scoring prediction, and the probability that the input data belongs to or can be used as model test data is output, and the probability value is between 0 and 1. The input data is sorted from high to low according to the probability value output by the model. The higher the score, the closer the data distribution of the input data is to the model test data. Finally, according to the specific needs of model training, a suitable number of data samples are selected from the input data as the validation set for model training. For example, 20% of the input data is selected from the probability value from high to low as the validation set provided by the federated learning participant to which the data belongs, and the remaining 80% is used as the training set provided by the federated learning participant to which the data belongs. In this way, the original data A-data provided by federated learning participant A is split into training set A-train and validation set A-valid, and the original data B-data provided by federated learning participant B is split into training set A-train and validation set B-valid.
[0053] In step S1, the method for determining whether the data distribution of the original data provided by each federated learning participant is consistent is as follows: Figure 3 As shown, specifically including:
[0054] Step S11, dividing the original data provided by each federated learning participant into a training set, a validation set, and a test set that are consistent with the data distribution of the original data; for example, the original data provided by federated learning participant A is recorded as A-data, and the original data A-data is divided into a training set A-train, a validation set A-valid, and a test set A-test that are consistent with the data distribution of the original data; the original data provided by federated learning participant B is recorded as B-data, and the original data B-data is divided into a training set B-train, a validation set B-valid, and a test set B-test;
[0055] Step S12, assigning corresponding data labels to the divided training sets belonging to each federated learning participant, for example, adding data label 0 to the training set A-train provided by federated learning participant A, recorded as A-train|0, and adding data label 1 to the training set B-train provided by federated learning participant B, recorded as B-train|1;
[0056] Step S13, use the training sets with data labels belonging to each federated learning participant, for example, when there are only A and B federated learning participants, use the training sets A-train|0 and B-train|1 to train to form a federated classification model M2; then input the validation sets belonging to each federated learning participant, such as the test sets A-valid and B-valid provided by federated learning participants A and B respectively, into the model M2 to evaluate the model performance of the model M2 in distinguishing the ownership of the input data, and obtain the optimal federated classification model M2.
[0057] Step S14, input the test sets A-test and B-test into the model M2, and obtain a number of local performance evaluation indicators of the model M2 for distinguishing the input data of each attribution party;
[0058] Step S15, aggregate the values of local performance evaluation indicators obtained by the federated classification model M2 to distinguish the input data owners, obtain a global evaluation indicator value, and based on the global evaluation indicator value, determine whether the data distribution of the original data provided by each federated learning participant is consistent.
[0059] The following example takes the case where there are only two federated learning participants, A and B, and combines Figure 4 , the method for judging whether the data distribution of the original data provided by each federated learning participant is consistent is specifically described:
[0060] There are two existing federated learning participants, A and B, and the data distribution of the original data provided by A and B is inconsistent. First, the original data provided by A is divided into a training set, a validation set, and a test set that are consistent with the distribution of the original data, and are recorded as A-train, A-valid, and A-test respectively. Similarly, the original data provided by B is divided into a training set, a validation set, and a test set that are consistent with the original data, and are recorded as B-train, B-valid, and B-test respectively, and data labels 0 are added to A-train and A-valid, recorded as A-train|0 and A-valid|0, and data labels 1 are added to B-train and B-valid, recorded as B-train|1 and B-valid|1. Then, A-train|0 and B-train|1 are used as training samples for training, and A-valid|0 and B-valid|1 are used as validation sets for optimization to obtain the optimal federated classification model M2. Model M2 is used to distinguish whether the input data belongs to A or B.
[0061] After the training of model M2 is completed, A-test and B-test are input into model M2 to obtain the evaluation index AUC (AUC (Area Under Cruve) is defined as the area under the ROC curve and the area enclosed by the coordinate axis. It is a commonly used local performance evaluation index for evaluating the prediction performance of the model. In the present invention, the AUC evaluation index is used to evaluate the probability that the input data is provided by the corresponding party), and the evaluation index AUC for evaluating whether B-test belongs to party B is obtained. Then, the index values of the two AUC indicators are aggregated, and based on the aggregation results, for example, the threshold judgment method is used to determine whether the data distribution of the original data provided by party A and party B is consistent. The threshold judgment method is specifically as follows:
[0062] Determine whether the difference between the value of the accumulated value of each performance evaluation index AUC and 0.5 is less than the threshold δ and greater than -δ.
[0063] If so, it is determined that the data distribution of the original data provided by each federated learning participant is consistent;
[0064] If not, it is determined that the data distribution of the original data provided by each federated learning participant is inconsistent.
[0065] More specifically, when the difference between the value of the numerical accumulation result of each performance evaluation index AUC and 0.5 is greater than or equal to the threshold δ, it means that the data distribution of the original data provided by each federated learning participant is inconsistent. When the difference between the value of the numerical accumulation result of each performance evaluation index AUC and 0.5 is less than or equal to -δ, it means that the model performance of the federated classification model M2 is unqualified and the model M2 needs to be retrained.
[0066] The threshold δ is obtained based on empirical summary. When there are only two federated learning participants participating in federated learning, the threshold δ is preferably equal to 0.2.
[0067] The present invention also provides a data set partitioning system in a federated learning scenario, which can implement the above-mentioned data set partitioning method, such as Figure 5 As shown, the data set partitioning system includes:
[0068] The data distribution consistency judgment module is used to judge whether the data distribution of the original data provided by each federated learning participant is consistent;
[0069] The data label assignment module is used to assign corresponding data labels to and store the original data provided by each federated learning participant with consistent data distribution. For example, the original data A-data provided by federated learning participant A is assigned label 0, and the model test data is assigned corresponding data labels to and stored. For example, the model test data test is assigned data label 1.
[0070] The data acquisition module is connected to the data label assignment module and is used to obtain the original data after label assignment as the model training sample and obtain the model test data as the model verification sample;
[0071] The M1 federated classification model training module is connected to the data acquisition module and is used to use the original data and model test data provided by each federated learning participant for training and to use the validation set for optimization to obtain the optimal federated classification model M1;
[0072] The M1 model performance test module is connected to the data acquisition module and the M1 federated classification model training module, respectively, and is used to input the acquired original data belonging to each federated learning participant into the federated classification model M1. The federated classification model M1 outputs the probability that the model input data is model test data or can be used as model test data;
[0073] The validation set selection module is connected to the M1 model performance test module and is used to select a specified number of model input data from high to low prediction probabilities as the validation set provided by the federated learning participant to which the data belongs for verifying the model performance, and the remaining model input data as the training set provided by the federated learning participant to which the data belongs for training the model. The validation set and training set selection methods are specifically described in the above-mentioned data set division method and will not be repeated here.
[0074] like Figure 6 As shown, the data distribution consistency judgment module specifically includes:
[0075] A data partitioning unit, used to partition the original data provided by each federated learning participant into a training set, a validation set, and a test set consistent with the data distribution of the original data;
[0076] The data label assignment unit is connected to the data partitioning unit and is used to assign corresponding data labels to the training set and the validation set belonging to each federated learning participant, and to assign corresponding data labels to the model test data; the data label assignment method is specifically described in the above-mentioned data set partitioning method and will not be repeated here;
[0077] The M2 federated classification model training unit is connected to the data labeling unit and is used to train the training set with data labels belonging to each federated learning participant and optimize the validation set to obtain the optimal federated classification model M2;
[0078] The M2 model performance testing unit is connected to the data partitioning unit and the M2 federated classification model training unit, respectively, and is used to input the test sets belonging to each federated learning participant into the federated classification model M2, and obtain a number of local performance evaluation indicators of the federated classification model M2 for distinguishing the input data of each belonging party;
[0079] A numerical aggregation calculation unit, connected to the M2 model performance test unit, is used to aggregate the index values of various local performance evaluation indicators obtained by the federated classification model M2 to distinguish the input data ownership party, and obtain a global evaluation index value;
[0080] The data distribution consistency judgment unit is connected to the numerical aggregation calculation unit and is used to judge whether the data distribution of the original data provided by each federated learning participant is consistent based on the global evaluation index value.
[0081] It should be noted that the above specific implementations are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art should understand that various modifications, equivalent substitutions, changes, etc. can be made to the present invention. However, as long as these changes do not deviate from the spirit of the present invention, they should be within the scope of protection of the present invention. In addition, some terms used in the specification and claims of this application are not restrictive, but are only for the convenience of description.
Claims
1. A method for partitioning a data set in a federated learning scenario, characterized in that: The steps include: Step S1, judging whether the data distribution of the original data provided by each federated learning participant is consistent, including: step S11, dividing the original data provided by the federated learning participant into a training set, a validation set and a test set that are consistent with the data distribution of the original data; step S12, assigning corresponding data labels to the divided training set and validation set belonging to each of the federated learning participants; step S13, using the training set with data labels belonging to each of the federated learning participants for training, and using the validation set for optimization to obtain the optimal federated classification model M2; step S14, inputting the test set belonging to each of the federated learning participants into the federated classification model M2, and obtaining a number of local performance evaluation indicators of the federated classification model M2 for distinguishing the input data of each attribution party; step S15, performing aggregation calculation on the values of the local performance evaluation indicators obtained by the federated classification model M2 for distinguishing the attribution party of the input data, obtaining a global evaluation indicator value, and judging whether the data distribution of the original data provided by each of the federated learning participants is consistent according to the global evaluation indicator value; Step S2, using the original data with consistent data distribution provided by each federated learning participant and the model test data for training, and using the validation set for optimization to obtain the optimal federated classification model M1; Step S3, inputting the original data belonging to each of the federated learning participants into the federated classification model M1, and the federated classification model M1 outputs the probability that the model input data is the model test data; Step S4, selecting a specified number of the model input data from high to low according to the prediction probability and dividing them into a verification set provided by the federated learning participant to which the data belongs for verifying the model performance, and dividing the remaining model input data into a training set provided by the federated learning participant to which the data belongs for training the model; wherein, a higher prediction probability score indicates that the input data is closer to the data distribution of the model test data.
2. A data set partitioning system in a federated learning scenario, which can implement the data set partitioning method as claimed in any one of claims 1, characterized in that: The data set partitioning system comprises: A data distribution consistency judgment module, used to judge whether the data distribution of the original data provided by each federated learning participant is consistent, including: a data division unit, used to divide the original data provided by each federated learning participant into a training set, a validation set and a test set that are consistent with the data distribution of the original data; a data label assignment unit, connected to the data division unit, used to assign corresponding data labels to the divided training set and validation set belonging to each federated learning participant, and to assign corresponding data labels to the model test data; a M2 federated classification model training unit, connected to the data label assignment unit, used to train the training set with data labels belonging to each federated learning participant, and use the validation set to optimize to obtain the optimal federated classification model M2; M 2. A model performance testing unit, connected to the data partitioning unit and the M2 federated classification model training unit, respectively, for inputting the test set belonging to each federated learning participant into the federated classification model M2, and obtaining a number of local performance evaluation indicators of the federated classification model M2 for distinguishing the input data of each attribution party; a numerical aggregation calculation unit, connected to the M2 model performance testing unit, for performing aggregation calculation on the values of the local performance evaluation indicators obtained by the federated classification model M2 for distinguishing the attribution party of the input data, and obtaining a global evaluation indicator value; a data distribution consistency judgment unit, connected to the numerical aggregation calculation unit, for judging whether the data distribution of the original data provided by each federated learning participant is consistent according to the global evaluation indicator value; A data label assignment module, used to assign corresponding data labels to the original data provided by each federated learning participant with consistent data distribution and store them, and to assign corresponding data labels to model test data and store them; A data acquisition module, connected to the data label assignment module, used to acquire the original data after label assignment as a model training sample, and acquire the model test data as a model verification sample; M1 federated classification model training module, connected to the data acquisition module, used to use the original data and the model test data provided by each federated learning participant for training, and use the validation set to optimize and obtain the optimal federated classification model M1; An M1 model performance test module, connected to the data acquisition module and the M1 federated classification model training module, respectively, for inputting the acquired original data belonging to each of the federated learning participants into the federated classification model M1, and the federated classification model M1 outputs the probability that the model input data is the model test data; A verification set selection module is connected to the M1 model performance test module and the data acquisition module, and is used to select a specified number of the model input data from high to low according to the prediction probability as the verification set provided by the federated learning participant to which the data belongs for verifying the model performance, and the remaining model input data as the training set provided by the federated learning participant to which the data belongs for training the model; wherein, a higher prediction probability score indicates that the input data is closer to the data distribution of the model test data.
