Gradient-based method for assessing completeness of data set

Abstract

Disclosed in the present invention is a gradient-based method for assessing the completeness of a data set. The method comprises the following steps: setting a base data set at a client, and using the base data set to participate in federated training, so as to obtain gradient information; performing gradient inversion on the gradient information, so as to obtain data information of the client; and packaging the data information to obtain a data set, putting the data set into a base model to test the accuracy thereof, and determining the completeness of the data set according to the accuracy. In the method, information of a gradient uploaded by a client can be used to obtain an original real data set of the client by means of inversion, and data obtained by means of inversion is then made into a test set, which is then put into a base model to test the accuracy thereof, such that the completeness of the data set of the client can be accurately determined.

Description

A gradient-based dataset completeness assessment method Technical Field

[0001] The present invention belongs to the field of data quality assessment, and in particular relates to a gradient-based data set integrity assessment method. Background Art

[0002] Federated learning is an emerging distributed machine learning paradigm used to collaboratively train a global model across a large number of clients. To attract potential high-quality data holders to participate in federated learning, incentives are provided based on the estimated contribution of each client's data to the federated collaboration. Unfortunately, some clients exploit the inability of existing evaluation metrics to fully quantify their contributions, falsely claiming to possess qualified data or to have genuinely completed local deep learning. These clients deceive the server by fabricating their contributions, thereby obtaining incentives that are not theirs.

[0003] Considering that the gradients uploaded by the client contain a large amount of information, for example, L Zhu et al. discovered that shared gradients in distributed machine learning are not secure. They designed several steps to easily obtain the original dataset, thereby compromising client privacy. J Geiping et al. achieved high-resolution image reconstruction using shared gradients in federated learning and demonstrated that trained neural networks also face this risk. H Yin et al., building on previous work, further demonstrated that gradient recovery can be used in neural networks with more than 50 layers and larger batch sizes.

[0004] Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a gradient-based dataset completeness assessment method to address the problem that the prior art cannot fully quantify its contribution.

[0006] To achieve the above object, the present invention provides a gradient-based dataset integrity assessment method, comprising the following steps:

[0007] Set a basic data set on the client, use the basic data set to participate in federated training, and obtain gradient information;

[0008] Performing gradient inversion on the gradient information to obtain client data information;

[0009] The data information is packaged into a data set and put into the basic model to test its accuracy, and the completeness of the data set is judged according to the accuracy.

[0010] Preferably, the basic data set includes a noise data set, a partially complete data set, and a complete data set;

[0011] The completeness of the partially complete data set includes 0%, 20%, 40%, 60%, 80%, and 100%.

[0012] Preferably, the gradient inversion process includes:

[0013] Calculate the gradient of the data in the client using the gradient formula, divide the gradient evenly, and update the divided gradient to obtain the gradient of the next round;

[0014] Randomly initialize a virtual input and label input, and input the initialized virtual input and label input into the model to obtain a virtual gradient;

[0015] The virtual gradient is optimized based on the gradient of the next round to obtain the real training data.

[0016] Preferably, the gradient inversion process further includes:

[0017] When the gradient of a certain step is obtained, the restored dataset samples and labels are obtained by minimizing the formula based on the virtual gradient.

[0018] Preferably, the method for obtaining the basic model includes: constructing a model, and training the constructed model using a basic data set to obtain the basic model.

[0019] Preferably, the packaged format includes CIFAR-10.

[0020] Preferably, the method for judging the completeness of the data set according to the accuracy rate includes: setting the accuracy rate threshold to 85%, when the accuracy rate is greater than or equal to 85%, judging that the data set is complete, and when the accuracy rate is less than 85%, judging that the data set is incomplete.

[0021] Compared with the prior art, the present invention has the following advantages and technical effects:

[0022] This paper proposes a gradient-based dataset integrity assessment method. This method uses information about the gradients uploaded by the client to invert the client's actual dataset. Furthermore, a base dataset and base model are added to the federated learning server. The inverted data is used as a test set and then tested on the base model to determine its accuracy. This allows for accurate assessment of the client's dataset integrity. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0024] FIG1 is a flow chart of a gradient-based dataset integrity assessment method according to an embodiment of the present invention;

[0025] FIG2 is a graph showing changes in accuracy versus completeness according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] Example 1

[0029] As shown in Figure 1, the present invention proposes a gradient-based dataset integrity assessment method. This method adds a basic dataset D to the traditional federated learning framework. base , basic model M base . Basic dataset D base :Different from the traditional federated learning setting where all data comes from the client, we will provide a smaller, clean and reliable basic dataset D for the server under this framework. base In order to obtain a basis for evaluating the integrity of client data, the server will collect data sets from each client before the federated learning training begins. base Contains about 10% of the total amount of data from each client sample, and the collected data samples will not participate in the local model calculation on the client. At the same time, in order to protect the privacy of the client, the dataset is not transparent to other clients. Basic model M base :It is the server using the basic data set D base The model with higher accuracy is obtained through training. The basic model is based on the basic data set D base The high-accuracy model obtained after ResNet18 training has an accuracy of 93.8%.

[0030] The specific steps are as follows:

[0031] 1. Set up datasets of different proportions: noisy datasets, partially complete datasets, and complete datasets. Six datasets with different levels of completeness are set: 0%, 20%, 40%, 60%, 80%, and 100%. The client will use these six datasets to participate in federated learning training and send gradient information to the server.

[0032] 2. Before the server formally aggregates: the server will use the gradient information to invert the client's data information. Afterwards, the server will package the inverted data into a dataset in the CIFAR-10 format and put it into the M base Test its accuracy, the results are shown in Figure 2:

[0033] 3. If the accuracy of the dataset generated by the client through gradient inversion is higher than 85% when tested in the basic model, federated learning will continue. Otherwise, the gradient uploaded by the client will be excluded from the global aggregation of this round.

[0034] The steps of the gradient inversion algorithm are as follows:

[0035] A set of data on the client is (x t，i ,y t，i ), where x t,i is the data sample of client i in round t, y t,i is the corresponding label. Then the gradient of client i in round t can be expressed as formula 1:

[0036] in, Represents the loss function l about the model output F(x t，i , W t ) and label y t，i For the model parameter W t The purpose of this calculation is to determine the direction and degree of change of the loss function relative to the model parameters. represents partial derivative, l represents loss function, W t Represents the model parameters sent by the t-round server, F(x t,i ,W t ) represents a differentiable model.

[0037] After obtaining the gradient of client i in round t, the gradient is averaged into N parts, and then the gradient of the next round is updated. For details, see formulas (2) and (3):

[0038] Here, η represents the learning rate, which is a hyperparameter used to control the step size or speed of each model parameter update. It determines the magnitude of the model parameter update in each iteration.

[0039] Now you can get the gradient information of client k from the federation The goal is to follow the gradient Recover data (x t,k ,y t,k). In order to recover data from the gradient, the evaluation layer first randomly initializes a virtual input x' and label input y', which is the first line in Algorithm 1. Then these "virtual data" are input into the model and the "virtual gradient" is obtained, as shown in Equation (4):

[0040] Among them, W represents the randomly initialized model parameters, the purpose of which is to initialize the virtual gradient.

[0041] Next, optimize the virtual gradient close to the original gradient. This optimization also makes the virtual data close to the real training data x' * ,y' * If you already have a gradient for a certain step Then we can get the restored dataset samples and labels x' by minimizing formula (5) * ,y' * .

[0042] in, Indicates the value of the parameter that makes the function reach the minimum value. arg min is a combination of parameters (x', y') and min (minimum value), which means the parameter value when solving the function to reach the minimum value.

[0043] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A gradient-based dataset integrity assessment method, characterized in that: The following steps are involved: Set a basic data set on the client, use the basic data set to participate in federated training, and obtain gradient information; Performing gradient inversion on the gradient information to obtain data information of the client; The data information is packaged into a data set and put into the basic model to test its accuracy, and the completeness of the data set is determined based on the accuracy.

2. The gradient-based dataset integrity assessment method according to claim 1, characterized in that: The basic data set includes a noise data set, a partially complete data set, and a complete data set; The completeness of the partially complete data set includes 0%, 20%, 40%, 60%, 80%, and 100%.

3. The gradient-based dataset integrity assessment method according to claim 1, characterized in that: The process of gradient inversion includes: Calculate the gradient of the data in the client using the gradient formula, divide the gradient evenly, and update the divided gradient to obtain the gradient of the next round; Randomly initialize a virtual input and label input, and input the initialized virtual input and label input into the model to obtain a virtual gradient; The virtual gradient is optimized based on the gradient of the next round to obtain the real training data.

4. The gradient-based dataset integrity assessment method according to claim 3, characterized in that: The gradient inversion process also includes: When the gradient of a certain step is obtained, the restored dataset samples and labels are obtained by minimizing the formula based on the virtual gradient.

5. The gradient-based dataset integrity assessment method according to claim 1, characterized in that: The method for obtaining the basic model includes: constructing a model, and training the constructed model through a basic data set to obtain the basic model.

6. The gradient-based dataset integrity assessment method according to claim 1, characterized in that: The packaged formats include CIFAR-10.

7. The gradient-based dataset integrity assessment method according to claim 1, characterized in that: The method for judging the completeness of a data set according to the accuracy rate includes: setting an accuracy rate threshold to 85%, when the accuracy rate is greater than or equal to 85%, judging that the data set is complete, and when the accuracy rate is less than 85%, judging that the data set is incomplete.

Images