A risk assessment method for model gradient leakage during learning

Abstract

The present invention relates to the field of artificial intelligence technology and discloses a method for assessing the risk of model gradient leakage input information during a learning process. The method comprises the following steps: S1: sampling a local dataset to obtain a local sampled dataset; S2: analyzing the local sampled dataset to obtain the parameter gradients of the local sampled data samples; S3: calculating the mutual information value between the local sampled data samples and the sampled sample parameter gradients; and S4: when the mutual information value reaches a set level, issuing a privacy leakage risk alert; otherwise, uploading the parameter gradients to a server. Compared with existing gradient leakage input indicators, the gradient information ratio in this method for assessing the risk of model gradient leakage input information during a learning process can measure the risk of each sample in a batch of samples being leaked by the batch sample gradient. Since the privacy importance of different samples varies, participants will introduce a privacy protection mechanism for the uploaded batch sample gradients.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for assessing the risk of model gradient leakage input information during a learning process. Background Art

[0002] Recent research on the effectiveness of privacy protection in federated learning has confirmed that input data can be reconstructed based on model gradients through certain means. This means that the privacy protection mechanism of shared model gradients in federated learning still carries the risk of data leakage. Different input samples have different sample feature values, and the risk of leakage in federated learning also varies. We have invented a new metric to measure the risk of input sample leakage in federated learning frameworks based on deep learning models. This metric can be used to evaluate and screen input sample data and assess the privacy security of federated learning frameworks during their construction and application.

[0003] According to Chinese patent CN112765559A, in the prior art, the mutual information value between a sample and its gradient is calculated using a pre-trained statistical model to measure the risk of gradient privacy leakage. This technique requires a pre-trained statistical model as a prerequisite, and the model's performance can affect the accuracy of the mutual information calculation. In actual gradient leakage data attacks, the target sample is a batch of samples, which typically contains multiple samples. The data thief can only obtain the batch sample gradient of the target participant and cannot know the gradient corresponding to each sample in the batch. Therefore, the only way to reconstruct the batch sample is to reverse the batch sample gradient. The degree of sample feature restoration in the reconstructed batch sample varies, that is, the degree to which the target sample feature is leaked varies. This is mainly determined by the ratio of the sample's gradient to the information content of the batch sample gradient. The larger the gradient information content ratio, the more dominant the sample gradient is in the batch sample gradient. When using the batch sample gradient for reverse analysis, it is easier to reconstruct the sample feature. The prior art does not effectively reflect this correlation feature. Summary of the Invention

[0004] The purpose of this invention is to provide a risk assessment method for model gradient leakage during the learning process. This method addresses the issue of batch gradients shared by participants in federated learning leaking local input data features. Different input samples, due to differences in feature information content and feature distribution, have varying degrees of leakage risk in different federated learning models. Therefore, it is necessary to design an evaluation metric for the amount of input feature information leaked by batch gradients to quantify the risk of feature information leakage for individual samples in the batch, thereby supporting security assessments of federated learning frameworks constructed using gradient-based deep learning models.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for assessing the risk of model gradient leakage of input information during learning includes the following steps:

[0007] S1: Sample the local dataset to obtain a local sampled dataset.

[0008] S2: Analyze the local sampled data set to obtain the parameter gradient of the local sampled data sample.

[0009] S3: Calculate the mutual information value of the local sampled data sample and the sampled sample parameter gradient.

[0010] S4: When the mutual information value reaches the set level, a privacy leakage risk reminder is issued; otherwise, the parameter gradient is uploaded to the server.

[0011] Preferably, in S1, it is assumed that a participant has a batch of samples (X, Y)≡((x (1) ,y (1) ),(x (2) ,y (2) ),...,(x (B) ,y (B) )), the batch size is B, where (x (i) ,y (i) ) represents the i-th sample in the batch, x (i) represents the sample characteristics, y (i) Represents the sample label.

[0012] Preferably, the batch samples (X, Y) in S2 are input into the neural network model to obtain their corresponding batch sample gradients G and upload them to the parameter server to achieve the purpose of jointly building the model without leaving the local data.

[0013] Preferably, the batch sample gradient G uploaded by the participant in S3 may be reconstructed into a local batch sample (X, Y) by the parameter server or other participants through reverse engineering. However, the possibility of each sample in the batch sample being reconstructed is different. In order to quantify the risk of sample feature reconstruction, a risk assessment method for measuring the leakage of different sample features in the batch sample is proposed: gradient information ratio,

[0014] The batch sample gradient G has a mean relationship with the gradient of each sample in the batch sample:

[0015]

[0016] Preferably, in S3, if the gradient G of a certain sample (i)The greater the amount of information in the batch sample gradient G, the greater the possibility of the sample being leaked. Therefore, we need to measure the possibility of the corresponding sample being leaked by calculating the amount of information in the sample gradient of the batch sample gradient. The gradient is a high-dimensional vector with two properties: size and direction. The vector of each sample needs to be projected onto the batch sample gradient vector to achieve comparison in the same direction and scale, and measure the size of the information in the gradient information of each sample in the batch sample gradient.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] This risk assessment method for model gradient leakage input information during the learning process, compared with the existing gradient leakage input indicators, the gradient information ratio can measure the risk of each sample in the batch sample being leaked by the batch sample gradient. In addition to assessing the sample leakage risk, it can also meet the application of the following scenarios: different samples have different privacy importance, so the participants will introduce a privacy protection mechanism (such as differential privacy) for the uploaded batch sample gradients. The traditional privacy budget is specified based on the entire input set and model, and the gradient information ratio indicator we proposed can assess the leakage risk of different samples, so that the participants can give smaller privacy budgets to samples with higher privacy importance, that is, different samples are given different privacy protection strengths, thereby meeting the purpose of differentiated privacy protection, thereby reducing the impact of privacy protection strategies on model training and usability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flowchart of the gradient leakage input risk assessment-gradient information ratio process for federated learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] Example

[0023] See also Figure 1The risk assessment method for model gradient leakage input information during the learning process provided by the embodiment of the present invention includes the following steps:

[0024] S1: Sample the local dataset to obtain a local sampled dataset.

[0025] S2: Analyze the local sampled data set to obtain the parameter gradient of the local sampled data sample.

[0026] S3: Calculate the mutual information value of the local sampled data sample and the sampled sample parameter gradient.

[0027] S4: When the mutual information value reaches the set level, a privacy leakage risk reminder is issued; otherwise, the parameter gradient is uploaded to the server.

[0028] Suppose a participant has a batch of samples (X, Y)≡((x (1) ,y (1) ),(x (2) ,y (2) ),...,(x (B) ,y (B) )), the batch size is B, where (x (i) ,y (i) ) represents the i-th sample in the batch, x (i) represents the sample characteristics, y (i) Represents the sample label. The batch sample (X, Y) is input into the neural network model to obtain its corresponding batch sample gradient G and upload it to the parameter server to achieve the purpose of jointly building the model without leaving the local data. The batch sample gradient G uploaded by the participant may be reconstructed into the local batch sample (X, Y) by the parameter server or other participants through reverse engineering, but the possibility of each sample in the batch sample being reconstructed is different. In order to quantify the risk of sample features being reconstructed, a risk assessment method for measuring the leakage of different sample features in the batch sample is proposed: gradient information ratio,

[0029] The batch sample gradient G has a mean relationship with the gradient of each sample in the batch sample:

[0030]

[0031] If the gradient G of a sample (i) The greater the amount of information in the batch sample gradient G, the greater the possibility of the sample being leaked. Therefore, we need to measure the possibility of the corresponding sample being leaked by calculating the amount of information in the batch sample gradient of the sample gradient. The gradient is a high-dimensional vector with two properties: size and direction. The vector of each sample needs to be projected onto the batch sample gradient vector to achieve comparison in the same direction and scale, and measure the size of the gradient information of each sample in the batch sample gradient. Using the vector projection formula, we can get the gradient G of the i-th sample(i) Projection vector on the batch sample gradient vector G As shown in formula (2).

[0032]

[0033] The projection vector The length of the batch sample gradient vector G is ||G (i) ||cosθ (i) , cosθ (i) is the cosine of the angle between the sample gradient and the batch sample gradient, and the direction is G / ||G||. At this time, each sample can obtain the corresponding projection vector through this formula. Based on this, it is proposed that the length of each projection vector ||G (i) ||cosθ (i) To compare the amount of information of each sample gradient on the batch sample gradient vector, as shown in formula (3).

[0034]

[0035] in <G (i) ,G> represents the quantity product, the Info(x (i) ) value, the greater the amount of information it occupies in the batch sample gradient G, and the more easily the corresponding sample features are leaked by the batch sample gradient. In order to more clearly represent the proportion of the amount of information of the sample gradient to the batch sample gradient, a gradient information ratio Info(x (i) ) to express the information ratio of each sample gradient to the batch sample gradient,

[0036]

[0037] Substituting formula (3) into (4), we can obtain:

[0038]

[0039] Where Info=(InfoR(x (0) ),InfoR(x (1) ),...,InfoR(X (b) )),|min(Info)| means taking the absolute value of the minimum projection vector length. ∈ is introduced to make the ratio always positive. InfoR(x (i) ) From the perspective of the information ratio of the target sample gradient to the batch sample gradient, it more intuitively expresses the risk of sample features being leaked by the batch sample gradient, corresponding to the risk degree of the batch sample gradient leaking a certain input information.

[0040] The gradient information ratio metric measures the risk of batch gradients leaking input information as follows:

[0041] 1) The participants select a batch of B samples (X, Y) to be trained and calculate the batch sample gradient G and the gradient G of each sample in it. (i) ;

[0042] 2) Select a sample x (i) , calculate the gradient information ratio of the selected sample according to the gradient information ratio indicator of formula (5);

[0043] 3) Through the gradient information ratio InfoR(x (i) ), get the sample x (i) The risk level of leakage of the batch sample gradient G is used to evaluate the risk of leakage of the input sample features.

[0044] The above-mentioned embodiment of the present invention provides a risk assessment method for model gradient leakage input information during the learning process. Compared with the existing gradient leakage input indicator, the gradient information ratio can measure the risk level of each sample in the batch sample being leaked by the batch sample gradient. The privacy importance of different samples is different, so the participants will introduce a privacy protection mechanism for the uploaded batch sample gradients.

[0045] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for assessing the risk of model gradient leakage input information during learning, characterized by: The following steps are involved: S1: Sample the local data set to obtain a local sampled data set; S2: Analyze the local sampled data set to obtain the parameter gradient of the local sampled data sample; S3: Calculate the mutual information value of the local sampled data sample and the sampled sample parameter gradient; S4: When the mutual information value reaches the set level, a privacy leakage risk reminder is issued; otherwise, the parameter gradient is uploaded to the server; The batch sample gradients uploaded by the participants in S3 may be reconstructed into local batch samples by the parameter server or other participants through reverse engineering. However, the possibility of reconstruction of each sample in the batch sample is different. In order to quantify the risk of sample feature reconstruction, a risk assessment method for measuring the leakage of different sample features in the batch sample is proposed: gradient information ratio; the batch sample gradient has a mean relationship with the gradient of each sample in the batch sample: Among them, the batch size is B; Calculate the amount of information of the sample gradient in the batch sample gradient to measure the possibility of the corresponding sample being leaked, and use the vector projection formula to get the i-th sample gradient G (i) Projection vector on the batch sample gradient vector G The formula is as follows: The projection vector The length of the batch sample gradient vector G is ||G (i) ||cosθ (i) , cosθ (i) is the cosine of the angle between the sample gradient and the batch sample gradient, and the direction is G / ||G||; By comparing the lengths of each projection vector ||G (i) ||cosθ (i) To compare the amount of information of each sample gradient on the batch sample gradient vector, the formula is as follows: in <G (i) ,G> represents the scalar product.

2. The method for risk assessment of model gradient leakage input information during learning according to claim 1, characterized in that: In S1, it is assumed that a participant has a batch of samples (X, Y)≡((x (1) ,y (1) ),(x (2) ,y (2) ),...,(x (B) ,y (B) )), where (x (i) ,y (i) ) represents the i-th sample in the batch, x (i) represents the sample characteristics, y (i) Represents the sample label.

3. The risk assessment method for model gradient leakage input information during learning according to claim 1, characterized in that: The batch samples in S2 are input into the neural network model to obtain their corresponding batch sample gradients and upload them to the parameter server to achieve the purpose of jointly building the model without leaving the local data.

Images