A method for evaluating a risk of leaking input information in a federated learning process

Abstract

The application relates to the field of artificial intelligence and discloses a risk evaluation index method for leaking input information in a federal learning process, which comprises the following steps: S1: a participant selects a batch of to-be-trained samples (X, Y) with a number of B, and calculates batch sample gradient G and gradient G of each sample in the batch sample gradient G (i) ; S2: a certain sample x (i) is selected, and gradient information ratio InfoR(x (i) ) of the selected sample is calculated; S3: the gradient information ratio InfoR(x (i) ) is used to obtain the risk degree of sample x (i) being leaked by the batch sample gradient G, so that the evaluation of the input sample feature leakage risk is achieved. The application can meet the requirement that the participant gives a smaller privacy budget to a sample with higher privacy importance, that is, different privacy protection strengths are given to different samples, so that the purpose of differentiated privacy protection is met, and the influence of the privacy protection strategy on model training and availability is reduced.

Description

Technical Field

[0001] This invention relates to the field of data privacy protection in federated learning within the field of artificial intelligence technology, specifically to a risk assessment index method for leaking input information during federated learning. Background Technology

[0002] With the significant improvement in information processing capabilities and computing efficiency, a large number of deep learning-based applications have emerged that process data such as images, text, and speech. Deep learning models and applications rely on large amounts of high-quality data; however, in reality, few single entities possess such vast amounts of data. Insufficient data volume or low data quality makes it difficult to support the construction and application of deep learning models. Collecting the relevant data from various entities raises issues of data copyright and privacy protection. Therefore, federated learning models, designed to aggregate data to achieve economies of scale while protecting data copyright and privacy, have been developed and are gradually being applied. Federated learning is a distributed deep learning framework where multiple participants or computing nodes share aggregated machine learning model gradient parameters without directly exchanging data, thus resolving the contradiction between data sharing and privacy protection.

[0003] Recent research on the effectiveness of privacy protection in federated learning has confirmed that input data can be reconstructed based on model gradients through certain methods. This means that privacy protection mechanisms that share model gradients in federated learning still carry the risk of data leakage. Furthermore, different input samples have different feature values, and the risk of leakage varies depending on the input sample. We have invented a new metric to measure the degree of risk of input sample leakage in a deep learning-based federated learning framework. This metric can be used to evaluate and filter input sample data and assess the privacy security of the framework during its construction and application.

[0004] The existing solution includes the following steps: ① Sampling the local dataset used as input to the local model by a participant in federated learning to obtain a local sampled dataset; ② Training the local model using the local sampled dataset to obtain the parameter gradient of the local sampled data samples; ③ Calculating the mutual information value between the local sampled data samples and the parameter gradient of the sampled samples using a pre-trained statistical model; ④ When the mutual information value is greater than or equal to a preset threshold, issuing a privacy risk warning for the gradient of the parameter; otherwise, uploading the gradient of the parameter to the parameter aggregation server.

[0005] In the aforementioned prior art, the mutual information value between a sampled sample and its gradient is calculated using a pre-trained statistical model, thereby measuring the gradient privacy leakage risk of the sample. This technique requires a pre-trained statistical model as a prerequisite, and the model performance affects the accuracy of the mutual information calculation. Furthermore, this technique only issues risk warnings for samples with high mutual information values ​​calculated using its proposed method, ignoring the gradient privacy leakage risk of other samples, which still still pose a gradient privacy leakage risk.

[0006] The aforementioned problems stem from two aspects: first, the reliance on pre-trained statistical models for mutual information calculation reduces the efficiency of technology construction and application; second, the sample-gradient mutual information value calculated based on the overall characteristics of the sampled data reflects the risk of overall sample leakage, and is not directly related to the gradient privacy leakage risk of a single sample. In actual gradient leakage data attacks, the target sample is a batch of samples, which usually contains multiple samples. Data thieves can only obtain the gradient of the target participant's batch of samples, and cannot know the gradient corresponding to each sample in the batch. Therefore, they can only reverse engineer the gradient of the batch of samples to reconstruct the batch of samples. The degree of feature restoration of the reconstructed batch of samples varies, that is, the degree to which the features of the target sample are leaked varies. This is mainly determined by the ratio of the gradient of a sample to the information of the gradient of the batch of samples. The larger the gradient information ratio, the more dominant the gradient of that sample is in the gradient of the batch of samples. When using the gradient of the batch of samples for reverse analysis, it is easier to reconstruct the features of that sample. Existing technologies do not effectively reflect this correlation.

[0007] To address this, we propose a risk assessment index method for the leakage of input information during the federated learning process. Summary of the Invention

[0008] (a) Technical problems to be solved

[0009] To address the shortcomings of existing technologies, this invention provides a risk assessment index method for leaking input information during federated learning, thus solving the aforementioned problems.

[0010] (II) Technical Solution

[0011] To achieve the above-mentioned objectives, the present invention provides the following technical solution: a risk assessment index method for leaking input information during federated learning, comprising the following steps:

[0012] S1: The participants select a batch of B training samples (X,Y) and calculate the gradient G of the batch samples and the gradient G of each sample. (i) ;

[0013] S2: Select a sample x (i) Calculate the gradient information ratio InfoR(x) of the selected samples. (i) );

[0014] S3: By using gradient information ratio InfoR(x) (i) ), obtain sample x (i) The risk level of leakage by the gradient G of the batched samples is used to assess the risk of leakage of input sample features.

[0015] Preferably, the batch sample gradient G has a mean relationship with the gradients of each sample in the batch:

[0016]

[0017] Preferably, the gradient G of the i-th sample is obtained using the vector projection formula. (i) The projection vector onto the batch sample gradient vector G

[0018] The formula is as follows:

[0019]

[0020] Where the projection vector The length of the gradient vector G on the batch samples is ||G (i) ||cosθ (i) cosθ (i) Let G be the cosine of the angle between the sample gradient and the batch sample gradient, with the direction being G / ||G||.

[0021] Preferably, the step of comparing the lengths ||G| of each projection vector is... (i) ||cosθ (i) To compare the information content of each sample's gradient on the batch sample gradient vector, the formula is as follows:

[0022]

[0023] in <G (i) G> represents the dot product.

[0024] Preferably, the gradient information ratio in S2 is calculated using the following formula:

[0025]

[0026] Info = (InfoR(x) (0) ),InfoR(x (1) ),…,InfoR(x (B) )),|min(Info)| represents taking the absolute value of the minimum projection vector length. The introduction of ∈ is to ensure that the ratio is always positive.

[0027] (III) Beneficial Effects

[0028] Compared with existing technologies, this invention provides a risk assessment index method for leaking input information during federated learning, which has the following beneficial effects:

[0029] 1. This federated learning method for assessing the risk of input information leakage, compared to existing gradient leakage metrics, uses the gradient information ratio to measure the risk of each sample in a batch being leaked by the gradients of the batch samples. Besides assessing sample leakage risk, it can also meet the application needs of scenarios where different samples have different privacy importance. In such cases, participants will introduce privacy protection mechanisms (such as differential privacy) for the uploaded batch sample gradients. Traditional privacy budgets are specified based on the entire input set and the model. The gradient information ratio metric proposed in this invention can assess the leakage risk of different samples, thus allowing participants to allocate smaller privacy budgets to samples with higher privacy importance, i.e., different samples receive different levels of privacy protection, thereby achieving differentiated privacy protection and reducing the impact of privacy protection strategies on model training and usability. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the gradient information ratio for assessing the input risk of gradient leakage in federated learning, as described in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example

[0033] Please see Figure 1 The risk assessment index method for leaking input information during federated learning provided in this embodiment, in the model construction of federated learning, assumes that a certain participant has a batch of samples (X,Y) = ((x (1) ,y (1) ),(x (2) ,y (2) ),…,(x (B) ,y (B) The batch size is B, where (x (i) ,y (i) ) represents the i-th sample in the batch, x (i) Represents sample features, y (i)The sample label is represented by a gradient G. A batch of samples (X,Y) is input into the neural network model to obtain its corresponding batch gradient G, which is then uploaded to the parameter server to achieve the goal of jointly constructing the model without data leaving the local machine. The batch gradient G uploaded by the participants may be reconstructed into a local batch sample (X,Y) by the parameter server or other participants through reverse engineering, but the probability of each sample in the batch being reconstructed is different. To quantify the risk of sample feature reconstruction, a risk assessment metric for measuring the leakage of different sample features in the batch is proposed: gradient information ratio.

[0034] The gradient G of the batch samples has a mean relationship with the gradients of each sample in the batch:

[0035]

[0036] If the gradient G of a certain sample (i) The greater the information content of the gradient G of a sample relative to the gradient of the batch samples, the greater the likelihood that the sample will be leaked. Therefore, we need to measure the likelihood of a sample being leaked by calculating the information content of the sample gradient relative to the gradient of the batch samples.

[0037] The gradient is a high-dimensional vector with both magnitude and direction attributes. To compare the gradients of each sample on the same scale and direction, we need to project the gradients of each sample onto the gradient vector of the batch samples. This allows us to measure the relative importance of each sample's gradient information compared to the batch's gradient information. The gradient G of the i-th sample is obtained using the vector projection formula. (i) The projection vector onto the batch sample gradient vector G As shown in formula (2).

[0038]

[0039] Where the projection vector The length of the gradient vector G on the batch samples is ||G (i) ||cosθ (i) cosθ (i) Let G be the cosine of the angle between the sample gradient and the batch sample gradient, with the direction being G / ||G||. Then, the corresponding projection vector for each sample can be obtained using this formula. Based on this, it is proposed that the lengths ||G|| of each projection vector can be compared. (i) ||cosθ (i) To compare the information content of each sample gradient on the gradient vector of the batch samples, as shown in formula (3).

[0040]

[0041] in <G (i) G> represents the dot product, where Info(x) is the product of the data of a given sample. (i)A higher gradient information ratio (InfoR(x)) indicates a greater amount of information contained in the gradient of a sample within the batch gradient G, and the more easily its corresponding sample features are leaked by the batch gradient. To more clearly represent the proportion of information contained in the gradient of a sample within the batch gradient, a gradient information ratio (InfoR(x)) is proposed. (i) This can be used to express the proportion of information about the gradient of each sample relative to the gradient of the entire batch.

[0042]

[0043] Substituting formula (3) into (4) yields the following:

[0044]

[0045] Where Info = (InfoR(x) (0) ),InfoR(x (1) ),…,InfoR(x (B) ),|min(Info)| represents taking the absolute value of the minimum projection vector length. The ∈ is introduced to ensure that the ratio is always positive. InfoR(x (i) From the perspective of the ratio of the target sample gradient to the batch sample gradient, the risk of sample features being leaked by the batch sample gradient is expressed more intuitively, corresponding to the degree of risk of the batch sample gradient leaking a certain input information.

[0046] The steps for using the gradient information ratio metric to measure the risk of gradient leakage of input information in a batch of samples are as follows:

[0047] S1: The participants select a batch of B training samples (X,Y) and calculate the gradient G of the batch samples and the gradient G of each sample. (i) ;

[0048] S2: Select a sample x (i) The gradient information ratio of the selected sample is calculated according to the gradient information ratio index of formula (5);

[0049] S3: By using gradient information ratio InfoR(x) (i) ), obtain sample x (i) The risk level of leakage by the gradient G of the batched samples is used to assess the risk of leakage of input sample features.

[0050] The method provided by the above embodiments of the present invention can satisfy the requirement that participants allocate smaller privacy budgets for samples with higher privacy importance, that is, different privacy protection intensities are given to different samples, thereby satisfying the purpose of differentiated privacy protection and reducing the impact of privacy protection strategies on model training and usability.

[0051] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for risk assessment indicators of input information leakage in federated learning processes, characterized in that, The method comprises the following steps: S1: the participant selects a batch of training samples with a number of , and calculates the batch sample gradient vector and the gradient of each sample; S2: select a sample , calculate the gradient information ratio of the selected sample ; S3: Through gradient information ratio , get sample batch sample gradient vector leakage risk degree, to achieve input sample feature leakage risk assessment; The vector projection formula is used to obtain the first sample gradient The projection vector on the batch sample gradient vector is obtained ; The formula is as follows: ； where the projection vector The length of the batch sample gradient vector is , The cosine value of the sample gradient and the batch sample gradient vector is ; By comparing the length of each projection vector to compare the amount of information of each sample gradient on the batch sample gradient vector, the formula is as follows: ； wherein denotes the scalar product; The formula for calculating the gradient information ratio in S2 is as follows: The formula for calculating the gradient information ratio in S2 is as follows: ； , denotes taking the absolute value of the minimum projection vector length, and the introduction of e is to make the ratio constant positive parameter.

2. The method of claim 1, wherein the method further comprises: determining a risk of leaking input information in a federated learning process. The batch sample gradient vector There is a mean relationship with the gradient of each sample in the batch sample: 。

Images