Privacy-based deep fake detection method under federated collaboration
Through the federated learning framework and attention mechanism combined with random masks and differential privacy noise, the performance degradation and data privacy protection of deep forgery detection on different data sets is solved, achieving more efficient detection performance and privacy protection.
Patent Information
- Application Number
- CN202111510599.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-10
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-10
AI Technical Summary
Existing deep forgery detection technologies have degraded performance when migrating to different data sets, and due to legal restrictions on data privacy protection and concerns about information leakage, it has made it difficult to use detectors in practical applications.
Deep falsification detection is adopted using a federated learning framework, data privacy is protected by distributed training of the model and using large-width convolution and attention mechanisms at the input layer, combining random masks and differential privacy noise.
The generalization performance of deep forgery detection is improved, data privacy protection is ensured, the performance degradation of the model on different data sets is reduced, and more effective detection results are achieved.
Smart Images

Figure FDA0003405188730000021 
Figure FDA0003405188730000022 
Figure FDA0003405188730000023
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer image processing technology, and in particular to a privacy-based deep fake detection method under federal collaboration. Background Art
[0002] The images or videos generated by deep fakes usually contain human faces and use faces as the main content, based on which some multimedia information is generated for the purpose of defamation, slander and rumor-mongering. Such content often has a great adverse impact on the characters and the entire society, especially when it comes to confidential materials and important public figures, which limits the dissemination and use of this part of data.
[0003] Legal restrictions and concerns about information leakage make the use of this information very uncertain. Even for regular images, most people are reluctant to share them. It is extremely difficult to obtain data.
[0004] In addition, in deepfake detection, since the fake content generated by different deepfake methods has great differences in features, the detection performance of the detector trained on a single dataset will be greatly reduced or even fail when it is transferred to other datasets for detection. The generalization performance of the model is poor. Summary of the invention
[0005] In order to solve the problems existing in the above-mentioned prior art, the present invention provides a privacy-based deep fake detection method under federal collaboration, which applies federated learning framework technology to deep fake detection, uses a distributed framework for model training, and protects data privacy at the same time.
[0006] The technical solution is as follows:
[0007] A privacy-based deep fake detection method under federated collaboration, the steps are as follows:
[0008] S1. Preprocess the data;
[0009] S2. Train the neural network through the deep fake detection algorithm under the federated learning framework;
[0010] S2.1. In the input layer, a large-width convolution is performed.
[0011] S2.2. The input is mapped to the latent space Π, in which the channel is divided into two branches F a and F b , each branch consists of several repeated Shuffle blocks. The Shuffle block is a residual structure module consisting of single-core group convolution, channel shuffle and depth-separable convolution;
[0012] S2.3. Use the attention mechanism to supervise the two branches and interact between them: At the beginning of the second shuffle block, an attention mechanism is introduced to connect it with the last shuffle block; For wide networks, the importance of different channels is calculated through limited additional computing resources; Use the spatial attention mechanism to connect the two branches as supervision;
[0013] S2.4, after channel fusion, more expressive spatial flow and channel flow information is obtained, and the branch block is summarized as the following formula:
[0014] [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA ))
[0015] [f ca (X branchB )+f sa (X branchA )]·g(h(x branchB ))
[0016] Among them, f ca represents the channel attention mechanism, f sa represents the spatial attention mechanism, X branchA represents the input of branch A, X branchB represents the input of branch B, g and h represent the convolution data transformation on branches A and B respectively;
[0017] For channel attention, the input is compressed in the spatial dimension and combined with average pooling and max pooling;
[0018] For spatial attention, the channels are compressed to 1 and the maximum pooling and global pooling attention maps are merged;
[0019] The model ends with eight shuffle blocks and finally a fully connected layer to get the final real / fake classification result;
[0020] S3. Apply random masking and differential privacy noise to the uploaded model locally on each client node.
[0021] Furthermore, the timing flow of the federated learning framework is as follows:
[0022] A1. Assume that there are k∈{1,...,K} nodes participating in each round of training, and each node k stores a data set D locally. k , containing n k samples, the data of each node do not intersect, that is, This is true for any i≠j; therefore, the complete data set D=∪ k∈[k] D k contain Sample points;
[0023] A2. Order W represents the model parameters, and solves a global optimization objective function G and a local optimization objective function {G k :k=1,...,K}:
[0024]
[0025] Among them, G represents the global objective function, G k represents the local objective function of the kth client, D represents the dataset, and D k represents the dataset of the kth client, p k represents the weight of the k-th client, Dk / D, and W represents the global model parameters;
[0026] A3. Calculate the optimal w.
[0027] Furthermore, the steps to calculate the optimal W are as follows:
[0028] A3.1. First, randomly initialize W0 or use the training parameters;
[0029] A3.2. In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain the new local model parameters W t k ;
[0030] A3.3. Using the stochastic gradient descent algorithm, the local update made by the k-th client in the t-th round can be expressed as follows:
[0031] W t k ←W t-1 -▽G k (D k :W t-1 )
[0032] Among them, W t k represents the model parameters of the kth client in the tth round, W t-1 represents the global model parameters of the t-1th round, ▽ represents the calculated gradient, G k represents the objective function of the kth client, D krepresents the dataset of the kth client;
[0033] After completing the current round of local iterative updates, the node uploads its own local model parameters after implementing random masking and differential privacy noise, and submits them to the central server for aggregation in a federated average manner;
[0034] W t ←h(W t 1 ,...,W t K ).
[0035] Furthermore, the specific steps of the preprocessing are as follows:
[0036] Assume a central server and m user nodes, each of which stores {n1,...,n m} data, the total is
[0037] S1.1. At each node, use local data to calculate local statistics. express;
[0038] S1.2. Each node uploads the statistical value to the central server, and the central server uses the local statistical value to calculate the global statistical value. The calculation formula is as follows:
[0039]
[0040]
[0041] in mean represents the mean of the population, var represents the variance of the population, and n k The number of data representing the kth client, represents the mean value of the k-th client data, Represents the variance of the k-th client data.
[0042] Further, the steps of implementing the random mask body are as follows:
[0043] At the end of each iteration, before uploading the model, the model parameters are set to empty with probability P, 0≤P≤30%;
[0044] When uploading the model to the central server, the model parameters are uploaded in a semi-complete form.
[0045] Furthermore, the specific steps for implementing differential privacy noise are as follows:
[0046] For the last local iteration of each global round, a differential privacy mechanism is applied:
[0047] Adding small noise to the updated parameters Make it become As the parameter to be uploaded.
[0048] The beneficial effects of the present invention are:
[0049] The privacy-based deep fake detection method under federal collaboration described in the present invention adopts a multi-party joint training method. Each node uses its own local data to train a local deep fake detection model. In each global aggregation, the local node only uploads model parameters but not the original data. The central server uses a certain aggregation algorithm to aggregate the global model parameters, and then sends them to each node for a new round of iterative updates.
[0050] Compared with the existing technology, the following innovations have been made:
[0051] (1) The present invention proposes a data preprocessing method in federated learning, which minimizes the performance degradation caused by data heterogeneity between nodes in federated learning;
[0052] (2) This paper proposes a new shallow network structure for deep fake detection, which greatly improves the detection performance;
[0053] (3) During local training, differential privacy noise is added to the model gradient, and random masks are added to the parameters to be uploaded to the central server. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is the pseudo code diagram of the algorithm of the present invention;
[0055] Figure 2 This is a timing flow chart of the federated learning of the present invention;
[0056] Figure 3 This is a flowchart of the deep fake detection network of the present invention;
[0057] Figure 4 It is a diagram of the derivation process of the data preprocessing calculation formula of the present invention;
[0058] Figure 5 The experimental results obtained after two different data preprocessing methods;
[0059] Figure 6 The ROC curve of the experimental results under different privacy budgets of differential privacy;
[0060] Figure 7 ROC curve of experimental results for different random mask sizes. DETAILED DESCRIPTION
[0061] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Figure 1-7 Provide further explanation.
[0062] Example 1
[0063] A privacy-based deep fake detection method under federated collaboration, the steps are as follows:
[0064] S1. Preprocess the data;
[0065] S2. Train the neural network through the deep fake detection algorithm under the federated learning framework;
[0066] S2.1. Perform a wide convolution at the input layer: Perform a wide convolution at the input layer to map the input RGB channels into an 80-dimensional space.
[0067] S2.2. The input is mapped to the latent space Π, in which the channel is divided into two branches F a and F b , each branch consists of several repeated Shuffle blocks. The Shuffle block is a residual structure module consisting of single-core group convolution, channel shuffle and depth-separable convolution;
[0068] S2.3. Use the attention mechanism to supervise the two branches and interact between them: At the beginning of the second shuffle block, an attention mechanism is introduced to connect it with the last shuffle block; For wide networks, the importance of different channels is calculated through limited additional computing resources; Use the spatial attention mechanism to connect the two branches as supervision;
[0069] S2.4, after channel fusion, more expressive spatial flow and channel flow information is obtained, and the branch block is summarized as the following formula:
[0070] [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA ))
[0071] [f ca (X branchB )+f sa (X branchA )]·g(h(x branchB ))
[0072] Among them, fca represents the channel attention mechanism, f sa represents the spatial attention mechanism, X branchA represents the input of branch A, X branchB represents the input of branch B, g and h represent the convolution data transformation on branches A and B respectively;
[0073] For channel attention, the input is compressed in the spatial dimension and combined with average pooling and max pooling;
[0074] For spatial attention, the channels are compressed to 1 and the maximum pooling and global pooling attention maps are merged;
[0075] The model ends with eight shuffle blocks and finally a fully connected layer to get the final real / fake classification result;
[0076] S3. Apply random masking and differential privacy noise to the uploaded model locally on each client node.
[0077] The timing flow of the federated learning framework is as follows:
[0078] A1. Assume that there are k∈{1,...,K} nodes participating in each round of training, and each node k stores a data set D locally. k , containing n k samples, the data of each node do not intersect, that is, This is true for any i≠j; therefore, the complete data set D=∪ k∈[k] D k contain Sample points;
[0079] A2. Order W represents the model parameters, and solves a global optimization objective function G and a local optimization objective function {G k :k=1,...,K}:
[0080]
[0081] Where G represents the global objective function, G k represents the local objective function of the kth client, D represents the dataset, and D k represents the dataset of the kth client, p k represents the weight of the k-th client, Dk / D, and W represents the global model parameters;
[0082] A3. Calculate the optimal w.
[0083] Preferably, the steps to calculate the optimal W are as follows:
[0084] A3.1. First, randomly initialize W0 or use the training parameters;
[0085] A3.2. In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain the new local model parameters W t k ;
[0086] A3.3. Using the stochastic gradient descent algorithm, the local update made by the k-th client in the t-th round can be expressed as follows:
[0087] W t k ←W t-1 -▽G k (D k :W t-1 )
[0088] Where W t k represents the model parameters of the kth client in the tth round, W t-1 represents the global model parameters of the t-1th round, ▽ represents the calculated gradient, G k represents the objective function of the kth client,
[0089] D k represents the dataset of the kth client;
[0090] After completing the current round of local iterative updates, the node uploads its own local model parameters after implementing random masking and differential privacy noise, and submits them to the central server for aggregation in a federated average manner;
[0091] W t ←h(W t 1 ,...,W t K ).
[0092] The specific steps of the pre-processing are as follows:
[0093] Assume a central server and m user nodes, each of which stores {n1,...,n m} data, the total is
[0094] S1.1. At each node, use local data to calculate local statistics. express;
[0095] S1.2. Each node uploads the statistical value to the central server, and the central server uses the local statistical value to calculate the global statistical value. The calculation formula is as follows:
[0096]
[0097]
[0098] in mean represents the mean of the population, var represents the variance of the population, and n k The number of data representing the kth client, represents the mean value of the k-th client data, represents the variance of the kth client data;
[0099] The steps of implementing the random mask body are as follows:
[0100] At the end of each iteration, before uploading the model, the model parameters are set to empty with probability P, 0≤P≤30%;
[0101] When uploading the model to the central server, the model parameters are uploaded in a semi-complete form. Semi-complete means that not all model parameters are uploaded, but as mentioned above, each data has a 0-30% probability of being empty, so the overall value will also be 0-30% empty to ensure the security of local data.
[0102] The specific steps for implementing differential privacy noise are as follows:
[0103] For the last local iteration of each global round, a differential privacy mechanism is applied:
[0104] Adding small noise to the updated parameters Make it become As the parameter to be uploaded.
[0105] Example 2
[0106] As a new embodiment or a supplement to Embodiment 1.
[0107] (1) Federated Learning Framework
[0108] This algorithm is a deep fake detection algorithm under federated learning. Now we will first explain the timing process of the federated learning framework.
[0109] Assume that there are k∈{1,...,K} nodes participating in each round of training, and each node k stores a dataset D locally. k , containing n k samples, the data of each node are disjoint (horizontal federation), that is This is true for any i≠j; therefore, the complete data set D=∪ k∈[k] D k contain Sample points. W represents the model parameters. What needs to be solved in the end is a global optimization objective function G and a local optimization objective function {G k :k=1,...,K}problem:
[0110]
[0111] Next, the optimal w is obtained by calculation.
[0112] First, randomly initialize (or use the training parameters) W0. In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain the new local model parameters W t k .
[0113] This process uses the stochastic gradient descent algorithm (SGD). The local update made by the k-th client in the t-th round can be expressed as follows:
[0114] W t k ←W t-1 -▽G k (D k :W t-1 )
[0115] After completing the current round of local iterative updates, the node uses a certain mechanism to upload its own local model parameters, which are then aggregated in a certain way by the central server.
[0116] W t ←h(W t 1 ,...,W t K )
[0117] like Figure 2The flowchart of this process is shown in Figure 1. ① indicates that the central server sends the model parameters of the previous round (or initial) to each client as the model parameters of the local model; ② indicates that each client node uses local data, i.e., model parameters, to perform a certain number of local iterative updates to obtain new local model parameters; ③ indicates that each node uploads the model parameters to the central server in a certain way after the local update is completed; ④ indicates that the central server aggregates the local model parameters uploaded by each node to obtain a new round of global model parameters. The cycle repeats until the model converges and training stops.
[0118] (2) Data preprocessing
[0119] Before training a neural network, the data is usually preprocessed, for example, regularized. In past federated learning methods, due to the fact that data cannot be exchanged with each other, during data preprocessing, only the local statistical characteristics of each node (such as the mean and variance of local data) are often used to independently regularize the data. However, due to the problem of data heterogeneity in federated learning, such as color differences, brightness differences, different forgery methods, etc., this may cause serious deviations in the data of each node after regularization, and may cause the local stochastic gradient direction to deviate from the global gradient direction. Therefore, under the constraint of privacy protection in federated learning, the present invention proposes a global data preprocessing method, which is called FZSN (federated Z-score normalization).
[0120] Specifically, suppose there is a central server and m user nodes, each of which stores {n1,...,n m} data, the total is First, each node uses its local data to calculate local statistics (mean, variance), and then uses Then each node uploads the statistical value to the central server, and the central server uses these local statistical values (excluding any original data) to calculate the global statistical value. The calculation formula is as follows:
[0121]
[0122]
[0123] in This formula is obtained by our derivation. The derivation process is detailed in the attachment. Figure 4 shown.
[0124] (3) Deep Fake Detection Network
[0125] The overall network flow diagram is as follows: Figure 3As shown in the figure, according to the idea of channel shuffling and grouped convolution in ShuffleNet, it is used as the basic repetitive unit in the present invention and the network.
[0126] Our model has a very shallow network structure for the following reasons: from a feature perspective, deep fake artifacts are highly expressed at mid-levels, while at high semantic levels, methods tend to generalize poorly due to feature variability; from a federated learning perspective, we often need to tolerate devices with low computational performance, so shallow networks are more conducive to implementation on lightweight devices; finally, as a distributed model, shallow networks with fewer model parameters can reduce communication overhead.
[0127] At the input layer, a large width convolution is performed to fully express all features and reduce the information loss in the pooling stage. The input is mapped to the latent space Π. In the latent space Π, the channel is divided into two branches F a and F b , each branch consists of many repeated Shuffle blocks. The Shuffle block is a residual structure module consisting of single-core grouped convolution, channel shuffle and depth-wise separable convolution (DWConv).
[0128] The attention mechanism is used to supervise the two branches and interact between the branches. At the beginning of the second shuffle block, the attention mechanism is introduced to connect it with the last shuffle block. For wide networks, only limited additional computing resources are needed to calculate the importance of different channels. Another point-by-point cross-branch spatial attention mechanism, because the information in the two branches is incomplete, the information of one branch is introduced into the other branch as supervision. After channel fusion, more expressive spatial flow and channel flow information can be obtained. The branch block is summarized as the following formula:
[0129] [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA ))
[0130] [f ca (X branchB )+f sa (X branchA )]·g(h(x branchB ))
[0131] For channel attention, the input is compressed in the spatial dimension and combined with average pooling and max pooling. For spatial attention, the channels are also compressed to 1 and the max pooling and global pooling attention maps are merged. The model ends with eight shuffle blocks that are connected to the previous selection. Finally, there is a fully connected layer to get the final classification result of real / fake.
[0132] (4) Random Masking and Differential Privacy
[0133] In order to solve the problem of privacy protection and privacy measurement, random masks and differential privacy noise are added to the uploaded model locally on each client node. The pseudo code of the process is as follows: Figure 1 shown.
[0134] Differential privacy is a widely used and standard method for privacy protection. When a user completes a round of model training using all local data, it is called a local round of training. When a user completes e local rounds of training, the node is called a global round of training. For the last local iteration of each global round, a differential privacy mechanism is applied, called DP-SGD. That is, a small noise is added to the updated parameters (gradients). Make it become As the parameter to be uploaded. For example: If M is an additive mechanism M(W) = W + N, and N is Gaussian or Laplace noise. Set the L2 norm of the maximum gradient equal to 1, so the magnitude of the gradient remains unchanged, and set δ equal to 1e-5, which is the inverse of the order of magnitude of the dataset size. The noise amplitude σ is set to 2.
[0135] From the perspective of data security and communication effectiveness, the present invention also implements random masks in the model. In the last round of each iteration, before uploading the model, a part of the model parameters are set to empty with a certain probability. When the model is uploaded to the central server, the model parameters are uploaded in a semi-complete form. This method can reduce the number of model parameters and increase the possibility of malicious clients deriving data from model parameters. And under the random additive noise mechanism of differential privacy, the connection between model parameters and training data becomes model, and privacy is therefore more guaranteed. The quantitative analysis experiment part on privacy protection is shown in Example 3.
[0136] Example 3
[0137] This example is a supplement to Example 1 and Example 2 and is used to illustrate the experimental data.
[0138] (1) Dataset used in the experiment
[0139]
[0140] (2) Visualize the experimental results of different categories of images and observe which part of the face the trained model focuses on.
[0141] (3) Figure 5 As shown in the figure, under the condition of non-independent and identically distributed, the experimental results are obtained after using two different data preprocessing methods. From the observation of the figure, it can be seen that the use of the experimental preprocessing method we proposed can make the model converge faster and the loss function converge more completely.
[0142] (4) Performance on the FaceForensics++ dataset and comparison with other methods:
[0143]
[0144]
[0145] Both training and testing are performed on the same dataset
[0146] (5) Experimental results on some other typical datasets, and the same dataset is used for model training and testing.
[0147] Dataset Fed-MesoNet Our Approach Celeb-DFv2 88.72 92.51 UADFA 95.64 98.82 DeepFakeDetection 94.93 97.73 DFDC 86.40 90.96
[0148] (6) Cross-database testing
[0149] The training set adopts the following three distribution methods, including:
[0150] 1: After the FF++ dataset is fully mixed, it is distributed to 5 clients to jointly train the joint model.
[0151] Second: Each client holds fake images and real image data generated by a forging method, and aggregates them to obtain a global model.
[0152] 3: Each client holds the fake images and real image data generated by the two forging methods, and aggregates them to obtain the global model.
[0153] Then, four different categories of images in the test set were tested separately.
[0154]
[0155]
[0156] Each client holds a dataset generated using the DeepFake method, which is aggregated to form a joint model.
[0157] Dataset FF++(DeepFake) Celeb-DFv2 UADFA DeepFakeDetection Accuracy 95.22 93.37 97.14 95.65
[0158] (7) Under different privacy budgets (epsilon) of differential privacy, the ROC curve of the experimental results is as follows: Figure 6 shown.
[0159] The ROC curve of the experimental results with different random mask sizes is as follows: Figure 7 shown.
[0160] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A privacy-based deep fake detection method under federal collaboration, characterized in that: Here are the steps: S1. Preprocess the data; S2. Train the neural network through the deep fake detection algorithm under the federated learning framework; S2.
1. In the input layer, a large-width convolution is performed. S2.
2. The input is mapped to the latent space Π, in which the channel is divided into two branches F a and F b , each branch consists of several repeated Shuffle blocks. The Shuffle block is a residual structure module consisting of single-core group convolution, channel shuffle and depth-separable convolution; S2.
3. Use the attention mechanism to supervise the two branches and interact between them: At the beginning of the second shuffle block, an attention mechanism is introduced to connect it with the last shuffle block; For wide networks, the importance of different channels is calculated through limited additional computing resources; Use the spatial attention mechanism to connect the two branches as supervision; S2.4, after channel fusion, more expressive spatial flow and channel flow information is obtained, and the branch block is summarized as the following formula: [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA )) [f ca (X branchB )+f sa (X branchA )]·g(h(x branchB )) Among them, f ca represents the channel attention mechanism, f sa represents the spatial attention mechanism, X branchA represents the input of branch A, X branchB represents the input of branch B, g and h represent the convolution data transformation on branches A and B respectively; For channel attention, the input is compressed in the spatial dimension and combined with average pooling and max pooling; For spatial attention, the channels are compressed to 1 and the maximum pooling and global pooling attention maps are merged; The model ends with eight shuffle blocks and finally a fully connected layer to get the final real / fake classification result; S3. Apply random masking and differential privacy noise to the uploaded model locally on each client node.
2. The privacy-based deep fake detection method under federal collaboration as described in claim 1 is characterized in that: The timing flow of the federated learning framework is as follows: A1. Assume that there are k∈{1,...,K} nodes participating in each round of training, and each node k stores a data set D locally. k , containing n k samples, the data of each node do not intersect, that is, This is true for any i≠j; therefore, the complete data set D=∪ k∈[k] D k contain Sample points; A2. Order W represents the model parameters, and solves a global optimization objective function G and a local optimization objective function {G k :k=1,...,K}: Among them, G represents the global objective function, G k represents the local objective function of the kth client, D represents the dataset, and D k represents the dataset of the kth client, p k represents the weight of the k-th client, Dk / D, and W represents the global model parameters; A3. Calculate the optimal w.
3. The privacy-based deep fake detection method under federal collaboration as described in claim 2 is characterized in that: The steps to calculate the optimal W are as follows: A3.
1. First, randomly initialize W0 or use the training parameters; A3.
2. In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain new local model parameters A3.
3. Using the stochastic gradient descent algorithm, the local update made by the k-th client in the t-th round can be expressed as follows: in, represents the model parameters of the kth client in the tth round, W t-1 represents the global model parameters of the t-1th round, Indicates the calculation gradient, G k represents the objective function of the kth client, D k represents the dataset of the kth client; After completing the current round of local iterative updates, the node uploads its own local model parameters after implementing random masking and differential privacy noise, and submits them to the central server for aggregation in a federated average manner; IN t ←h(W t 1 ,...,IN t K )。 4. The privacy-based deep fake detection method under federal collaboration as claimed in claim 1, characterized in that: The specific steps of the pretreatment are as follows: Assume a central server and m user nodes, each of which stores {n1,...,n m } data, the total is S1.
1. At each node, use local data to calculate local statistics. express; S1.
2. Each node uploads the statistical value to the central server, and the central server uses the local statistical value to calculate the global statistical value. The calculation formula is as follows: in mean represents the mean of the population, var represents the variance of the population, and n k The number of data representing the kth client, represents the mean value of the k-th client data, Represents the variance of the k-th client data.
5. The privacy-based deep fake detection method under federal collaboration as described in claim 1 is characterized in that: The steps to implement a random mask body are as follows: At the end of each iteration, before uploading the model, the model parameters are set to empty with probability P, 0≤P≤30%; When uploading the model to the central server, the model parameters are uploaded in a semi-complete form.
6. The privacy-based deep fake detection method under federal collaboration as claimed in claim 1, characterized in that: The specific steps to implement differential privacy noise are as follows: For the last local iteration of each global round, a differential privacy mechanism is applied: Adding small noise to the updated parameters Make it become As the parameter to be uploaded.
