Application method of detection method for privacy type deep forgery in federal collaboration

Abstract

The invention provides an application method of a privacy type deep counterfeiting detection method in federated collaboration, and belongs to the technical field of computer image processing. According to the specific embodiment, the method comprises the steps of preprocessing data; and carrying out model training on a plurality of client nodes by using a deep forgery detection algorithm based on federated learning, and locally carrying out random masking and differential privacy noise processing on a to-be-uploaded model. The method has the advantages that a multi-party joint co-training mode is adopted, each node trains a local deep counterfeiting detection model through local data, in the global aggregation process, the local nodes only upload model parameters instead of original data, a central server integrates global model parameters through a specific aggregation algorithm, and the global model parameters are integrated through a multi-party joint co-training mode. And then the data is issued to each node for a new round of iterative updating.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing technology, and in particular to an application method of a method for privacy-oriented deep fakes in federal collaboration. Background Art

[0002] The rapid development of deep fake technology has brought serious challenges to personal privacy and social security. This technology can use advanced generative models to modify faces or other images to produce multimedia content that looks real but is actually fake. Such content may be used for bad purposes such as fake news, political attacks, and commercial fraud, posing a potential threat to the reputation and personal safety of victims.

[0003] Especially for deep fakes in key areas or important social figures, the risks cannot be ignored. The government and relevant agencies are in urgent need of a technology that can quickly and accurately identify deep fake content to protect public interests and social security. However, due to the restrictions of laws and regulations and privacy protection, as well as concerns about information leakage, the application of this identification technology has been subject to many challenges and restrictions.

[0004] In the field of deepfake detection, an important issue is the generalization performance of the model. Since the fake content generated by different deepfake methods has huge differences in characteristics, detectors trained on a single dataset usually perform poorly when verified on other datasets. This leads to insufficient generalization performance of the model and is unable to effectively deal with different types of deepfake content.

[0005] Therefore, it is necessary to build a more robust and generalized deep fake detection model with the help of cutting-edge technologies such as federated learning and transfer learning. At the same time, we should strengthen the protection measures for data privacy and information leakage to provide a more reliable foundation for the development of deep fake detection technology. Only through the cooperation of law, technology and ethics can we better resist the challenges brought by deep fake technology and safeguard the public interest and stability of society. Summary of the invention

[0006] To solve the current problems, the present invention proposes an innovative method: an application method of a privacy-oriented deep fake method in federated collaboration. This method applies advanced federated learning framework technology to the field of deep fake detection, and realizes model training through a distributed framework while protecting data privacy.

[0007] By adopting the federated learning framework, we can train the model without sharing sensitive data. Each participant can process their own data locally to ensure data privacy. At the same time, by training the model in a distributed framework, we can effectively improve the model's parameter convergence speed and performance, thereby better coping with the challenges of deep fake detection. The steps are as follows:

[0008] S1. The first step is to preprocess the data;

[0009] S2. Next, the neural network is trained. Here, the deep fake detection algorithm under the federated learning framework is used;

[0010] S2.1. Perform a wide convolution at the input layer of the model.

[0011] S2.2, after the data is mapped to the latent space Π, the latent space Π divides the channel into two branches, namely F a and F b Each branch contains multiple repeated Shuffle blocks, which are residual structure modules composed of single-core grouped convolution, channel shuffle and depth-wise separable convolution.

[0012] S2.3. Use the attention mechanism to supervise the two branches and encourage the two branches to interact with each other. The attention mechanism is introduced at the beginning of the second Shuffle block and connected to the last Shuffle block. In a network with a larger width, the importance of different channels is calculated using limited additional computing resources. At the same time, the spatial attention mechanism is used to connect the two branches for supervision.

[0013] S2.4, after channel fusion, more expressive spatial flow and channel flow information are obtained. Here, let f ca represents the channel attention mechanism, f sa represents the spatial attention mechanism, X branchA represents the input of branch A, X branchB represents the input of branch B, g and h represent the convolution data transformation on branches A and B respectively;

[0014] The branch block is summarized as the following formula:

[0015] [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA ))

[0016] [f ca (X branchB )+f sa (XbranchA )]·g(h(x branchB ))

[0017] For channel attention, the input is compressed in the spatial dimension and average pooling and maximum pooling are applied simultaneously; for spatial attention, the channel is compressed to 1 and two attention maps are generated by combining maximum pooling and global pooling; the end of the model contains eight Shuffle blocks, and finally a fully connected layer is connected to output the final real / fake classification result.

[0018] S3. Apply random masking and differential privacy noise to the uploaded model locally on each client node.

[0019] Furthermore, the timing flow of the federated learning framework is as follows:

[0020] A1. Assume there are k∈{1,...,K} nodes, and each node k stores a data set D k , containing n k samples, the data of each node do not intersect, the complete data set D = ∪ k∈[k] D k contain Sample points;

[0021] A2. Let W represent the model parameters, Let G denote the global objective function, G k represents the local objective function, D represents the data set, and D k represents the kth data set, p k represents the kth weight, W represents the global model parameter; there are global optimization objective function G and local optimization objective function {G k :k=1,...,K}:

[0022]

[0023] A3. Calculate the optimal w.

[0024] Furthermore, the steps to calculate the optimal W are as follows:

[0025] A3.1. Randomly initialize or use pre-trained parameters;

[0026] A3.2. In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain new local model parameters

[0027]

[0028] A3.3, using the stochastic gradient descent algorithm, assume represents the model parameters of the kth client in the tth round, W t-1 represents the global model parameters of the t-1th round, Indicates the calculation gradient, G k represents the objective function of the kth client, D k represents the data set of the kth client; the local update made by the kth client in the tth round can be expressed as follows:

[0029]

[0030] After completing the current round of local iterative updates, the node will randomly mask the local model parameters and add differential privacy noise, and then upload the processed local model parameters to the central server. The central server will aggregate the model parameters uploaded by all nodes in a federated average manner.

[0031]

[0032] Furthermore, the specific steps of the preprocessing are as follows:

[0033] S1.1. At each node, use local data to calculate local statistics. express;

[0034] S1.2. Each node uploads its local statistical value to the central server, and the central server uses these local statistical values ​​to calculate the global statistical value. mean represents the mean of the population, var represents the variance of the population, and n k Express the number of k-th data, represents the mean of the kth data, Represents the variance of the kth data.

[0035] The calculation formula is as follows:

[0036]

[0037] Further, the steps of implementing the random mask body are as follows:

[0038] At the end of each iteration, before uploading the model, the node sets the model parameters to null with probability P, where 0≤P≤30%.

[0039] When uploading the model to the central server, the model parameters are uploaded in a semi-complete form, which means that only some parameters are visible or complete, and other parameters may be encrypted or hidden.

[0040] Furthermore, the specific steps for implementing differential privacy noise are as follows:

[0041] For the last local iteration of each global round, a differential privacy mechanism is applied to add a small amount of noise to the updated parameters. Make it become As the parameter to be uploaded.

[0042] The beneficial effects of the present invention are:

[0043] The privacy-based deep fake detection method under federated collaboration of this invention adopts a multi-party joint training method. Each participating node uses local data to train a local deep fake detection model. Then, in the global aggregation process, the local node only uploads model parameters instead of original data. The central server uses a specific aggregation algorithm to aggregate these model parameters into global model parameters, and then sends them back to each node for a new round of iterative updates. Compared with the existing technology, the following innovations have been made:

[0044] (1) This paper proposes a novel shallow network structure that can significantly improve the performance of deep fake detection;

[0045] (2) The present invention proposes a data preprocessing method for federated learning, which effectively reduces the performance degradation caused by data heterogeneity between nodes;

[0046] (3) During local training, differential privacy noise is added to the model gradient, and the parameters to be uploaded to the central server are randomly masked;

[0047] (4) Core mechanisms for privacy protection

[0048] Data localization training: Each participating node (such as different social media platforms, security agencies or research institutions) only uses its locally stored data set for model training. This distributed data processing method fundamentally avoids the privacy leakage risks that may be caused by data centralization;

[0049] Sharing model parameters instead of data: During the training process, each node periodically encrypts the parameters of the local model (such as weights and biases) and uploads them to a trusted central server instead of directly transmitting the original data. This step ensures the privacy of the data because even the central server cannot reversely deduce the specific original data content.

[0050] (5) Global model aggregation and optimization

[0051] Secure aggregation algorithm: The central server uses advanced encryption technology and secure multi-party computing protocols to aggregate the received model parameters. These algorithms ensure data security and privacy protection during the aggregation process, while effectively resisting potential man-in-the-middle attacks and data tampering.

[0052] Global model update and distribution: After secure aggregation, the central server generates global model parameters and securely distributes them back to each participating node. Each node uses these global parameters to update its local model, thereby integrating knowledge from different data sources and improving the model's generalization ability and detection accuracy.

[0053] (6) Iterative optimization and performance improvement

[0054] Multiple rounds of iterative training: The above process (local training, parameter uploading, global aggregation, model updating) will be repeated for multiple rounds to form a closed-loop iterative optimization mechanism. As the number of iterations increases, the global model gradually converges, and the ability to detect deep fake content will also be significantly improved.

[0055] Dynamic adjustment and optimization: The system supports dynamic adjustment of training parameters (such as learning rate, batch size, etc.) according to the performance of each node, network conditions, and convergence of the global model to optimize training efficiency and model performance.

[0056] (7) Application scenarios and advantages

[0057] Wide applicability: This method can be applied to a variety of scenarios that require deep fake detection, including but not limited to social media content review, judicial evidence collection, and political propaganda authenticity verification.

[0058] Strong privacy protection: By only sharing model parameters instead of original data, user privacy and data security are effectively protected, complying with increasingly stringent privacy protection regulations.

[0059] Flexibility and scalability: It supports participating nodes of different sizes and fields, and can be easily expanded to a larger-scale federated learning network, further improving the robustness and detection capabilities of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0061] Figure 1 A pseudo code diagram of the algorithm of the present invention is shown;

[0062] Figure 2 A time sequence flow chart of the federated learning of the present invention is depicted;

[0063] Figure 3 A flow chart of the deep fake detection network of the present invention is shown;

[0064] Figure 4 A diagram of the derivation process of the data preprocessing calculation formula of the present invention is presented;

[0065] Figure 5 Visualizations of experimental results for different categories of images are shown;

[0066] Figure 6 The experimental results obtained by two different data preprocessing methods are compared;

[0067] Figure 7 The ROC curve of experimental results under different differential privacy budgets is shown;

[0068] Figure 8 The ROC curve of the experimental results under different random mask sizes is shown. DETAILED DESCRIPTION

[0069] In order to more clearly illustrate the purpose, technical solutions and advantages of the present invention, the present invention is described in detail in conjunction with the drawings and embodiments. It should be clear that the following specific embodiments are only used to explain the present invention, rather than to limit the present invention. Figure 1 Provide further explanation.

[0070] Example 1

[0071] A method for applying a privacy-based deep fake detection method in federated collaboration, the steps are as follows:

[0072] S1. The first step is to preprocess the data;

[0073] S2. Next, the neural network is trained. Here, the deep fake detection algorithm under the federated learning framework is used;

[0074] S2.1. Perform a wide convolution at the input layer of the model.

[0075] S2.2, after the data is mapped to the latent space Π, the latent space Π divides the channel into two branches, namely F a and F b Each branch contains multiple repeated Shuffle blocks, which are residual structure modules composed of single-core grouped convolution, channel shuffle and depth-wise separable convolution.

[0076] S2.3. Use the attention mechanism to supervise the two branches and encourage the two branches to interact with each other. The attention mechanism is introduced at the beginning of the second Shuffle block and connected to the last Shuffle block. In a network with a larger width, the importance of different channels is calculated using limited additional computing resources. At the same time, the spatial attention mechanism is used to connect the two branches for supervision.

[0077] S2.4, after channel fusion, more expressive spatial flow and channel flow information are obtained. Here, let f ca represents the channel attention mechanism, f sa represents the spatial attention mechanism, X branchA represents the input of branch A, X branchB represents the input of branch B, g and h represent the convolution data transformation on branches A and B respectively;

[0078] The branch block is summarized as the following formula:

[0079] [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA ))

[0080] [f ca (X branchB )+f sa (X branchA )]·g(h(x branchB ))

[0081] For channel attention, the input is compressed in the spatial dimension and average pooling and maximum pooling are applied simultaneously; for spatial attention, the channel is compressed to 1 and two attention maps are generated by combining maximum pooling and global pooling; the end of the model contains eight Shuffle blocks, and finally a fully connected layer is connected to output the final real / fake classification result.

[0082] S3. Apply random masking and differential privacy noise to the uploaded model locally on each client node.

[0083] The timing flow of the federated learning framework is as follows:

[0084] A1. Assume there are k∈{1,...,K} nodes, and each node k stores a data set D k , containing n k samples, the data of each node do not intersect, the complete data set D = ∪ k∈[k] D k contain Sample points;

[0085] A2. Let W represent the model parameters, Let G denote the global objective function, G k represents the local objective function, D represents the data set, and D k represents the kth data set, p k represents the kth weight, W represents the global model parameter; there are global optimization objective function G and local optimization objective function {G k :k=1,...,K}:

[0086]

[0087] A3. Calculate the optimal w.

[0088] The steps to calculate the optimal W are as follows:

[0089] A3.1. Randomly initialize or use pre-trained parameters;

[0090] A3.2. In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain new local model parameters

[0091]

[0092] A3.3, using the stochastic gradient descent algorithm, assume represents the model parameters of the kth client in the tth round, W t-1 represents the global model parameters of the t-1th round, Indicates the calculation gradient, G k represents the objective function of the kth client, D k represents the data set of the kth client; the local update made by the kth client in the tth round can be expressed as follows:

[0093]

[0094] After completing the current round of local iterative updates, the node will randomly mask the local model parameters and add differential privacy noise, and then upload the processed local model parameters to the central server. The central server will aggregate the model parameters uploaded by all nodes in a federated average manner.

[0095]

[0096] The specific steps of the pre-processing are as follows:

[0097] S1.1. At each node, use local data to calculate local statistics. express;

[0098] S1.2. Each node uploads its local statistical value to the central server, and the central server uses these local statistical values ​​to calculate the global statistical value. mean represents the mean of the population, var represents the variance of the population, and n k Express the number of k-th data, represents the mean of the kth data, Represents the variance of the kth data.

[0099] The calculation formula is as follows:

[0100]

[0101] The steps to implement a random mask body are as follows:

[0102] At the end of each iteration, before uploading the model, the node sets the model parameters to null with probability P, where 0≤P≤30%.

[0103] When uploading the model to the central server, the model parameters are uploaded in a semi-complete form, which means that only some parameters are visible or complete, and other parameters may be encrypted or hidden.

[0104] The specific steps to implement differential privacy noise are as follows:

[0105] For the last local iteration of each global round, a differential privacy mechanism is applied to add a small amount of noise to the updated parameters. Make it become As the parameter to be uploaded.

[0106] Example 2

[0107] As a new embodiment, or a supplement to Embodiment 1.

[0108] (1) Federated Learning Framework

[0109] This algorithm is a deep fake detection algorithm under federated learning. Now we will first explain the timing process of the federated learning framework.

[0110] Assume there are k∈{1,...,K} nodes, and each node k stores a data set D k , containing n k samples, the data of each node do not intersect, the complete data set D = ∪ k∈[k] D k contain Sample points;

[0111] Let W denote the model parameters, Let G denote the global objective function, G k represents the local objective function, D represents the data set, and D k represents the kth data set, p k represents the kth weight, W represents the global model parameter; there are global optimization objective function G and local optimization objective function {G k :k=1,...,K}:

[0112]

[0113] Next, the optimal w is obtained by calculation.

[0114] Randomly initialize or use pre-trained parameters;

[0115] In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain new local model parameters

[0116] Using the stochastic gradient descent algorithm, represents the model parameters of the kth client in the tth round, W t-1 represents the global model parameters of the t-1th round, Indicates the calculation gradient, G k represents the objective function of the kth client, D k represents the data set of the kth client; the local update made by the kth client in the tth round can be expressed as follows:

[0117]

[0118] After completing the current round of local iterative updates, the node will randomly mask the local model parameters and add differential privacy noise, and then upload the processed local model parameters to the central server. The central server will aggregate the model parameters uploaded by all nodes in a federated average manner.

[0119]

[0120] According to the flowchart in Figure 2, the whole process is as follows: ① The central server sends the model parameters of the previous round (or initial) to each client as the initial parameters of the local model; ② Each client node uses local data and model parameters to perform several rounds of local iterative updates to obtain new local model parameters; ③ After each node completes the local update, it uploads the model parameters to the central server in a specific way; ④ The central server aggregates the local model parameters uploaded by each node to obtain a new round of global model parameters. This process is repeated until the model converges and training stops.

[0121] (2) Data preprocessing

[0122] In traditional federated learning methods, data preprocessing such as regularization is usually based only on the local statistical characteristics of each node (such as mean and variance) because the data cannot be exchanged with each other. However, due to the heterogeneity of data in federated learning, such as color, brightness, and forgery methods, independent regularization may cause serious deviation of data and cause deviation between local gradient and global gradient. Under the premise of privacy protection, the present invention introduces a global data preprocessing method FZSN (federated Z-score normalization) to solve the above problems.

[0123] Specifically, suppose there is a central server and m user nodes, each of which stores {n 1 ,...,n m} data, the total is First, each node uses its local data to calculate local statistics (mean, variance), and then uses Then each node uploads the statistical value to the central server, and the central server uses these local statistical values ​​(excluding any original data) to calculate the global statistical value. The calculation formula is as follows:

[0124]

[0125] in This formula is obtained by us through derivation. The derivation process is detailed in the attachment. Figure 4 shown.

[0126] (3) Deep Fake Detection Network

[0127] according to Figure 3As shown, the model of the present invention uses the channel shuffle and group convolution ideas in ShuffleNet as the basic repetitive unit. There are three reasons why the model of the present invention adopts a shallow network structure: first, from the perspective of features, at the medium level, the significance of deep fake artifacts is the highest, while at the high-level semantics, the diversity of features leads to poor generalization of the method; second, considering the needs of federated learning, it is often necessary to deal with devices with low computing performance, so shallow networks are more suitable for running on lightweight devices; finally, as a distributed model, shallow networks have fewer model parameters, which is conducive to reducing the problem of communication overhead.

[0128] In the input layer, a large-width convolution operation is performed to ensure that all features are fully expressed and reduce the information loss in the pooling stage. Then, the input data is mapped to the latent space Π. In the latent space Π, the channel is divided into two branches, namely F a and F b , each branch contains multiple repeated Shuffle blocks. Each Shuffle block consists of single-core grouped convolution, channel shuffle and depth-wise separable convolution (DWConv), forming a residual structure module.

[0129] An attention mechanism is introduced to supervise and realize the interaction between the two branches. The attention mechanism is connected to the last shuffle block at the beginning of the second shuffle block. For wide networks, only very little additional computing resources are needed to calculate the importance of different channels. In addition, a cross-branch spatial attention mechanism is introduced. Since the information in the two branches is complementary, the information of one branch is introduced into the other branch as supervision. After channel fusion, more expressive spatial flow and channel flow information can be obtained. The branch block is summarized as the following formula:

[0130] [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA ))

[0131] [f ca (X branchB )+f sa (X branchA )]·g(h(x branchB ))

[0132] For channel attention, the input is processed by compressing it in the spatial dimension and combining average pooling and max pooling. For spatial attention, the channels are also compressed to 1, and the max pooling and global pooling are combined to generate two attention maps. The end of the model includes eight modules connected to the shuffle blocks selected above. Finally, after a fully connected layer, the final real / fake classification result is obtained.

[0133] (4) Random Masking and Differential Privacy

[0134] In order to address the privacy protection and privacy measurement issues, the model uploaded by each client node is processed locally, including adding random masks and differential privacy noise. The specific process can be found in Figure 1 Pseudo code shown.

[0135] Differential privacy is a widely used privacy protection method that has become a standard. At the end of each local round, the user uses all local data to train the model. When a user completes e local rounds of training, the node is said to have completed a global round of training. In the last local iteration of each global round, a differential privacy mechanism, namely DP-SGD, is used. A small noise is added to the updated parameters (gradients). Make it become As the parameter to be uploaded. For example: If M is an additive mechanism M(W) = W + N, and N is Gaussian or Laplace noise. To ensure that the magnitude of the gradient remains constant, the L2 norm of the maximum gradient is set to 1, and the constant is set to 1e-5, which is the inverse order of the dataset size. The magnitude of the noise is set to 2.

[0136] In order to consider data security and communication effectiveness, the invention introduces a random mask into the model. In the last round of each iteration, there is a certain probability that some model parameters will be set to empty, and then the model will be uploaded to the central server in a semi-complete form. This approach effectively reduces the number of model parameters, increases the difficulty of deriving data from model parameters, and improves data security. Combined with the differential privacy random additive noise mechanism, the association between model parameters and training data becomes more ambiguous, so privacy is protected to a greater extent. For specific quantitative analysis experiments on privacy protection, please refer to Example 3.

[0137] Example 3

[0138] This example is a supplement to Example 1 and Example 2 and is used to illustrate the experimental data.

[0139] (1) Dataset used in the experiment

[0140]

[0141]

[0142] (2) Based on Figure 5 The visualization results show that the experimental results of different categories of images indicate that the trained model mainly focuses on specific parts of the face.

[0143] (3) From Figure 6 It can be observed that under the condition of non-IID, the experimental results obtained after using two different data preprocessing methods. The experimental preprocessing method proposed in the present invention can accelerate the convergence of the model and make the loss function converge more completely.

[0144] (4) On the FaceForensics++ dataset, the present invention evaluated the model performance and compared it with other methods.

[0145]

[0146] Both training and testing are performed on the same dataset

[0147] (5) The experimental results used several other typical datasets and adopted the method of training and testing the model on the same dataset.

[0148]

[0149]

[0150] (6) Cross-database testing

[0151] The training set uses the following three distribution methods:

[0152] 1. After the FF++ dataset is fully mixed, it is distributed to 5 clients to jointly train the joint model.

[0153] 2. Each client holds fake images and real image data generated by a forging method, and aggregates them to obtain a global model.

[0154] 3. Each client holds the fake images and real image data generated by the two forging methods, and aggregates them to obtain the global model.

[0155] Then we test the four different categories of images in the test set.

[0156]

[0157] Each client holds a dataset generated using the DeepFake method, which is aggregated to form a joint model.

[0158]

[0159]

[0160] (7) Under different privacy budgets (epsilon) of differential privacy, the ROC curve of the experimental results is as follows: Figure 7 shown.

[0161] The ROC curve of the experimental results with different random mask sizes is as follows: Figure 8 shown.

[0162] The above only represents a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Within the technical scope disclosed by the present invention, any technician familiar with the technical field can implement the present invention by equivalent substitution or modification, and these changes should be included in the protection scope of the present invention.

Claims

1. An application method of a detection method for privacy-oriented deep fakes in federated collaboration, characterized in that: Here are the steps: S1, the first step is to preprocess the data; Each node uses its own local data to calculate local statistics. Then each node uploads the statistical value to the central server, and the central server uses these local statistical values ​​to calculate the global statistical value. The calculation formula is as follows: in S2. Next, the neural network is trained using a deep fake detection algorithm under the federated learning framework; S2.

1. Perform a wide convolution at the input layer of the model. S2.2, after the data is mapped to the latent space Π, the latent space Π divides the channel into two branches, namely F a and F b ; Each branch contains multiple repeated Shuffle blocks, and the Shuffle block is a residual structure module composed of single-core group convolution, channel shuffle and depth-separable convolution; S2.3, use the attention mechanism to supervise the two branches and promote the interaction between the two branches; introduce the attention mechanism at the beginning of the second Shuffle block and connect it with the last Shuffle block; in a network with a larger width, use limited additional computing resources to calculate the importance of different channels; at the same time, use the spatial attention mechanism to connect the two branches for supervision; S2.4, after channel fusion, more expressive spatial flow and channel flow information are obtained. Here, let f ca represents the channel attention mechanism, f sa represents the spatial attention mechanism, X branchA represents the input of branch A, X branchB represents the input of branch B, g and h represent the convolution data transformation on branches A and B respectively; The branch block is summarized as the following formula: [f ca (X branchA )+f sa (X branchB )]·g(h(x branchA )) [f ca (X branchB )+f sa (X branchA )]·g(h(x branchB )) For channel attention, the input is compressed in the spatial dimension and average pooling and maximum pooling are applied simultaneously; for spatial attention, the channel is compressed to 1 and two attention maps are generated by combining maximum pooling and global pooling; the end of the model contains eight Shuffle blocks, and finally a fully connected layer is connected to output the final real / fake classification result; S3. Implement random masking and differential privacy noise on the uploaded model locally on each client node.

2. The application method of the privacy-oriented deep fake detection method in federated collaboration as claimed in claim 1, wherein the timing flow of the federated learning framework is as follows: A1. Assume there are k∈{1,...,K} nodes, and each node k stores a data set D k , containing n k samples, the data of each node do not intersect, the complete data set D = ∪ k∈[k] D k contain Sample points; A2. Let W represent the model parameters, Let G denote the global objective function, G k represents the local objective function, D represents the data set, and D k represents the kth data set, p k represents the kth weight, W represents the global model parameter; there are global optimization objective function G and local optimization objective function {G k :k=1,...,K}: A3. Calculate the optimal w.

3. According to the method for applying the privacy-oriented deep fake detection method in federated collaboration as claimed in claim 2, the steps for calculating the optimal W are as follows: A3.

1. Randomly initialize or use pre-trained parameters; A3.

2. In each round t∈{1,2,...,T}, the central server transmits the global model parameters W obtained by the previous round of aggregation to all nodes. t-1 , which is applied to the local objective function G k , and use local data to perform r rounds of local iterative updates on the local model to obtain the new local model parameters W t k ; A3.3, using the stochastic gradient descent algorithm, let W t k represents the model parameters of the kth client in the tth round, W t-1 represents the global model parameters of the t-1th round, ▽ represents the calculated gradient, G k represents the objective function of the kth client, D k represents the data set of the kth client; the local update made by the kth client in the tth round can be expressed as follows: W t k ←W t-1 -▽G k (D k :W t-1 ) After completing the current round of local iterative updates, the node will randomly mask the local model parameters and add differential privacy noise, and then upload the processed local model parameters to the central server; the central server will aggregate the model parameters uploaded by all nodes in a federated average manner. W t ←h(W t 1 ,...,W t K ).

4. The application method of the privacy-oriented deep fake detection method in federated collaboration as claimed in claim 1, wherein a central server and m user nodes are provided, and each node stores {n1, ..., n m} data, the total is The specific steps of the pre-processing are as follows: S1.

1. At each node, use local data to calculate local statistics. express; S1.

2. Each node uploads its local statistical value to the central server, and the central server uses these local statistical values ​​to calculate the global statistical value. mean represents the mean of the population, var represents the variance of the population, and n k Express the number of k-th data, represents the mean of the kth data, represents the variance of the kth data, The calculation formula is as follows:

5. The method for applying the privacy-oriented deep fake detection method in federated collaboration as claimed in claim 1, wherein the steps of implementing the random mask body are as follows: At the end of each iteration, before uploading the model, the node sets the model parameters to empty with probability P, where 0≤P≤30%; When uploading the model to the central server, the model parameters are uploaded in a semi-complete form, which means that only some parameters are visible or complete, and other parameters may be encrypted or hidden.

6. The application method of the privacy-oriented deep fake detection method in federated collaboration as claimed in claim 1, wherein the specific steps of implementing differential privacy noise are as follows: For the last local iteration of each global round, a differential privacy mechanism is applied to add a small amount of noise to the updated parameters. Make it become As the parameter to be uploaded.

Images