A method for detecting deep fake images

Abstract

The present invention discloses a method for detecting deep fakes in the field of image recognition technology, aiming to solve the technical problem of incompatibility between detection efficiency and detection accuracy. The method includes: the method for detecting deep fakes in the image provided by the present application reduces the detection dimension and improves the detection efficiency by extracting features in the image through binary convolution, which satisfies the need for use on devices with low computing power; extracts frequency domain features through binary fast Fourier transform, which complements the spatial domain features, and exploits the complementarity between spatial domain and frequency domain features to enhance the recognition of high-quality fake content; jump connections retain the original information, and weight distribution dynamically strengthens key features, alleviating the problem of accuracy loss. The number of module stacking can be freely controlled, making it applicable to different devices.

Description

Technical Field

[0001] The present invention relates to a method for detecting deep forgery in an image, and belongs to the technical field of image recognition. Background Art

[0002] Deepfake technology creates fake images of real people by synthesizing or tampering with images, audio, and video content. Image deepfake detection methods are used to detect forged images.

[0003] Existing deep detection models mainly rely on more complex models such as densely connected networks (DenseNet), visual transformers (ViT), and graph convolutional networks (GCN) to achieve higher recognition performance. Existing deep detection models are generally deployed on high-performance professional computers or servers. However, deep fake images mainly appear on social media platforms or web applications, and devices such as mobile phones or personal computers have limited computing resources, making it difficult to detect deep fakes. Deep detection methods that require lower computing resources improve model processing efficiency by reducing the amount of calculation, reducing the number of parameters, and accelerating inference. However, the performance of the model in deep fake detection tasks decreases.

[0004] Therefore, while existing image deepfake detection methods improve detection efficiency, they also suffer from a significant loss in detection accuracy. Summary of the Invention

[0005] The purpose of this application is to overcome the shortcomings of the existing technology and provide an image deep fake detection method that improves detection efficiency while reducing the loss of detection accuracy.

[0006] To achieve the above objectives, this application is implemented using the following technical solutions:

[0007] In a first aspect, the present application provides a method for detecting deep fake images, comprising:

[0008] Acquire an image to be detected, and extract a first feature map from the image to be detected based on binary convolution;

[0009] Inputting the first feature map extracted based on binarization convolution into a detection model based on multiple serially connected frequency recoupled binary modules to obtain a deep feature map, and obtaining a detection score for obtaining a detection result according to the deep feature map;

[0010] Wherein, at least one of the frequency recoupling binary modules is configured to perform the following operations:

[0011] A. Use a binary convolution layer to obtain a binary convolution from the second feature map of the input module;

[0012] B. Mining the frequency features of the binary convolution through binary fast Fourier transform, and using the mined frequency features as the main path features;

[0013] C. Performing a skip connection by adding element-by-element on the second feature map and the main path feature, and obtaining a weight distribution based on the features after the skip connection;

[0014] D. The main path feature is multiplied element-by-element by the weight distribution to obtain the third feature map output by the module.

[0015] In some embodiments of the first aspect, obtaining the weight distribution according to the features after the jump connection includes:

[0016] Use the global average pooling layer to convert the features after the jump connection into the first one-dimensional vector;

[0017] Use multiple fully connected layers to compress and expand the first one-dimensional vector;

[0018] The Sigmoid function is used to calculate the weight distribution of the first one-dimensional vector output by the fully connected layer.

[0019] In some embodiments of the first aspect, mining the frequency features of the binary convolution by binary fast Fourier transform to obtain the main path features includes:

[0020] Performing fast Fourier transform on the binary convolution to obtain transformed features;

[0021] extracting an amplitude vector and a phase vector from the transformed features;

[0022] extracting the frequency domain features of the amplitude vector and the frequency domain features of the phase vector respectively;

[0023] Reconstructing image space features from the frequency domain features of the amplitude vector and the frequency domain features of the phase vector using inverse Fourier transform;

[0024] The main path feature is obtained by residual connection between the reconstructed image space feature and the input second feature map.

[0025] In some embodiments of the first aspect, obtaining a detection score for obtaining a detection result according to the deep feature map includes:

[0026] Processing the deep feature map into a second one-dimensional vector using a global average pooling layer;

[0027] activating the second one-dimensional vector using a parameterized rectified linear unit to obtain an activated vector;

[0028] The activated vector is passed through a fully connected layer to obtain a detection score.

[0029] In some embodiments of the first aspect,

[0030] generating a detection model configured with a plurality of frequency recoupled binary modules connected in series, obtaining a trained detection model through multiple training iterations, and inputting the first feature map extracted based on the binarized convolution into the trained detection model to obtain a deep feature map;

[0031] Each iteration of the detection model includes:

[0032] Inputting a training set into the detection model to obtain a prediction result;

[0033] Compare the predicted results with the true labels of the corresponding samples in the training set, and calculate the loss value generated by the comparison according to the following binary cross entropy loss function:

[0034] ,

[0035] Where, is the loss value, is the counting sequence number of the samples in the training set, For the The label value of each sample is 0 or 1. To predict the The probability that a sample is 1, is the number of samples;

[0036] Starting from the loss value, the gradient of the model parameter loss of the detection model is calculated backward layer by layer;

[0037] According to the gradient direction and the preset learning rate, the detection model parameters are adjusted to minimize the loss value.

[0038] In some embodiments of the first aspect, acquiring the image to be detected and extracting the first feature map from the image to be detected based on binary convolution includes:

[0039] Get the RGB image of the image to be detected;

[0040] Obtaining a local binary pattern image of the RGB image according to an LBP algorithm;

[0041] The RGB image and the local binary pattern map are spliced ​​according to the channel dimension to form four-channel input data, and a binary convolution is performed on the four-channel input data to obtain the first feature map extracted based on the binary convolution.

[0042] In some embodiments of the first aspect, the output features of the first frequency recoupling binary module in the multiple serially connected frequency recoupling binary modules are residually connected with the input features thereof by element-by-element multiplication to become the input features of the next frequency recoupling binary module.

[0043] In a second aspect, the present application also provides a computer device comprising a processor and a memory connected to the processor, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the steps of the image deep fake detection method as described in any embodiment of the first aspect are performed.

[0044] In a third aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image deep fake detection method described in any embodiment of the first aspect.

[0045] In a fourth aspect, the present application also provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the image deep fake detection method described in any embodiment of the first aspect are implemented.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The image deep fake detection method provided in this application reduces the detection dimension and improves the detection efficiency by extracting features in the image through binary convolution, which meets the requirements for application on devices with low computing power; extracts frequency domain features through binary fast Fourier transform, which complements the spatial domain features, and exploits the complementarity of spatial domain and frequency domain features to enhance the recognition of high-quality fake content; jump connections retain the original information, and weight distribution dynamically strengthens key features, alleviating the problem of precision loss, and the number of module stacking can be freely controlled, making it suitable for different devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 is a flowchart of the steps of the image deep fake detection method provided in this embodiment;

[0050] Figure 2 This is a model framework diagram of the image deep fake detection method;

[0051] Figure 3 yes Figure 2 Framework flow chart of the medium frequency recoupling binary module;

[0052] Figure 4 yes Figure 3 Framework flow chart of fast Fourier transform;

[0053] Figure 5 It is a flowchart of the steps of obtaining the forged result of the face image to be detected based on the face forgery dataset;

[0054] Figure 6 This is a schematic block diagram of the principles of the computer device provided in this embodiment.

[0055] Description of the drawings: Figure 1 (a) shows the main execution steps of the image deep fake detection method, and (b) shows the execution steps of the frequency recoupling binary module. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Unless there is a conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0057] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this document generally indicates an "or" relationship between the related objects.

[0058] Example 1:

[0059] Figure 1 This is a flow chart of a method for detecting deep fake images in the first embodiment of the present invention. This flow chart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the present invention, different methods may be used without conflict. Figure 1 The steps shown or described are accomplished in the order shown.

[0060] The image deep fake detection method provided in this embodiment can be applied to a terminal, such as any smart phone, tablet computer or computer device with communication function. Figure 1 , the method of this implementation specifically includes the following steps:

[0061] Obtain an image to be detected and extract a first feature map from the image to be detected based on binary convolution. The first feature map extracted based on binary convolution can be called a shallow feature map. Extracting the first feature map from the image to be detected based on binary convolution greatly reduces the amount of computation and memory usage, making it suitable for real-time detection on mobile terminals or edge devices. In addition, the shallow features of the shallow feature map (such as edges and textures) provide a basis for subsequent deep analysis.

[0062] Inputting the first feature map extracted based on binarized convolution into a detection model based on multiple serially connected frequency recoupling binary modules to obtain a deep feature map, and obtaining a detection score for obtaining a detection result based on the deep feature map; by stacking multiple frequency recoupling modules (F-RBMs) to form a consistent lightweight architecture, deep features are gradually extracted, thereby enhancing the model's ability to capture complex forgery traces while reducing the requirements for the instantaneous computing power of the onboard device;

[0063] Wherein, at least one of the frequency recoupling binary modules is configured to perform the following operations:

[0064] A. Use a binary convolution layer to obtain a binary convolution from the second feature map of the input module; the binary convolution layer further compresses features within the module to maintain computational efficiency;

[0065] B. Mining the frequency features of the binary convolution through binary fast Fourier transform, and using the mined frequency features as main path features; performing fast Fourier transform / and fast Fourier transform under the binary constraint to reduce frequency domain calculation overhead;

[0066] C. Performing a skip connection by adding element-by-element to the second feature map and the main path feature, and obtaining a weight distribution based on the features after the skip connection. The input features are added to the main path features through the skip connection, preserving the original information, alleviating the information loss problem caused by binarization, and thus alleviating the overall loss of detection accuracy. Furthermore, a weight distribution is generated based on the features after the skip connection, adaptively strengthening key areas (such as forged boundaries and abnormal textures) and suppressing irrelevant noise.

[0067] D. The third feature map output by the module is obtained by element-by-element multiplication of the main path features and the weight distribution. The main path features are dynamically scaled through the weight distribution to enhance the significance of effective features, optimize the model's sensitivity to subtle forged signals, and alleviate the overall problem of loss in detection accuracy.

[0068] It is worth noting that in the frequency recoupling binary modules connected in series, the third feature map output by the previous frequency recoupling binary module is directly used as the second feature map input to the next frequency recoupling binary module or is simply processed thereafter.

[0069] In summary, the image deep fake detection method provided in this embodiment reduces the detection dimension and improves the detection efficiency by extracting features in the image through binary convolution, which meets the requirements for application on devices with low computing power; extracts frequency domain features through binary fast Fourier transform, which complements the spatial domain features, and exploits the complementarity of spatial domain and frequency domain features to enhance the recognition of high-quality fake content; jump connections retain the original information, and weight distribution dynamically strengthens key features, alleviating the problem of precision loss, and the number of module stacking can be freely controlled, making it suitable for different devices.

[0070] Example 2:

[0071] This embodiment provides a method for detecting deep fake images. This embodiment is optimized based on the first embodiment to improve the technical effect and refine the technical solution. For details not fully described in this embodiment, please refer to the first embodiment.

[0072] As one embodiment, the weight distribution is obtained according to the features after the jump connection, including: Figure 3 The perceptual compression and expansion branch on the right side of the middle obtains the weight distribution, specifically:

[0073] The global average pooling layer is used to convert the features after the jump connection into the first one-dimensional vector; the global average pooling layer can reduce the input dimension of the subsequent fully connected layer, significantly reducing the amount of calculation;

[0074] Use multiple fully connected layers to compress and expand the first one-dimensional vector; through two-step full connection, reduce parameter redundancy while retaining key information and avoid overfitting

[0075] The Sigmoid function is used to calculate the weight distribution of the first one-dimensional vector of the fully connected layer output; the Sigmoid function is used to map the fully connected layer output to the [0,1] interval to generate channel attention weights to indicate the importance of each channel.

[0076] In this embodiment, the perception compression and expansion branch takes the preceding and following features as inputs at the same time, perceives the change in information flow, and thus adaptively adjusts the output distribution of the main path features.

[0077] As one embodiment, the method of mining the frequency characteristics of the binary convolution by binary fast Fourier transform to obtain the main path characteristics includes:

[0078] Performing fast Fourier transform on the binary convolution to obtain transformed features;

[0079] Extracting an amplitude vector and a phase vector from the transformed features; extracting frequency domain features of the amplitude vector and the phase vector, respectively; obtaining the amplitude vector, which can reflect signal energy distribution (such as synthetic artifacts and discontinuous textures) and capture high-frequency noise in forged images, while the obtained phase vector reflects encoded structural information and identifies geometric anomalies in forged content (such as distorted facial contours and unnatural lighting); the amplitude vector and the phase vector achieve cross-domain feature complementarity, covering multi-dimensional representations for detecting forged traces, and can detect both high-frequency noise and low-frequency structural anomalies, thereby enhancing robustness against complex forgery methods (such as GAN and Diffusion models);

[0080] The frequency domain features of the amplitude vector and the frequency domain features of the phase vector are combined using inverse Fourier transform to reconstruct the image spatial features. This step forms a closed-loop process with the previous binary convolution and fast Fourier transform, avoiding the disconnection between frequency domain and spatial domain features and improving model interpretability.

[0081] The main path features are obtained through the residual connection between the reconstructed image spatial features and the second feature map of the input; through this step, the frequency domain analysis results (reconstructed features) and the spatial domain input features are added through the residual connection to achieve cross-domain feature fusion; in addition, the residual structure alleviates the gradient vanishing problem in deep network training and improves the model convergence stability.

[0082] This embodiment significantly improves the detection accuracy of the lightweight model through the collaboration of frequency domain deep mining and spatial domain residual optimization.

[0083] In addition, the binary fast Fourier transform can be expressed as:

[0084] ,

[0085] Where, is the output result of the binary fast Fourier transform, that is, the transformed feature, is the input image, that is, binary convolution, and are the height and width of the input image, and are the coordinates of the input image, and is the coordinate in Fourier space, is an imaginary unit;

[0086] For the convenience of calculation, it can be further expressed as:

[0087] ,

[0088] Where, is a real number, is an imaginary number;

[0089] Therefore, the amplitude and phase can be calculated by the following formulas:

[0090] ,

[0091] Where, is the amplitude, For phase.

[0092] This embodiment uses binary convolution multiple times. As one embodiment, the binary neural network model uses the following symbolic function to binarize weights and activation values ​​or other values:

[0093] ,

[0094] Where, To input content, is the output of the binary neural network model, To judge the action.

[0095] As one embodiment, obtaining a detection score for obtaining a detection result based on the deep feature map includes:

[0096] Processing the deep feature map into a second one-dimensional vector using a global average pooling layer;

[0097] activating the second one-dimensional vector using a parameterized rectified linear unit to obtain an activated vector;

[0098] The activated vector is passed through a fully connected layer to obtain a predicted probability, which is used as a detection score.

[0099] As one example, refer to Figure 5 , generating a detection model configured with multiple frequency re-coupled binary modules connected in series, obtaining a trained detection model through multiple training iterations, and inputting the first feature map extracted based on the binarized convolution into the trained detection model to obtain a deep feature map;

[0100] Each iteration of the detection model includes:

[0101] Inputting a training set into the detection model to obtain a prediction result;

[0102] Compare the predicted results with the true labels of the corresponding samples in the training set, and calculate the loss value generated by the comparison according to the following binary cross entropy loss function:

[0103] ,

[0104] Where, is the loss value, is the counting sequence number of the samples in the training set, For the The label value of each sample is 0 or 1. To predict the The probability that a sample is 1; the binary cross entropy loss function perfectly matches the binary classification task (true / fake) of deep fake detection, is the number of samples;

[0105] Starting from the loss value, the gradient of the model parameter loss of the detection model is calculated backward layer by layer;

[0106] According to the gradient direction and the preset learning rate, the detection model parameters are adjusted to minimize the loss value.

[0107] As one embodiment, the step of obtaining the image to be detected and extracting the first feature map from the image to be detected based on binary convolution includes:

[0108] Get the RGB image of the image to be detected;

[0109] Obtaining a local binary pattern map of the RGB image according to the LBP algorithm; the local binary pattern map can add more details to compensate for the problem that the RGB image is insensitive to texture, and the LBP map is robust to lighting changes and is suitable for forgery detection under complex lighting conditions;

[0110] The RGB image and the local binary pattern map are concatenated along the channel dimension to form four-channel input data. Binarized convolution is performed on this four-channel input data to obtain the first feature map extracted based on binary convolution. The combination of RGB and LBP covers features across different frequency bands in the spatial domain. The combined image facilitates the extraction of frequency features by subsequent binary convolution layers, improving the model's ability to detect complex forgeries.

[0111] In one embodiment, the output features of the first frequency-recoupling binary module in the series of multiple frequency-recoupling binary modules are residually connected by element-by-element multiplication with its input features, and then become the input features of the next frequency-recoupling binary module. This step is intended to calibrate the output of the first frequency-recoupling binary module. This operation (element-by-element multiplication) facilitates feature fusion and propagation between modules.

[0112] As one embodiment, after acquiring the image to be detected, the longest dimension is normalized to 256 pixels for ease of processing.

[0113] In addition, the data in the training set needs to be augmented, including random flipping (horizontally and vertically), rotation (90° or 270°), and color jittering within the range of 0% to 20%. The COCOFake dataset can be used as the training set.

[0114] As one example, refer to Figure 2 , a deep fake detection network can be constructed as a whole to perform the main steps of the image deep fake detection method provided in this embodiment. Figure 2 In the displayed deep fake detection network, the left side shows the RGB image and the LBP image obtained based on the RGB image, that is, the local binary pattern map. 3×3 is the convolution kernel size, and the same applies later. In this embodiment, a shallow feature extraction network is constructed based on two serial frequency recoupling binary modules and quantized convolution layers to extract feature maps from the image to be detected. The product is a basic binary neural network whose input is composed of n serial frequency recoupling modules. And after each convolution layer, a normalization layer and an element-by-element addition residual operation are adaptively set. Finally, a global average pooling layer, an activation function and a quantized linear layer are used to process it into a predicted probability, and the predicted probability is used as the detection score.

[0115] refer to Figure 3 The left side is the main path, n×n is the convolution kernel size of the binary convolution layer, and the main path features and input features are weighted through the perceptual compression and expansion branch on the right.

[0116] Figure 4 It is a further explanation of the binary fast Fourier transform module.

[0117] This example uses a 64-bit Ubuntu 18.04.5 operating system, Python 3.8.0, torch 1.9.1, torchvision 0.10.1, and an NVIDIA RTX 3090 GPU to complete the model training. The AdamW optimizer is used, and the initial learning rate is , the first moment estimate and the second moment estimate are set to 0.9 and 0.999 respectively, and the weight decay parameter is .

[0118] The image deep fake detection method provided in this embodiment uses a binary neural network to predict the detection results. Compared with other deep fake detection models based on architectures such as Transformer or graph convolution, the weights and activation values ​​in the network are binarized, which can effectively improve the computational efficiency of the model; it also perceives changes in frequency domain and texture information through re-coupling, and adaptively adjusts the feature output distribution, which helps to improve the detection accuracy of the model; this embodiment also proposes a binary fast Fourier transform module, which extracts frequency domain features from the phase and amplitude dimensions respectively, and mines the forgery traces of the image in the frequency domain, which helps to improve the robustness of the forgery detection model.

[0119] Example 3:

[0120] This embodiment provides a computer device, including a processor and a memory connected to the processor, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the steps of the image deep fake detection method provided in Embodiment 1 or 2 are performed.

[0121] The computer device may be a server or an electronic terminal. As one embodiment, Figure 6 , the computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data obtained and generated in the image deep fake detection method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the image deep fake detection method provided in embodiment one or two is implemented.

[0122] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0123] The computer device provided in this embodiment has the same technical effects as those in Embodiment 1 or 2, and will not be described in detail here.

[0124] Example 4:

[0125] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the image deep fake detection method provided in Embodiment 1 or Embodiment 2 are implemented.

[0126] The computer-readable storage medium provided in this embodiment has the same technical effects as those in Embodiment 1 or 2, and will not be described in detail here.

[0127] Embodiment 5:

[0128] This embodiment provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the steps of the image deepfake detection method provided in Embodiment 1 or Embodiment 2. The computer program product provided in this embodiment can be transmitted, distributed, and downloaded in the form of a signal via the Internet.

[0129] The computer program product provided in this embodiment has the same technical effects as those in Embodiment 1 or 2, and will not be described in detail here.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified with "first," "second," etc., may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0135] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0136] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting deep fake images, characterized in that: include, Acquire an image to be detected, and extract a first feature map from the image to be detected based on binary convolution; Inputting the first feature map extracted based on binarization convolution into a detection model based on multiple serially connected frequency recoupled binary modules to obtain a deep feature map, and obtaining a detection score for obtaining a detection result according to the deep feature map; Wherein, at least one of the frequency recoupling binary modules is configured to perform the following operations: A. Use a binary convolution layer to obtain a binary convolution from the second feature map of the input module; B. Mining the frequency features of the binary convolution through binary fast Fourier transform, and using the mined frequency features as the main path features; C. Performing a skip connection by adding element-by-element on the second feature map and the main path feature, and obtaining a weight distribution based on the features after the skip connection; D. multiplying the main road feature by the weight distribution element by element to obtain a third feature map output by the module; In the frequency recoupling binary modules connected in series, the third feature map output by the previous frequency recoupling binary module is directly used as the second feature map input by the next frequency recoupling binary module.

2. The image deepfake detection method according to claim 1, characterized in that: The weight distribution is obtained according to the features after the jump connection, including: Use the global average pooling layer to convert the features after the jump connection into the first one-dimensional vector; Use multiple fully connected layers to compress and expand the first one-dimensional vector; The Sigmoid function is used to calculate the weight distribution of the first one-dimensional vector output by the fully connected layer.

3. The image deepfake detection method according to claim 1, characterized in that: The method of mining the frequency characteristics of the binary convolution by binary fast Fourier transform to obtain the main path characteristics includes: Performing fast Fourier transform on the binary convolution to obtain transformed features; extracting an amplitude vector and a phase vector from the transformed features; extracting the frequency domain features of the amplitude vector and the frequency domain features of the phase vector respectively; Reconstructing image space features from the frequency domain features of the amplitude vector and the frequency domain features of the phase vector using inverse Fourier transform; The main path feature is obtained by residual connection between the reconstructed image space feature and the input second feature map.

4. The image deepfake detection method according to claim 1, characterized in that: The detection score for obtaining the detection result is obtained according to the deep feature map, include, Processing the deep feature map into a second one-dimensional vector using a global average pooling layer; activating the second one-dimensional vector using a parameterized rectified linear unit to obtain an activated vector; The activated vector is passed through a fully connected layer to obtain a detection score.

5. The image deepfake detection method according to claim 1, characterized in that: Obtain a trained detection model through multiple training iterations; Each iteration consists of: Inputting a training set into the detection model to obtain a prediction result; Compare the predicted results with the true labels of the corresponding samples in the training set, and calculate the loss value generated by the comparison according to the following binary cross entropy loss function: ， Where, is the loss value, is the counting sequence number of the samples in the training set, For the The label value of each sample is 0 or 1. To predict the The probability that a sample is 1, is the number of samples; Starting from the loss value, the gradient of the model parameter loss of the detection model is calculated backward layer by layer; According to the gradient direction and the preset learning rate, the detection model parameters are adjusted to minimize the loss value.

6. The image deepfake detection method according to claim 1, characterized in that: The step of obtaining an image to be detected and extracting a first feature map from the image to be detected based on binary convolution includes: Get the RGB image of the image to be detected; Obtaining a local binary pattern image of the RGB image according to an LBP algorithm; The RGB image and the local binary pattern map are spliced ​​according to the channel dimension to form four-channel input data, and a binary convolution is performed on the four-channel input data to obtain the first feature map extracted based on the binary convolution.

7. The image deepfake detection method according to claim 1, characterized in that: The output feature of the first frequency recoupling binary module in the multiple serially connected frequency recoupling binary modules is connected to its input feature through element-by-element multiplication by residual connection, and then becomes the input feature of the next frequency recoupling binary module.

8. A computer device, characterized in that: The method comprises a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the image deep fake detection method according to any one of claims 1 to 7 are performed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the image deep fake detection method described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the image deep fake detection method according to any one of claims 1 to 7 are implemented.

Images