Double-flow fine-grained forgery detection method based on phase characteristics

Abstract

The invention provides a double-flow fine-grained forgery detection method based on phase features, and relates to the field of image forgery detection, and the method comprises the steps: S1, obtaining a to-be-detected depth forgery data set, carrying out the image extraction of the to-be-detected depth forgery data set, and obtaining an original face image; s2, constructing a phase artifact mining module, and inputting the original face image into the phase artifact mining module for phase feature extraction to obtain a phase feature map; and S3, constructing a double-flow counterfeit detection network, inputting the original face image and the phase feature map into the double-flow counterfeit detection network, and judging whether the original face image is a counterfeit image or not. According to the invention, through a double-backbone network structure of the double-flow counterfeit detection network, feature extraction and fusion are carried out on the original face image and the phase feature map, counterfeit traces concerned by the two faces and the phase feature map are reserved, the ability of capturing common counterfeit features among different modes is enhanced, and the accuracy of counterfeit detection is further improved.

Description

Technical Field

[0001] The present invention relates to the field of image forgery detection, and in particular to a dual-stream fine-grained forgery detection method based on phase features. Background Art

[0002] In recent years, with the development of the internet, people have become increasingly aware of cybersecurity. As a booming technology, artificial intelligence (AI) has also drawn considerable attention to its security. The application of AI to video, audio, and image forgery and editing has also gradually grown, bringing techniques such as text and image generation, audio generation, face replacement, attribute editing, and facial reenactment into the public eye. The development of deepfake technology has created numerous opportunities, with applications in film and television special effects, medical cosmetology, virtual live streaming, public education, news dissemination, and artistic creation. However, the misuse of AI technology has also posed significant risks to the security of people's personal information. In particular, the use of generative adversarial networks and diffusion models to generate fake facial images and videos has caused significant public panic in recent years.

[0003] Given the range of hazards caused by the misuse of deepfake technology, deepfake detection technology has gradually developed. Early methods directly input real and fake images into traditional machine learning algorithms or simple convolutional neural networks (CNNs) for classification, which also achieved good results. This indirectly shows that forgery methods inevitably leave traces of forgery. However, with the development of forgery technology, traditional methods are unable to maintain high accuracy and are gradually being eliminated. Therefore, a large number of scholars have analyzed the vulnerabilities of deepfake technology from different perspectives on the possible existence of forgery traces and proposed many detection techniques based on different feature domains, such as:

[0004] 1. Construct a two-stream forgery detection network based on spatial and frequency domain features, and mine forgery traces through a multi-scale self-attention mechanism. However, this method uses convolution to reduce the dimensionality of the original input to reduce the huge computational complexity brought by multi-scale self-attention. This will lead to the model's insufficient ability to capture forgery features in a fine-grained manner, thus affecting detection accuracy.

[0005] 2. Based on a new perspective of frequency domain artifacts, by using spatial domain rich models to capture high-frequency noise residuals, more artifact traces are revealed. However, due to the influence of frequency domain amplitude, this method cannot deeply explore deep phase artifact traces, resulting in insufficient model generalization.

[0006] In summary, existing image forgery detection methods lack the modeling of fine-grained forgery features and the mining of common forgery traces, resulting in poor detection accuracy of forgery detection models and weak generalization ability for unknown forgery techniques. Summary of the Invention

[0007] In order to solve the above problems, the present invention provides a dual-stream fine-grained forgery detection method based on phase features, comprising the steps of:

[0008] S1: Obtain a deep fake data set to be tested, perform image extraction on the deep fake data set to be tested, and obtain the original face image;

[0009] S2: Construct a phase artifact mining module, input the original face image into the phase artifact mining module to extract phase features and obtain a phase feature map;

[0010] S3: Construct a two-stream forgery detection network, input the original face image and phase feature map into the two-stream forgery detection network, and determine whether the original face image is a forged image.

[0011] Optionally, step S1 specifically includes:

[0012] S11: extracting a video data set from the deep fake data set to be tested;

[0013] S12: extracting an initial image of each video data in the video data set at a preset interval to obtain an initial image set;

[0014] S13: performing facial positioning, alignment, and cropping on each initial image in the initial image set to obtain an original face image set;

[0015] S14: Selecting an original face image of a preset size from the original face image set.

[0016] Optional:

[0017] The phase artifact mining module includes multiple filter banks in different directions, and each filter bank includes multiple filters of different sizes.

[0018] Optionally, step S2 specifically includes:

[0019] S21: convert the original face image into a grayscale image;

[0020] S22: extract pixel x in the grayscale image;

[0021] S23: Input the pixel point x into the filter group M with direction θ θ ;

[0022] S24: Pass filter bank M θ The filter m of size n n,θ Perform frequency domain filtering on pixel point x and calculate the amplitude and phase offset of pixel point x in direction θ and dimension n;

[0023] S25: Repeat step S24 to calculate the amplitude and phase offset of all dimensions of the pixel point x in the direction θ, and calculate the phase measurement value of the pixel point x in the direction θ based on the amplitude and phase offset of each dimension in the direction θ;

[0024] S26: Repeat steps S23-S25 to calculate the phase metric values of pixel point x in all directions, normalize and merge the phase metric values in each direction, and calculate the phase eigenvalue of pixel point x;

[0025] S27: Repeat steps S22-S26 to calculate the phase eigenvalues of all pixels, and construct a phase characteristic map based on the phase eigenvalues of each pixel.

[0026] Optional:

[0027] The dual-stream forgery detection network includes: a first backbone network, a second backbone network, a correlation feature fusion module, and a classification head;

[0028] The first backbone network and the second backbone network have the same structure;

[0029] The first backbone network includes multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules, and a single dynamic fine-grained perception attention mechanism module is set between two adjacent feature extraction modules.

[0030] Optional:

[0031] The dynamic fine-grained perception attention mechanism module includes: a global average pooling module, a two-dimensional convolution layer with a convolution kernel size of 1, and a one-dimensional convolution layer with a convolution kernel size of 3.

[0032] Optional:

[0033] The dynamic fine-grained perception attention mechanism module is used to perform fine-grained transformation on the input initial feature map F to obtain the fine-grained feature map F′. The transformation process specifically includes:

[0034] The initial feature map F is converted into a global average pooling feature map U through the global average pooling module;

[0035] The global average pooling feature map U is input into the two-dimensional convolution layer, and the spatial dimension of the global average pooling feature map U is convolved to obtain the channel attention matrix U with single feature consistency. s ;

[0036] Input the global average pooling feature map U into the one-dimensional convolution layer, perform convolution operation on the channel dimension of the global average pooling feature map U, and obtain the channel attention matrix U of multiple feature correlations j ;

[0037] The channel attention matrix U of multiple feature correlations j The transposed matrix of Channel attention matrix U consistent with a single feature s Multiply and process with the softmax activation function to obtain the cross attention weight matrix F u ;

[0038] The cross attention weight matrix F u Channel attention matrix U associated with multiple features j Multiply it with the channel attention matrix U of the single feature consistency s After adding and processing with ReLU activation function, the first granularity feature U is obtained. s ';

[0039] The cross attention weight matrix F u The transposed matrix of Channel attention matrix U consistent with a single feature s Multiply it and then add it to the channel attention matrix U associated with multiple features j After adding and processing with ReLU activation function, the second granularity feature U is obtained. j ';

[0040] The first granularity feature U s ′ and the second granularity feature U j ′ is weighted fused, processed by the sigmoid activation function, and then multiplied with the initial feature map F to obtain the fine-grained feature map F′.

[0041] Optionally, step S3 specifically includes:

[0042] S31: Input the original face image into the first backbone network, and perform multiple fine-grained transformations and feature extractions on the original face image through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules in the first backbone network to obtain a first forgery detection feature map F r ;

[0043] S32: Input the phase feature map into the second backbone network, and perform multiple fine-grained transformations and feature extractions on the phase feature map through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules in the second backbone network to obtain a second forgery detection feature map F pc ;

[0044] S33: The first forgery detection feature map F r and the second forgery detection feature map F pc Input the correlation feature fusion module to obtain the fused forgery detection feature map F DSCF ;

[0045] S34: Fusion forgery detection feature map F DSCF Input the classification head and calculate the discriminant vector P; if the discriminant vector P is greater than the preset value, the original face image is judged to be a real image; otherwise, the original face image is judged to be a forged image.

[0046] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the dual-stream fine-grained forgery detection method based on phase features is implemented.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the dual-stream fine-grained forgery detection method based on phase features is implemented.

[0048] The present invention has the following beneficial effects:

[0049] 1. The phase artifact mining module uses filters of different sizes and directions to filter each pixel in the facial image in multiple directions and sizes, obtaining multi-directional phase measurement values. These values are then normalized and merged, focusing on mining deep common edge forgery traces in facial images. The resulting phase feature map has strong commonality and can clearly show obvious forgery traces, significantly improving the accuracy and generalization of forgery detection.

[0050] 2. The dynamic fine-grained perceptual attention mechanism module performs fine-grained transformation on the input feature map, reducing the network's attention granularity. This effectively addresses the problem of existing methods over-relying on global features, thereby more accurately allocating the weights of forged features and further improving the generalization of the model.

[0051] 3. Through the dual-backbone network structure of the two-stream forgery detection network, the original face image and phase feature map are extracted and then fused. While retaining the forgery traces of their respective focus, the ability to capture common forgery features between different modalities is enhanced, further improving the accuracy of forgery detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the phase feature map extraction process;

[0054] Figure 3 This is the structural diagram of the two-stream forgery detection network;

[0055] Figure 4 This is the structural diagram of the dynamic fine-grained perception attention mechanism module;

[0056] Figure 5 This is the structural diagram of the correlation feature fusion module;

[0057] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0058] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] The following will be combined with the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] Reference Figure 1 The present invention provides a dual-stream fine-grained forgery detection method based on phase features, comprising the steps of:

[0061] S1: Obtain a deep fake data set to be tested, perform image extraction on the deep fake data set to be tested, and obtain the original face image;

[0062] In some embodiments:

[0063] Step S1 specifically includes:

[0064] S11: extracting a video data set from the deep fake data set to be tested;

[0065] In some embodiments, the deep fake data sets to be tested include FaceForensic++ (c23), Celeb-DFv2, and WildDeepfake. The FacForensics++ dataset consists of 1,000 real videos extracted from Youtube and 4,000 fake videos generated by different fake methods, including Deepfake, Face2Face, FaceSwap, and NerualTextures. The Celeb-DFv2 dataset contains 590 real videos and 5,639 fake videos. The WildDeepfake dataset contains 3,805 real face videos and 3,509 fake face videos.

[0066] S12: extracting an initial image of each video data in the video data set at a preset interval to obtain an initial image set;

[0067] S13: performing facial positioning, alignment, and cropping on each initial image in the initial image set to obtain an original face image set;

[0068] In some embodiments, a face detector in dlib is used to locate the face of a person, and a face image of 256 dpi×256 dpi is aligned and cropped according to 1.5 times the face area;

[0069] S14: Selecting an original face image of a preset size from the original face image set.

[0070] S2: Construct a phase artifact mining module, input the original face image into the phase artifact mining module to extract phase features and obtain a phase feature map;

[0071] In some embodiments:

[0072] The phase artifact mining module includes multiple filter banks in different directions, and each filter bank includes multiple filters of different sizes.

[0073] In some embodiments, the phase artifact mining module includes Log-Gabor filters in five directions and six sizes in each direction.

[0074] In some embodiments, the extraction process of the phase feature map is as follows: Figure 2 As shown:

[0075] Step S2 specifically includes:

[0076] S21: convert the original face image into a grayscale image;

[0077] S22: extract pixel x in the grayscale image;

[0078] S23: Input the pixel point x into the filter group M with direction θ θ ;

[0079] S24: Pass filter bank M θ The filter m of size n n,θ Perform frequency domain filtering on pixel point x and calculate the amplitude and phase offset of pixel point x in direction θ and dimension n;

[0080] In some embodiments, a two-dimensional Fast Fourier Transform (FFT) is used to transform the image into the frequency domain, and Log-Gabor filters of different directions and sizes are applied to perform frequency domain filtering to extract phase information within a specific frequency range to eliminate the influence of background illumination and noise in the image. Finally, an inverse Fourier transform (IFFT) is performed to return to the spatial domain to obtain a filtered image. The above process can be expressed as the formula:

[0081]

[0082] Among them, ⊙ is element-by-element multiplication, e n,θ (x) and o n,θ (x) represents the convolution results of the original image I(x) with the odd-symmetric and even-symmetric Log-Gabor wavelets of size n and direction θ, respectively. e and o can be understood as the real and imaginary parts of the image. and are odd-symmetric and even-symmetric Log-Gabor wavelet filters of size n and direction θ, respectively. n,θ (x),o n,θ (x)] is solved independently to obtain the amplitude and phase shift Δφ n,θ (x);

[0083] S25: Repeat step S24 to calculate the amplitude and phase offset of all dimensions of the pixel point x in the direction θ, and calculate the phase measurement value of the pixel point x in the direction θ based on the amplitude and phase offset of each dimension in the direction θ;

[0084] In some embodiments, the filtered image I′(x)=[e n,θ (x),o n,θ (x)], this paper first solves the filtered image in each direction independently, and calculates the total amplitude of the image after filtering with n different scale filters by the formula:

[0085]

[0086] This step is the basis for subsequent calculations. Then the amplitude A is calculated by the formula n,θ (x) and phase shift Δφ n,θ (x), this step is to combine the amplitude and phase information:

[0087]

[0088] in, and They represent the weighted average phase angles of the real (e) and imaginary (o) parts of the image, respectively, and are calculated as follows:

[0089]

[0090] After obtaining the product of the image amplitude and phase offset and the image amplitude, the phase metric value PC of the image after filtering by the Log-Gabor filter of all sizes in the direction θ is calculated by the following formula: θ (x):

[0091]

[0092] Where ε is a small constant to avoid division by zero; W θ (x) is a weighting function that adjusts the contribution of filters of different sizes to the result; T θ It is a noise compensation function to reduce the impact of noise; Indicates rounding down;

[0093] S26: Repeat steps S23-S25 to calculate the phase metric values of pixel point x in all directions, normalize and merge the phase metric values in each direction, and calculate the phase eigenvalue of pixel point x;

[0094] In some embodiments, to solve the phase eigenvalue I at x pc (x), it is also necessary to integrate the filter responses of this point in all directions. In order to preserve the phase artifact traces in all directions to the greatest extent possible, the sum normalization method is used to merge the results of all directions, as shown in the formula:

[0095]

[0096] S27: Repeat steps S22-S26 to calculate the phase eigenvalues of all pixels, and construct a phase characteristic map based on the phase eigenvalues of each pixel.

[0097] S3: Construct a two-stream forgery detection network, input the original face image and phase feature map into the two-stream forgery detection network, and determine whether the original face image is a forged image.

[0098] In some embodiments, the structure of the dual-stream forgery detection network is as follows: Figure 3 As shown:

[0099] The dual-stream forgery detection network includes: a first backbone network, a second backbone network, a correlation feature fusion module, and a classification head;

[0100] The first backbone network and the second backbone network have the same structure;

[0101] The first backbone network includes multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules, and a single dynamic fine-grained perception attention mechanism module is set between two adjacent feature extraction modules.

[0102] In some embodiments, the dual-stream forgery detection network uses the optional Xception as the backbone network.

[0103] In some embodiments, the structure of the dynamic fine-grained perception attention mechanism module is as follows: Figure 4 As shown;

[0104] The dynamic fine-grained perception attention mechanism module includes: a global average pooling module, a two-dimensional convolution layer with a convolution kernel size of 1, and a one-dimensional convolution layer with a convolution kernel size of 3.

[0105] In some embodiments:

[0106] The dynamic fine-grained perception attention mechanism module is used to perform fine-grained transformation on the input initial feature map F to obtain the fine-grained feature map F′. The transformation process specifically includes:

[0107] The initial feature map F is converted into a global average pooling feature map U through the global average pooling module;

[0108] In some embodiments, given an initial feature map Where C represents the number of channels, H and W represent the height and width of the feature map respectively. The dynamic fine-grained perception attention mechanism module first converts the feature map into a global average pooling feature map through the global average pooling method. Represents the overall response of the channel dimension, which is used to further capture the fine-grained interaction between channels. The n-th channel element of U is expressed as:

[0109]

[0110] Among them, F n (i, j) is the eigenvalue of the nth channel;

[0111] The global average pooling feature map U is input into the two-dimensional convolution layer, and the spatial dimension of the global average pooling feature map U is convolved to obtain the channel attention matrix U with single feature consistency. s ;

[0112] Input the global average pooling feature map U into the one-dimensional convolution layer, perform convolution operation on the channel dimension of the global average pooling feature map U, and obtain the channel attention matrix U of multiple feature correlations j ;

[0113] The channel attention matrix U of multiple feature correlations j The transposed matrix of Channel attention matrix U consistent with a single feature s Multiply and process with the softmax activation function to obtain the cross attention weight matrix F u ;

[0114] In some embodiments, the cross-attention weight matrix F u The calculation process is:

[0115]

[0116] The cross attention weight matrix F u Channel attention matrix U associated with multiple features j Multiply it with the channel attention matrix U of the single feature consistency s After adding and processing with ReLU activation function, the first granularity feature U is obtained. s ';

[0117] In some embodiments, the first granularity characteristic U s The calculation process of ′ is:

[0118] U s ′=ReLU(F u ·U j +U s )

[0119] The cross attention weight matrix F u The transposed matrix of Channel attention matrix U consistent with a single feature s Multiply it and then add it to the channel attention matrix U associated with multiple features j After adding and processing with ReLU activation function, the second granularity feature U is obtained. j ';

[0120] In some embodiments, the second granularity characteristic U j The calculation process of ′ is:

[0121] U j ′=ReLU(F u T ·U s +U j )

[0122] The first granularity feature U s ′ and the second granularity feature U j ′ is weighted fused, processed by the sigmoid activation function, and then multiplied with the initial feature map F to obtain the fine-grained feature map F′.

[0123] In some embodiments, the calculation process of the fine-grained feature map F′ is:

[0124]

[0125] in, is the learnable fusion weight, and σ() is the sigmoid activation function.

[0126] In some embodiments:

[0127] Step S3 specifically includes:

[0128] S31: Input the original face image into the first backbone network, and perform multiple fine-grained transformations and feature extractions on the original face image through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules in the first backbone network to obtain a first forgery detection feature map F r ;

[0129] S32: Input the phase feature map into the second backbone network, and perform multiple fine-grained transformations and feature extractions on the phase feature map through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules in the second backbone network to obtain a second forgery detection feature map F pc ;

[0130] S33: The first forgery detection feature map F r and the second forgery detection feature map F pc Input the correlation feature fusion module to obtain the fused forgery detection feature map F DSCF ;

[0131] In some embodiments, the structure of the correlation feature fusion module is as follows: Figure 5 As shown, the correlation feature fusion module is used to fuse the dual-stream features, whose input is the first forgery detection feature map F r and the second forgery detection feature map F pc , the channel-level correlation matrix is obtained by calculating the cosine similarity of two feature maps in the channel dimension:

[0132]

[0133] Among them, x∈{1,2,...,H},y∈{1,2,...,W} and C x,y is the channel-level correlation value at the coordinate (x, y). Then, through correlation weighting and residual connection, the correlation-enhanced feature maps of different modalities are obtained, and a learnable parameter α is introduced to dynamically adjust the fusion ratio of the two modalities:

[0134] F DSCF =σ(α)×ReLU(F pc +C⊙F r )+(1-σ(α))×ReLU(F r +C⊙F pc )

[0135] Among them, σ is the sigmoid activation function, ⊙ is the element-by-element multiplication, and ReLU is the nonlinear activation function. The above process obtains the fused forgery detection feature map F DSCF .

[0136] S34: Fusion forgery detection feature map F DSCFInput the classification head and calculate the discriminant vector P; if the discriminant vector P is greater than the preset value, the original face image is judged to be a real image; otherwise, the original face image is judged to be a forged image.

[0137] In some embodiments, the obtained fused forgery detection feature map is used as a forgery feature vector to determine whether the image is forged, and the classification head is supervised learning using any binary classification loss; the obtained fused forgery detection feature map is passed through a classification head composed of a pooling layer, a fully connected layer, and a normalization layer to obtain a final discriminant vector P. If P>0.5, the image is considered to be a real image, and if P≤0.5, the image is considered to be a forged image.

[0138] In some embodiments, the binary classification loss function is a combination of FocalLoss and BCELoss, as shown in the following formula:

[0139] L Focal (p,y)=-αy(1-p) γ log(σ(p))-(1-α)(1-y)p γ log(1-σ(p))

[0140] Here, α is a hyperparameter used to balance classes, γ is the focusing factor of FocalLoss, p is the model's predicted value, σ(p) represents the output after sigmoid activation, and y is the true value. Specifically, γ = 2 and α = 0.6.

[0141] In some embodiments, training and testing are performed on three datasets: FaceForensic++, Celeb-DFv2, and WildDeepfake. The test comparison results of the forgery detection ability are shown in Table 1. The selected evaluation indicator ACC represents the accuracy, and AUC represents the area under the curve. The above two evaluation indicators can fully evaluate the forgery detection ability of the network. It can be seen from Table 1 that compared with other advanced forgery detection networks, the method proposed in the embodiments of the present invention can improve the ability to detect deep forgery faces using different forgery methods.

[0142] Table 1 Test comparison results of forgery detection capabilities

[0143]

[0144] In the cross-dataset generalization test, the model was trained on the FaceForensics++ dataset and then tested on the Celeb-DFv2 and WildDeepfake datasets respectively. The evaluation indicator was AUC (%). The average generalization test comparison results are shown in Table 2. The most advanced generalization performance was achieved on both the Celeb-DFv2 and WildDeepfake datasets.

[0145] Table 2. Comparison results of average generalization test

[0146]

[0147] In some embodiments, see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. An electronic device 60 provided in an embodiment of the present application includes a memory 61 and a processor 62; the memory 61 stores a computer program, wherein the computer program, when executed by the processor, implements the dual-stream fine-grained forgery detection method based on phase features.

[0148] Specifically, the processor 62 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 62 may also include onboard memory for caching purposes. The processor 62 may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiments of the present application.

[0149] Memory 61 can be, for example, any medium capable of containing, storing, conveying, disseminating, or transmitting instructions. For example, memory 61 can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, components, or propagation media. Specific examples of memory 61 include: magnetic storage devices, such as magnetic tape or hard disk drives (HDDs); optical storage devices, such as compact discs (CD-ROMs); random access memory (RAM) or flash memory; and / or wired or wireless communication links.

[0150] The present application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the described dual-stream fine-grained forgery detection method based on phase features. The computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not be incorporated into the device / apparatus / system. The computer-readable medium carries one or more programs, and when executed, implements the method described in the embodiments of the present application.

[0151] According to an embodiment of the present application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, radio frequency signals, or any suitable combination thereof.

[0152] Those skilled in the art will understand that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above-mentioned embodiments, but should be determined not only by the attached claims, but also by the equivalents of the attached claims. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A two-stream fine-grained forgery detection method based on phase features, characterized in that: Including steps: S1: Obtain a deep fake data set to be tested, perform image extraction on the deep fake data set to be tested, and obtain the original face image; S2: Construct a phase artifact mining module, input the original face image into the phase artifact mining module to extract phase features and obtain a phase feature map; S3: Construct a two-stream forgery detection network, input the original face image and phase feature map into the two-stream forgery detection network, and determine whether the original face image is a forged image.

2. The dual-stream fine-grained forgery detection method based on phase features according to claim 1 is characterized in that: Step S1 specifically includes: S11: extracting a video data set from the deep fake data set to be tested; S12: extracting an initial image of each video data in the video data set at a preset interval to obtain an initial image set; S13: performing facial positioning, alignment, and cropping on each initial image in the initial image set to obtain an original face image set; S14: Selecting an original face image of a preset size from the original face image set.

3. The dual-stream fine-grained forgery detection method based on phase features according to claim 1 is characterized by: The phase artifact mining module includes multiple filter banks in different directions, and each filter bank includes multiple filters of different sizes.

4. The dual-stream fine-grained forgery detection method based on phase features according to claim 3 is characterized in that: Step S2 specifically includes: S21: convert the original face image into a grayscale image; S22: extract pixel x in the grayscale image; S23: Input the pixel point x into the filter group M with direction θ θ ; S24: Pass filter bank M θ The filter m of size n n,θ Perform frequency domain filtering on pixel point x and calculate the amplitude and phase offset of pixel point x in direction θ and dimension n; S25: Repeat step S24 to calculate the amplitude and phase offset of all dimensions of the pixel point x in the direction θ, and calculate the phase measurement value of the pixel point x in the direction θ based on the amplitude and phase offset of each dimension in the direction θ; S26: Repeat steps S23-S25 to calculate the phase metric values of pixel point x in all directions, normalize and merge the phase metric values in each direction, and calculate the phase eigenvalue of pixel point x; S27: Repeat steps S22-S26 to calculate the phase eigenvalues of all pixels, and construct a phase characteristic map based on the phase eigenvalues of each pixel.

5. The dual-stream fine-grained forgery detection method based on phase features according to claim 1 is characterized in that: The dual-stream forgery detection network includes: a first backbone network, a second backbone network, a correlation feature fusion module, and a classification head; The first backbone network and the second backbone network have the same structure; The first backbone network includes multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules, and a single dynamic fine-grained perception attention mechanism module is set between two adjacent feature extraction modules.

6. The dual-stream fine-grained forgery detection method based on phase features according to claim 5 is characterized by: The dynamic fine-grained perception attention mechanism module includes: a global average pooling module, a two-dimensional convolution layer with a convolution kernel size of 1, and a one-dimensional convolution layer with a convolution kernel size of 3.

7. The dual-stream fine-grained forgery detection method based on phase features according to claim 6 is characterized in that: The dynamic fine-grained perception attention mechanism module is used to perform fine-grained transformation on the input initial feature map F to obtain the fine-grained feature map F′. The transformation process specifically includes: The initial feature map F is converted into a global average pooling feature map U through the global average pooling module; The global average pooling feature map U is input into the two-dimensional convolution layer, and the spatial dimension of the global average pooling feature map U is convolved to obtain the channel attention matrix U with single feature consistency. s ; Input the global average pooling feature map U into the one-dimensional convolution layer, perform convolution operation on the channel dimension of the global average pooling feature map U, and obtain the channel attention matrix U of multiple feature correlations j ; The channel attention matrix U of multiple feature correlations j The transposed matrix of Channel attention matrix U consistent with a single feature s Multiply and process with the softmax activation function to obtain the cross attention weight matrix F u ; The cross attention weight matrix F u Channel attention matrix U associated with multiple features j Multiply it with the channel attention matrix U of the single feature consistency s After adding and processing with ReLU activation function, the first granularity feature U is obtained. s '; The cross attention weight matrix F u The transposed matrix F u T Channel attention matrix U consistent with a single feature s Multiply it and then add it to the channel attention matrix U associated with multiple features j After adding and processing with ReLU activation function, the second granularity feature U is obtained. j '; The first granularity feature U s ′ and the second granularity feature U j ′ is weighted fused, processed by the sigmoid activation function, and then multiplied with the initial feature map F to obtain the fine-grained feature map F′.

8. The dual-stream fine-grained forgery detection method based on phase features according to claim 5 is characterized in that: Step S3 specifically includes: S31: Input the original face image into the first backbone network, and perform multiple fine-grained transformations and feature extractions on the original face image through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules in the first backbone network to obtain a first forgery detection feature map F r ; S32: Input the phase feature map into the second backbone network, and perform multiple fine-grained transformations and feature extractions on the phase feature map through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules in the second backbone network to obtain a second forgery detection feature map F pc ; S33: The first forgery detection feature map F r and the second forgery detection feature map F pc Input the correlation feature fusion module to obtain the fused forgery detection feature map F DSCF ; S34: Fusion forgery detection feature map F DSCF Input the classification head and calculate the discriminant vector P; if the discriminant vector P is greater than the preset value, the original face image is judged to be a real image; otherwise, the original face image is judged to be a forged image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the dual-stream fine-grained forgery detection method based on phase features according to any one of claims 1 to 8 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the dual-stream fine-grained forgery detection method based on phase features according to any one of claims 1 to 8 is implemented.

Images