A phase-feature-based fine-grained counterfeit detection method for dual-stream systems
By constructing a phase feature-based dual-stream forgery detection method, the problems of insufficient detection accuracy and generalization ability in existing technologies are solved, and high-precision and widely applicable detection of image forgery is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2025-04-25
- Publication Date
- 2026-07-17
Smart Images
Figure CN120452040B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image forgery detection, and more particularly to a two-stream fine-grained forgery detection method based on phase features. Background Technology
[0002] In recent years, with the development of the internet, people have become increasingly concerned about cybersecurity. As a rapidly developing technology, artificial intelligence (AI) has also attracted significant attention regarding its security. The application of AI in video, audio, and image forgery and editing has gradually developed, leading to the widespread public awareness of forgery techniques such as text-image generation, audio generation, face replacement, attribute editing, and facial reenactment. The development of deepfake technology has brought many opportunities, with applications in film and television special effects, medical aesthetics, virtual live streaming, public education, news dissemination, and artistic creation. However, the misuse of AI technology has also caused significant harm to people's personal information security, particularly the use of generative adversarial networks and diffusion models to generate fake facial images and videos in recent years, which has caused considerable public panic.
[0003] Given the series of harms caused by the misuse of deepfake technology, deepfake detection technology has gradually developed. Early methods directly input real and forged images into traditional machine learning algorithms or simple convolutional neural networks (CNNs) for classification, achieving decent results. This indirectly illustrates that forgery methods inevitably leave traces. However, with the advancement of forgery technology, traditional methods have become increasingly difficult to maintain high accuracy and have been gradually phased out. Therefore, many scholars have analyzed the vulnerabilities of deepfake technology from different perspectives regarding the potential presence of forgery traces, proposing numerous detection techniques based on different feature domains, such as:
[0004] 1. A dual-stream forgery detection network based on spatial and frequency domain features is constructed, and forgery traces are mined through a multi-scale self-attention mechanism. However, this method uses convolution to reduce the dimensionality of the original input in order to reduce the huge amount of computation brought about by multi-scale self-attention. This will result in insufficient fine-grained capture ability of the model for forgery features, thus affecting the detection accuracy.
[0005] 2. Based on a new perspective of frequency domain artifacts, this method uses a rich spatial domain model to capture high-frequency noise residuals, revealing more traces of forgery. However, due to the influence of frequency domain amplitude, this method cannot delve into deep phase forgery traces, resulting in insufficient model generalization.
[0006] In summary, existing image forgery detection methods lack modeling of fine-grained forgery features and mining of common forgery traces, resulting in poor detection accuracy and weak generalization ability of forgery detection models to unknown forgery techniques. Summary of the Invention
[0007] To address the above problems, this invention provides a dual-stream fine-grained forgery detection method based on phase features, comprising the following steps:
[0008] S1: Obtain the depth spoofing data set to be tested, extract images from the depth spoofing data set to be tested, and obtain the original face image;
[0009] S2: Construct a phase artifact mining module. Input the original face image into the phase artifact mining module to extract phase features and obtain a phase feature map.
[0010] S3: Construct a dual-stream forgery detection network. Input the original face image and phase feature map into the dual-stream forgery detection network to determine whether the original face image is a forged image.
[0011] Optionally, step S1 specifically includes:
[0012] S11: Extract the video data set from the depth fake data set to be tested;
[0013] S12: Extract the initial image of each video data in the video data set at a preset interval to obtain the initial image set;
[0014] S13: Locate, align, and crop the faces of each initial image in the initial image set to obtain the original face image set;
[0015] S14: Select an original face image of a preset size from the original face image set.
[0016] Optional:
[0017] The phase artifact mining module includes multiple filter banks in different directions, and each filter bank includes multiple filters of different sizes.
[0018] Optionally, step S2 specifically includes:
[0019] S21: Convert the original face image to a grayscale image;
[0020] S22: Extract pixel x from the grayscale image;
[0021] S23: Input the filter bank M with the pixel x-axis direction θ. θ ;
[0022] S24: Through filter bank M θ A filter m of size n n,θ Perform frequency domain filtering on pixel x to calculate the magnitude and phase offset of pixel x in the direction θ and size n;
[0023] S25: Repeat step S24 to calculate the magnitude and phase offset of all dimensions of pixel x in direction θ. Using the magnitude and phase offset of each dimension in direction θ, calculate the phase metric value of pixel x in direction θ.
[0024] S26: Repeat steps S23-S25 to calculate the phase measurement values of pixel x in all directions, normalize and merge the phase measurement values in each direction, and calculate the phase feature value of pixel x.
[0025] S27: Repeat steps S22-S26 to calculate the phase feature values of all pixels and construct a phase feature map using the phase feature values of each pixel.
[0026] Optional:
[0027] The dual-stream forgery detection network includes: a first backbone network, a second backbone network, a relevance feature fusion module, and a classification head;
[0028] The first and second backbone networks have the same structure;
[0029] The first backbone network includes multiple dynamic fine-grained perceptual attention mechanism modules and multiple feature extraction modules, with each dynamic fine-grained perceptual attention mechanism module positioned between two adjacent feature extraction modules.
[0030] Optional:
[0031] The dynamic fine-grained perceptual attention mechanism module includes: a global average pooling module, a two-dimensional convolutional layer with a kernel size of 1, and a one-dimensional convolutional layer with a kernel size of 3.
[0032] Optional:
[0033] The dynamic fine-grained perceptual attention mechanism module is used to perform fine-grained transformation on the initial input feature map F to obtain a fine-grained feature map F′. The transformation process specifically includes:
[0034] The initial feature map F is transformed into a global average pooling feature map U through the global average pooling module;
[0035] The global average pooling feature map U is input into a two-dimensional convolutional layer, and a convolution operation is performed on the spatial dimension of the global average pooling feature map U to obtain a single feature consistency channel attention matrix U. s ;
[0036] The global average pooling feature map U is input into a one-dimensional convolutional layer, and a convolution operation is performed on the channel dimension of the global average pooling feature map U to obtain the channel attention matrix U of multiple feature correlations. j ;
[0037] Channel attention matrix U that correlates multiple features j transpose matrix Channel attention matrix U consistent with a single feature s After multiplication and processing through the softmax activation function, the cross-attention weight matrix F is obtained. u ;
[0038] The cross-attention weight matrix F u Channel attention matrix U associated with multiple features j Multiply by the channel attention matrix U, which is consistent with the single feature. s The summation, followed by processing with the ReLU activation function, yields the first-granularity feature U. s ′;
[0039] The cross-attention weight matrix F u transpose matrix Channel attention matrix U consistent with a single feature s Multiply by the channel attention matrix U, which is associated with multiple features. j The summation, followed by processing with the ReLU activation function, yields the second-granularity feature U. j ′;
[0040] The first granularity feature U s ′ and second granularity feature U j The feature map is then weighted and fused, processed by the sigmoid activation function, and multiplied with the initial feature map F to obtain the fine-grained feature map F′.
[0041] Optionally, step S3 specifically includes:
[0042] S31: Input the original face image into the first backbone network. Through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules within the first backbone network, perform multiple fine-grained transformations and feature extractions on the original face image to obtain the first forgery detection feature map F. r ;
[0043] S32: Input the phase feature map into the second backbone network. Through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules within the second backbone network, perform multiple fine-grained transformations and feature extractions on the phase feature map to obtain the second forgery detection feature map F. pc ;
[0044] S33: Transfer the first forgery detection feature map F r Second counterfeit detection feature map F pc Input the correlation feature fusion module to obtain the fused forgery detection feature map F. DSCF ;
[0045] S34: Integrate forgery detection feature maps F DSCF Input the classification header and calculate the discrimination vector P; if the discrimination vector P is greater than the preset value, the original face image is judged to be a real image; otherwise, the original face image is judged to be a fake image.
[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned phase feature-based dual-stream fine-grained forgery detection method.
[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned phase feature-based dual-stream fine-grained forgery detection method.
[0048] The present invention has the following beneficial effects:
[0049] 1. By using filters of different directions and sizes in the phase artifact mining module, multi-directional and multi-size filtering is performed on each pixel in the face image to obtain multi-directional phase measurement values. Then, the phase measurement values of each direction are normalized and merged to focus on mining deep common edge forgery traces in the face image. This makes the obtained phase feature map have strong commonality and can show obvious forgery traces, significantly improving the accuracy and generalization of forgery detection.
[0050] 2. By using a dynamic fine-grained perceptual attention mechanism module to perform fine-grained transformation on the input feature map, the attention granularity of the network is reduced, which can effectively solve the problem of existing methods over-reliance on global features, thereby more accurately allocating fake feature weights and further improving the generalization of the model.
[0051] 3. By using the dual-backbone network structure of the dual-stream forgery detection network, feature extraction and fusion are performed on the original face image and phase feature map. While retaining the forgery traces of interest in each, the ability to capture common forgery features between different modalities is enhanced, further improving the accuracy of forgery detection. Attached Figure Description
[0052] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;
[0053] Figure 2 This is a schematic diagram of the phase feature map extraction process;
[0054] Figure 3 A structural diagram of a dual-stream forgery detection network;
[0055] Figure 4 This is a structural diagram of the dynamic fine-grained perceptual attention mechanism module;
[0056] Figure 5 This is a structural diagram of the correlation feature fusion module;
[0057] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;
[0058] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0059] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0060] Reference Figure 1 This invention provides a dual-stream fine-grained counterfeit detection method based on phase features, comprising the following steps:
[0061] S1: Obtain the depth spoofing data set to be tested, extract images from the depth spoofing data set to be tested, and obtain the original face image;
[0062] In some embodiments:
[0063] Step S1 specifically includes:
[0064] S11: Extract the video data set from the depth fake data set to be tested;
[0065] In some embodiments, the dataset to be tested for deepfake includes three types: FaceForensic++ (c23), Celeb-DFv2, and WildDeepfake. The FaceForensic++ dataset consists of 1000 real videos extracted from YouTube and 4000 fake videos generated using different forgery methods, including Deepfake, Face2Face, FaceSwap, and NerualTextures. The Celeb-DFv2 dataset contains 590 real videos and 5639 fake videos. The WildDeepfake dataset contains 3805 real face videos and 3509 fake face videos.
[0066] S12: Extract the initial image of each video data in the video data set at a preset interval to obtain the initial image set;
[0067] S13: Locate, align, and crop the faces of each initial image in the initial image set to obtain the original face image set;
[0068] In some embodiments, the face detector in dlib is used to locate the face, and a 256dpi×256dpi face image is cropped by aligning with and cropping the face region by 1.5 times;
[0069] S14: Select an original face image of a preset size from the original face image set.
[0070] S2: Construct a phase artifact mining module. Input the original face image into the phase artifact mining module to extract phase features and obtain a phase feature map.
[0071] In some embodiments:
[0072] The phase artifact mining module includes multiple filter banks in different directions, and each filter bank includes multiple filters of different sizes.
[0073] In some embodiments, the phase artifact mining module includes Log-Gabor filters of 5 directions, each with 6 different sizes.
[0074] In some embodiments, the extraction process of the phase feature map is as follows: Figure 2 As shown:
[0075] Step S2 specifically includes:
[0076] S21: Convert the original face image to a grayscale image;
[0077] S22: Extract pixel x from the grayscale image;
[0078] S23: Input the filter bank M with the pixel x-axis direction θ. θ ;
[0079] S24: Through filter bank M θ A filter m of size n n,θ Perform frequency domain filtering on pixel x to calculate the magnitude and phase offset of pixel x in the direction θ and size n;
[0080] In some embodiments, a two-dimensional Fast Fourier Transform (FFT) is used to transform the image in the frequency domain. Log-Gabor filters of different orientations and sizes are applied for frequency domain filtering to extract phase information within a specific frequency range, thereby eliminating the influence of background illumination and noise in the image. Finally, the image is returned to the spatial domain by an Inverse Fourier Transform (IFFT) to obtain the filtered image. The above process can be expressed as the formula:
[0081]
[0082] Where ⊙ represents element-wise multiplication, e n,θ (x) and o n,θ (x) represents the convolution results of the original image I(x) with odd-symmetric and even-symmetric Log-Gabor wavelets of size n and direction θ, respectively. e and o can be understood as the real and imaginary parts of the image. and These are odd-symmetric and even-symmetric Log-Gabor wavelet filters of size n and direction θ, respectively. For the filtered image I′(x)=[e n,θ (x),o n,θ Solve independently for the amplitude (x)] and phase offset Δφ n,θ (x);
[0083] S25: Repeat step S24 to calculate the magnitude and phase offset of all dimensions of pixel x in direction θ. Using the magnitude and phase offset of each dimension in direction θ, calculate the phase metric value of pixel x in direction θ.
[0084] In some embodiments, the filtered image I′(x)=[e n,θ (x),o n,θ [x], This paper first solves the filtered image independently for each direction, and then calculates the total amplitude of the image after filtering by n different scale filters using the formula:
[0085]
[0086] This step forms the basis for subsequent calculations. Then, the amplitude A is calculated using the formula. n,θ (x) and phase offset Δφ n,θ The product of (x) is used to combine magnitude and phase information:
[0087]
[0088] in, and Let represent the weighted average phase angles of the real part (e) and imaginary part (o) of the image, respectively, and calculate them using the following formula:
[0089]
[0090] After obtaining the product of the image's amplitude and phase shift, as well as the image's amplitude, the phase metric PC of the image after filtering by Log-Gabor filters of all sizes with direction θ is calculated using the following formula. θ (x):
[0091]
[0092] Here, ε is a small constant to avoid division by zero; W θ (x) is a weighting function that adjusts the contribution of filters of different sizes to the result; T θ This is a noise compensation function to reduce the impact of noise. Indicates rounding down;
[0093] S26: Repeat steps S23-S25 to calculate the phase measurement values of pixel x in all directions, normalize and merge the phase measurement values in each direction, and calculate the phase feature value of pixel x.
[0094] In some embodiments, in order to solve for the phase eigenvalue I at x pc (x) also requires combining the filter responses at that point in all directions. To preserve phase spoofing traces in all directions to the greatest extent possible, a summation-normalization method is used to merge the results from all directions, as shown in the formula:
[0095]
[0096] S27: Repeat steps S22-S26 to calculate the phase feature values of all pixels and construct a phase feature map using the phase feature values of each pixel.
[0097] S3: Construct a dual-stream forgery detection network. Input the original face image and phase feature map into the dual-stream forgery detection network to determine whether the original face image is a forged image.
[0098] In some embodiments, the structure of the dual-stream forgery detection network is as follows: Figure 3 As shown:
[0099] The dual-stream forgery detection network includes: a first backbone network, a second backbone network, a relevance feature fusion module, and a classification head;
[0100] The first and second backbone networks have the same structure;
[0101] The first backbone network includes multiple dynamic fine-grained perceptual attention mechanism modules and multiple feature extraction modules, with each dynamic fine-grained perceptual attention mechanism module positioned between two adjacent feature extraction modules.
[0102] In some embodiments, the dual-stream forgery detection network may optionally use Xception as its backbone network.
[0103] In some embodiments, the structure of the dynamic fine-grained perceptual attention mechanism module is as follows: Figure 4 As shown;
[0104] The dynamic fine-grained perceptual attention mechanism module includes: a global average pooling module, a two-dimensional convolutional layer with a kernel size of 1, and a one-dimensional convolutional layer with a kernel size of 3.
[0105] In some embodiments:
[0106] The dynamic fine-grained perceptual attention mechanism module is used to perform fine-grained transformation on the initial input feature map F to obtain a fine-grained feature map F′. The transformation process specifically includes:
[0107] The initial feature map F is transformed into a global average pooling feature map U through the global average pooling module;
[0108] In some embodiments, given an initial feature map Where C represents the number of channels, and H and W represent the height and width of the feature map, respectively. The dynamic fine-grained perceptual attention mechanism module first transforms the feature map into a globally average pooled feature map using a global average pooling method. Representing the overall response along the channel dimension, used to further capture fine-grained inter-channel interactions, the nth channel element of U is represented as:
[0109]
[0110] Among them, F n (i,j) are the eigenvalues of the nth channel;
[0111] The global average pooling feature map U is input into a two-dimensional convolutional layer, and a convolution operation is performed on the spatial dimension of the global average pooling feature map U to obtain a single feature consistency channel attention matrix U. s ;
[0112] The global average pooling feature map U is input into a one-dimensional convolutional layer, and a convolution operation is performed on the channel dimension of the global average pooling feature map U to obtain the channel attention matrix U of multiple feature correlations. j ;
[0113] Channel attention matrix U that correlates multiple features j transpose matrix Channel attention matrix U consistent with a single feature s After multiplication and processing through the softmax activation function, the cross-attention weight matrix F is obtained. u ;
[0114] In some embodiments, the cross-attention weight matrix F u The calculation process is as follows:
[0115]
[0116] The cross-attention weight matrix F u Channel attention matrix U associated with multiple features j Multiply by the channel attention matrix U, which is consistent with the single feature. s The summation, followed by processing with the ReLU activation function, yields the first-granularity feature U. s ′;
[0117] In some embodiments, the first granularity feature U s The calculation process for ′ is as follows:
[0118] U s =ReLU(F) u ·U j +U s )
[0119] The cross-attention weight matrix F u transpose matrix Channel attention matrix U consistent with a single feature s Multiply by the channel attention matrix U, which is associated with multiple features. j The summation, followed by processing with the ReLU activation function, yields the second-granularity feature U. j ′;
[0120] In some embodiments, the second granularity feature U j The calculation process for ′ is as follows:
[0121] U j =ReLU(F) u T ·U s +U j )
[0122] The first granularity feature U s ′ and second granularity feature U j The feature map is then weighted and fused, processed by the sigmoid activation function, and multiplied with the initial feature map F to obtain the fine-grained feature map F′.
[0123] In some embodiments, the calculation process of the fine-grained feature map F′ is as follows:
[0124]
[0125] in, The weights are learnable fusion weights, and σ() is the sigmoid activation function.
[0126] In some embodiments:
[0127] Step S3 specifically includes:
[0128] S31: Input the original face image into the first backbone network. Through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules within the first backbone network, perform multiple fine-grained transformations and feature extractions on the original face image to obtain the first forgery detection feature map F. r ;
[0129] S32: Input the phase feature map into the second backbone network. Through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules within the second backbone network, perform multiple fine-grained transformations and feature extractions on the phase feature map to obtain the second forgery detection feature map F. pc ;
[0130] S33: Transfer the first forgery detection feature map F r Second counterfeit detection feature map F pc Input the correlation feature fusion module to obtain the fused forgery detection feature map F. DSCF ;
[0131] In some embodiments, the structure of the correlation feature fusion module is as follows: Figure 5 As shown, the correlation feature fusion module fuses the features of two streams, and its input is the first forgery detection feature map F. r Second counterfeit detection feature map F pc The channel-level correlation matrix is obtained by calculating the cosine similarity of the two feature maps along the channel dimension.
[0132]
[0133] Where x∈{1,2,...,H}, y∈{1,2,...,W} and C x,y Let (x, y) be the channel-level correlation value at coordinates (x, y). Then, the correlation-enhanced feature maps of different modalities are obtained through correlation weighting and residual connections, and a learnable parameter α is introduced to dynamically adjust the fusion ratio of the two modalities.
[0134] F DSCF =σ(α)×ReLU(F) pc +C⊙F r )+(1-σ(α))×ReLU(F r +C⊙F pc )
[0135] Where σ is the sigmoid activation function, ⊙ represents element-wise multiplication, and ReLU is a non-linear activation function. The above process yields the fused forgery detection feature map F. DSCF .
[0136] S34: Integrate forgery detection feature maps F DSCFInput the classification header and calculate the discrimination vector P; if the discrimination vector P is greater than the preset value, the original face image is judged to be a real image; otherwise, the original face image is judged to be a fake image.
[0137] In some embodiments, the obtained fused forgery detection feature map is used as a forgery feature vector to determine whether an image has been forged, and the classification head is supervised learning using an arbitrary binary classification loss; the obtained fused forgery detection feature map is passed through a classification head composed of a pooling layer, a fully connected layer and a normalization layer to obtain the final discrimination vector P, where P > 0.5 is considered a real image, and P ≤ 0.5 is considered a forged image.
[0138] In some embodiments, the binary classification loss function is a combination of FocalLoss and BCELoss, as shown in the following equation:
[0139] L Focal (p,y)=-αy(1-p) γ log(σ(p))-(1-α)(1-y)p γ log(1-σ(p))
[0140] Here, α is a hyperparameter used to balance the classes, γ is the FocalLoss focusing factor, p is the model's predicted value, σ(p) represents the output after sigmoid activation, and y is the true value. Specifically, γ = 2, α = 0.6.
[0141] In some embodiments, training and testing are performed on three datasets: FaceForensic++, Celeb-DFv2, and WildDeepfake. The test results of the forgery detection capability are shown in Table 1. The selected evaluation metrics ACC represents accuracy and AUC represents the area under the curve. These two evaluation metrics can fully evaluate the forgery detection capability of the network. As can be seen from Table 1, compared with other advanced forgery detection networks, the method proposed in this embodiment can improve the ability to detect deep forged faces using different forgery methods.
[0142] Table 1. Comparison of test results for counterfeit detection capabilities
[0143]
[0144] In the cross-dataset generalization test, the training was performed on the FaceForensics++ dataset, and then tested on the Celeb-DFv2 and WildDeepfake datasets respectively. The evaluation metric was AUC(%). The test results of average generalization are shown in Table 2. State-of-the-art generalization was achieved on both the Celeb-DFv2 and WildDeepfake datasets.
[0145] Table 2. Test comparison results of average generalization.
[0146]
[0147] In some embodiments, please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 60 provided in this application includes a memory 61 and a processor 62; the memory 61 stores a computer program, wherein the computer program, when executed by the processor, implements the aforementioned phase-feature-based dual-stream fine-grained forgery detection method.
[0148] Specifically, processor 62 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 62 may also include onboard memory for caching purposes. Processor 62 may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.
[0149] Memory 61 can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory 61 can be, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory 61 include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.
[0150] This application also provides a computer-readable medium storing a computer program that, when executed by a processor, implements the phase-feature-based dual-stream fine-grained forgery detection method described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.
[0151] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.
[0152] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A dual-stream fine-grained counterfeit detection method based on phase features, characterized in that, Including the following steps: S1: Obtain the depth spoofing data set to be tested, extract images from the depth spoofing data set to be tested, and obtain the original face image; S2: Construct a phase artifact mining module. Input the original face image into the phase artifact mining module to extract phase features and obtain a phase feature map. S3: Construct a dual-stream forgery detection network. Input the original face image and phase feature map into the dual-stream forgery detection network to determine whether the original face image is a forged image. The phase artifact mining module includes multiple filter banks in different directions, and each filter bank includes multiple filters of different sizes; Step S2 specifically includes: S21: Convert the original face image to a grayscale image; S22: Extract pixels from the grayscale image ; S23: Pixels Input direction is filter bank ; S24: Through the filter bank medium-sized filter For pixels Perform frequency domain filtering to calculate pixel points In direction The magnitude and phase offset at dimension n; S25: Repeat step S24 to calculate and obtain the pixel points. In direction The magnitude and phase offset of all dimensions, via direction The magnitude and phase offset of each dimension are used to calculate the pixel value. In direction Phase measurement value on; S26: Repeat steps S23-S25 to calculate and obtain the pixel points. The phase measurement values in all directions are normalized and merged to calculate the pixel value. The phase eigenvalues; S27: Repeat steps S22-S26 to calculate the phase feature values of all pixels and construct a phase feature map using the phase feature values of each pixel. The dual-stream forgery detection network includes: a first backbone network, a second backbone network, a relevance feature fusion module, and a classification head; The first and second backbone networks have the same structure; The first backbone network includes multiple dynamic fine-grained perceptual attention mechanism modules and multiple feature extraction modules, with each dynamic fine-grained perceptual attention mechanism module positioned between two adjacent feature extraction modules.
2. The dual-stream fine-grained counterfeit detection method based on phase features according to claim 1, characterized in that, Step S1 specifically includes: S11: Extract the video data set from the depth fake data set to be tested; S12: Extract the initial image of each video data in the video data set at a preset interval to obtain the initial image set; S13: Locate, align, and crop the faces of each initial image in the initial image set to obtain the original face image set; S14: Select an original face image of a preset size from the original face image set.
3. The dual-stream fine-grained counterfeit detection method based on phase features according to claim 1, characterized in that: The dynamic fine-grained perceptual attention mechanism module includes: a global average pooling module, a two-dimensional convolutional layer with a kernel size of 1, and a one-dimensional convolutional layer with a kernel size of 3.
4. The dual-stream fine-grained counterfeit detection method based on phase features according to claim 3, characterized in that: The dynamic fine-grained perceptual attention mechanism module is used for the initial feature map of the input. Perform fine-grained transformation to obtain fine-grained feature maps. The transformation process specifically includes: The initial feature map is processed by the global average pooling module. Transform into a global average pooling feature map ; Global average pooling feature map Input a 2D convolutional layer and process the global average pooling feature map. Convolution operations are performed on the spatial dimension to obtain a channel attention matrix that is consistent with a single feature. ; Global average pooling feature map Input a one-dimensional convolutional layer and process the global average pooling feature map. Convolution operations are performed along the channel dimension to obtain a channel attention matrix with multiple feature correlations. ; Channel attention matrix that correlates multiple features transpose matrix Channel attention matrix consistent with single feature After multiplication and processing through the softmax activation function, the cross-attention weight matrix is obtained. ; Cross-attention weight matrix Channel attention matrix associated with multiple features Multiply by the channel attention matrix that is consistent with a single feature. The features are added together and then processed by the ReLU activation function to obtain the first-level features. ; Cross-attention weight matrix transpose matrix Channel attention matrix consistent with single feature Multiply by the channel attention matrix associated with multiple features. The features are added together and then processed by the ReLU activation function to obtain the second granularity features. ; First granularity feature Second granularity features Weighted fusion is performed, followed by processing with the sigmoid activation function, and then compared with the initial feature map. After multiplication, a fine-grained feature map is obtained. .
5. The dual-stream fine-grained counterfeit detection method based on phase features according to claim 1, characterized in that, Step S3 specifically includes: S31: Input the original face image into the first backbone network. Through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules within the first backbone network, perform multiple fine-grained transformations and feature extractions on the original face image to obtain the first forgery detection feature map. ; S32: Input the phase feature map into the second backbone network. Through multiple dynamic fine-grained perception attention mechanism modules and multiple feature extraction modules within the second backbone network, perform multiple fine-grained transformations and feature extractions on the phase feature map to obtain the second forgery detection feature map. ; S33: First forgery detection feature map Second counterfeit detection feature map Input the correlation feature fusion module to obtain the fused forgery detection feature map. ; S34: Integrate forgery detection feature maps Input the classification header and calculate the discriminant vector. If the discriminant vector If the value is greater than the preset value, the original face image is determined to be a real image; otherwise, the original face image is determined to be a fake image.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the phase feature-based dual-stream fine-grained forgery detection method as described in any one of claims 1 to 5.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the phase feature-based dual-stream fine-grained forgery detection method as described in any one of claims 1 to 5.