Methods, devices, equipment, media, and products for detecting deepfake images
By using the spatial and positional attention modules in the multi-task biased detection network model, combined with a pre-trained visual encoder and face parsing model, and optimizing the loss function, the problem of insufficient generalization performance of deep forgery detection models in unknown domains is solved, and more accurate forgery image detection is achieved.
Patent Information
- Application Number
- CN202611234624.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-17
AI Technical Summary
Existing deepfake detection models, due to their inherent shortcut learning characteristics, tend to capture low-cost content clues such as facial identity and clothing color as classification criteria. This leads to performance collapse in generalization in unseen domains or scenarios where the label content is reversed, resulting in the inability to obtain correct forgery detection results.
A multi-task bias-reduction detection network model is adopted. By setting up spatial attention and positional attention modules, the model's attention is focused on fine-grained face regions, suppressing bias information in the training data. The spatial localization mask is determined by a pre-trained visual encoder and face parsing model and iteratively updated. The positional loss function and the true/false classification loss function are optimized to improve the model's generalization ability.
It improves the generalization performance of deepfake image detection, enabling accurate identification of fake images even in scenarios with biased training data content. It also suppresses the model's learning of irrelevant information and enhances its detection capabilities in unknown domains.
Smart Images

Figure CN123119554A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image detection technology, and in particular to a method, apparatus, device, medium and product for detecting deepfake images. Background Technology
[0002] The misuse of deepfake technology poses a significant threat to social trust and personal information security.
[0003] Existing forgery detection methods rely on pre-collected unbiased training data. When there is severe content bias in the training dataset, such as false strong coupling between face identity or background environment and true / false classification labels, existing deep forgery detection models, due to their inherent shortcut learning characteristics, are prone to capturing low-cost content clues such as face identity and clothing color as the basis for classification. However, this classification basis is not a feature that can truly identify forged images, causing the model's generalization performance to collapse in unseen unknown domains or scenarios where label content is reversed, resulting in the inability to obtain correct forgery detection results. Summary of the Invention
[0004] This invention provides a method, apparatus, device, medium, and product for detecting deepfake images. It addresses the shortcomings of existing deepfake detection models, which, due to their inherent shortcut learning characteristics, tend to capture low-cost content clues such as facial identity and clothing color as classification criteria. However, these classification criteria are not truly capable of identifying the features of fake images, leading to performance degradation in unseen domains or scenarios with inverted label content, resulting in incorrect fakefake detection results. The technical solution of this invention, through the use of multiple transformers and the spatial and positional attention modules within these transformers, focuses the attention of the multi-task bias-reducing detection network model on fine-grained facial regions, suppressing bias information in the training data and achieving more generalized fakefake detection results.
[0005] This invention provides a method for detecting deepfake images, comprising the following steps.
[0006] The image to be detected is input into a multi-task bias-reduction detection network model to obtain the true / false detection result output by the multi-task bias-reduction detection network model; the multi-task bias-reduction detection network model includes at least multiple transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; the multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images.
[0007] According to the method for detecting deepfake images provided by the present invention, the multi-task bias-reduction detection network model is trained based on the following method: A spatial positioning mask is determined based on the sample image and a preset face analysis model; the spatial positioning mask is used to constrain the location of forgery regions. The visual encoder is iteratively updated based on the sample image and the spatial localization mask to obtain the multi-task bias-reduction detection network model.
[0008] According to a method for detecting deepfake images provided by the present invention, the step of determining a spatial localization mask based on the sample image and a preset face parsing model includes: The sample image is input into the face analysis model to obtain the face region mask output by the face analysis model; The face region mask is subjected to pooling processing to obtain the spatial positioning mask.
[0009] According to a method for detecting deepfake images provided by the present invention, the step of iteratively updating the visual encoder based on the sample image and the spatial localization mask to obtain the multi-task bias-reducing detection network model includes: For each iteration in the iterative update, the sample image is input into the first transformer in the visual encoder to obtain the position self-attention feature map and the prediction scalar value output by the last transformer in the visual encoder; The position loss function is determined based on the position self-attention feature map and the spatial positioning mask; If the position loss function fails to converge, the spatial attention module and position attention module in the visual encoder, as well as the matrix parameters corresponding to the visual encoder, are updated. The true / false classification loss function is determined based on the predicted scalar value and the true / false labels; If the true / false classification loss function fails to converge, the matrix parameters and binary classification branch corresponding to the visual encoder are updated. The corresponding visual encoder is determined as the multi-task bias-reduction detection network model when both the position loss function and the true / false classification loss function converge.
[0010] According to a method for detecting deepfake images provided by the present invention, the step of determining the position loss function based on the position self-attention feature map and the spatial localization mask includes: in, This represents the location loss function. Indicates the first The location self-attention feature map of each sample, Represents the set of real numbers. and This represents the pixel length and pixel width of the self-attention feature map at the location. This represents the activation function. Represents pixel coordinates in two-dimensional space. Indicates the first The spatial positioning mask for each sample.
[0011] According to a method for detecting deepfake images provided by the present invention, the step of determining the true / false classification loss function based on the predicted scalar value and the true / false labels includes: in, This represents the true / false classification loss function. Indicates the first The authenticity labels of each sample, Indicates the first The predicted scalar value for each sample.
[0012] According to a method for detecting deepfake images provided by the present invention, the matrix parameters corresponding to the visual encoder are updated. The updating of the matrix parameters corresponding to the visual encoder includes: Discrete singular value decomposition is performed on the matrix parameters based on the left singular orthogonal basis, the right singular orthogonal basis, and the singular value diagonal matrix to obtain the updated matrix parameters.
[0013] The present invention also provides a device for detecting deepfake images, comprising the following modules: The detection module is used to input the image to be detected into a multi-task bias-reduction detection network model and obtain the true / false detection result output by the multi-task bias-reduction detection network model; the multi-task bias-reduction detection network model includes at least a plurality of transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; the multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images.
[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for detecting deepfake images as described above.
[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting deepfake images as described above.
[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for detecting deepfake images as described above.
[0017] The present invention provides a method, apparatus, device, medium, and product for detecting deepfake images. The method inputs the image to be detected into a multi-task bias-reduction detection network model, and obtains the authenticity detection result output by the multi-task bias-reduction detection network model. The multi-task bias-reduction detection network model includes at least multiple transformers. Each transformer includes a spatial attention module and a positional attention module. The multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images. The technical solution of this invention, through the multiple transformers and the spatial and positional attention modules within the transformers, enables the attention of the multi-task bias-reduction detection network model to focus on fine-grained facial regions, suppressing bias information in the training data, and thus obtaining more generalized forgery detection results. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating the method for detecting deepfake images provided by the present invention.
[0020] Figure 2 This is a schematic diagram of the process for training a multi-task bias-removing detection network model provided by the present invention.
[0021] Figure 3 This is a schematic diagram of the structure of the deepfake image detection device provided by the present invention.
[0022] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0024] To address the aforementioned problems in the prior art, this invention provides a method for detecting deepfake images. Figure 1 This is a flowchart illustrating the deepfake image detection method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following step 110.
[0025] Step 110: Input the image to be detected into the multi-task bias-reduction detection network model to obtain the true / false detection result output by the multi-task bias-reduction detection network model; the multi-task bias-reduction detection network model includes at least multiple transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; the multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images.
[0026] Specifically, the image to be detected can be input into a multi-task bias-de-bias detection network model to obtain the true / false detection result output by the model. Essentially, the image to be detected is input into the first transformer in the multi-task bias-de-bias detection network model. For each transformer in the multi-task bias-de-bias detection network model except the first one, the output of the previous transformer is used as the input of the current transformer (it's easy to understand that the first transformer does not have a corresponding previous transformer; its input is the image to be detected). Inside the transformer, the inputs are fed into the spatial attention module and the positional attention module, respectively, and then the outputs of the spatial attention module and the positional attention module are fused to obtain the output of the transformer. The multi-task bias-removal detection network model is trained on a pre-trained visual encoder based on sample images. These sample images correspond to real and fake labels and can include various types of real and fake human images. The pre-trained visual encoder can be, for example, a Contrastive Language-Image Pre-training with Vision Transformer - Large with 14×14 pixel patches (CLIP-ViT-L / 14) visual encoder. The CLIP-ViT-L / 14 visual encoder consists of 24 Transformer blocks. After pre-training the CLIP-ViT-L / 14 visual encoder, the dense parameters of its shallow and deep feature extraction layers are frozen to prevent degradation of representations during forward propagation. Furthermore, the pre-trained visual encoder can utilize its prior knowledge to suppress the model from learning irrelevant information during training. While existing techniques also attempt to improve generalization using frequency domain analysis, image decoupling techniques, or traditional model augmentation techniques... However, decoupling methods often suffer from insufficient reconstruction decoder capabilities, leading to significant leakage of content information into the forgery branch; frequency domain methods, on the other hand, tend to create new structural shortcuts. (Sample image) Belongs to the sample image dataset ,in, express The corresponding authenticity label, Indicates the first One sample.
[0027] The deepfake image detection method provided by this invention inputs the image to be detected into a multi-task bias-reduction detection network model, and obtains the authenticity detection result output by the multi-task bias-reduction detection network model. The multi-task bias-reduction detection network model includes at least multiple transformers. Each transformer includes a spatial attention module and a positional attention module. The multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images. This invention's technical solution, through the multiple transformers and the spatial and positional attention modules within them, enables the multi-task bias-reduction detection network model to focus its attention on fine-grained facial regions, improving detection performance in scenarios with biased training data content and obtaining accurate fake detection results.
[0028] In one embodiment, the multi-task bias-reduction detection network model is trained in the following manner: A spatial positioning mask is determined based on the sample image and a preset face analysis model; the spatial positioning mask is used to constrain the location of forgery regions. The visual encoder is iteratively updated based on the sample image and the spatial localization mask to obtain the multi-task bias-reduction detection network model.
[0029] Specifically, Figure 2 This is a flowchart illustrating the training process of the multi-task bias-removing detection network model provided by the present invention, as shown below. Figure 2 As shown, a spatial positioning mask can be determined based on sample images and a preset face analysis model; the spatial positioning mask is used to constrain the location of forged regions.
[0030] Furthermore, the visual encoder can be iteratively updated based on sample images and spatial localization masks to obtain a multi-task bias-reduction detection network model.
[0031] In the above embodiments, the image-level labels are refined to the location-level labels by determining the spatial positioning mask to locate the forgery area. This enables the multi-task bias-removal detection network model to learn more fine-grained deep forgery features and suppresses the model from learning incorrect association features.
[0032] In one embodiment, determining the spatial localization mask based on the sample image and a preset face analysis model includes: The sample image is input into the face analysis model to obtain the face region mask output by the face analysis model; The face region mask is subjected to pooling processing to obtain the spatial positioning mask.
[0033] Specifically, such as Figure 2As shown, the sample image is input into the face parsing model to obtain a binary mask of the face region (i.e., a face region mask), which is used to indicate the coarse-grained forgery region. This face parsing model is, for example, the M2FP (Mask2Former for Parsing Humans) model.
[0034] Furthermore, the face region mask can be pooled to obtain a soft mask as a soft label for coarse-grained fake region localization.
[0035] In one embodiment, the iterative update of the visual encoder based on the sample image and the spatial localization mask to obtain the multi-task bias-reducing detection network model includes: For each iteration in the iterative update, the sample image is input into the first transformer in the visual encoder to obtain the position self-attention feature map and the prediction scalar value output by the last transformer in the visual encoder; The position loss function is determined based on the position self-attention feature map and the spatial positioning mask; If the position loss function fails to converge, the spatial attention module and position attention module in the visual encoder, as well as the matrix parameters corresponding to the visual encoder, are updated. The true / false classification loss function is determined based on the predicted scalar value and the true / false labels; If the true / false classification loss function fails to converge, the matrix parameters and binary classification branch corresponding to the visual encoder are updated. The corresponding visual encoder is determined as the multi-task bias-reduction detection network model when both the position loss function and the true / false classification loss function converge.
[0036] Specifically, such as Figure 2 As shown, for each iteration in the iterative update, inputting the sample image into the first transformer in the visual encoder yields the positional self-attention feature map output by the last transformer in that visual encoder. and predicted scalar values It's easy to understand that this is for ease of demonstration. Figure 2 Only one specific structure of Transformer is shown in the image; the remaining blank boxes all represent Transformers.
[0037] Furthermore, the positional loss function can be determined based on the positional self-attention feature map and the spatial localization mask. If the positional loss function fails to converge, the spatial attention module and the positional attention module in the visual encoder, as well as the corresponding matrix parameters of the visual encoder, are updated. This parameter update process includes parameter backpropagation and updating, and the remaining parts of the visual encoder are frozen during the parameter update.
[0038] Furthermore, the true / false classification loss can be calculated based on the predicted scalar value and the true / false labels. If the classification loss function does not converge, the matrix parameters and binary classification branch corresponding to the visual encoder are updated. This parameter update process includes parameter backpropagation and updating, and the rest of the visual encoder is frozen during the parameter update.
[0039] The parameter update for the spatial attention module can be expressed as: The parameter update for the position attention module can be expressed as: ,in, This represents the attention mechanism. Indicates a query. Indicates the Key key. Indicates the value. Indicates the scaling factor. This represents the dimension of a vector. Represents the normalized exponential function, Indicates transpose. , This indicates that click attention is calculated using a scaled approach. Indicates the preset weight. Indicates the first Spatial localization mask corresponding to each sample image.
[0040] Once both the location loss function and the true / false classification loss function converge, the corresponding visual encoder is determined as a multi-task bias-reduction detection network model.
[0041] It's easy to understand that if the loss function converges at a certain location, but the true / false classification loss function does not converge, then the matrix parameters and binary classification branch corresponding to the visual encoder continue to be updated. Similarly, if the true / false classification loss function converges, but the location loss function does not converge, then the spatial attention module and location attention module in the visual encoder, as well as the matrix parameters corresponding to the visual encoder, continue to be updated until both the location loss function and the true / false classification loss function converge. Here, "convergence" means that the loss function reaches a local minimum and tends to plateau.
[0042] In the above embodiments, through a two-stage alternating collaborative optimization mechanism, the classification gradient path is explicitly disconnected in the first stage (i.e., the parameter update stage based on the position loss function) to reshape the spatial self-attention weight distribution of the model, so that its physical gaze domain converges to the core facial tissue. In the second stage (i.e., the parameter update stage based on the true / false classification loss function) to perform true / false hard classification, the position gradient path is disconnected, prompting the binary classification branch to search for generalized forgery traces within the narrowed facial space. This alternating gradient disconnection control method reduces the interference of multi-task gradients and helps to decouple the content space from the classification label.
[0043] In one embodiment, determining the location loss function based on the location self-attention feature map and the spatial localization mask includes: in, This represents the location loss function. Indicates the first The location self-attention feature map of each sample, Represents the set of real numbers. and This represents the pixel length and pixel width of the self-attention feature map at the location. This represents the activation function. Represents pixel coordinates in two-dimensional space. Indicates the first The spatial positioning mask for each sample.
[0044] Specifically, Indicates the first One sample (image) This represents the Sigmoid activation function and the position loss function. The mean squared error regression operator was used for calculation. In the above embodiments, the spatial self-attention weight distribution of the visual encoder is reshaped by the position loss function, so that its physical gaze domain converges to the core facial tissue.
[0045] In one embodiment, determining the true / false classification loss function based on the predicted scalar value and the true / false labels includes: in, This represents the true / false classification loss function. Indicates the first The authenticity labels of each sample, Indicates the first The predicted scalar value for each sample.
[0046] Specifically, the true / false classification loss function The cross-entropy loss function was used for calculation.
[0047] In the above embodiment, since spatial localization constraints have been implicitly applied in the first stage, the model is suppressed from using irrelevant content features such as background and identity as shortcuts for learning in the second stage. This forces the model to capture fake features within the limited facial space, thereby achieving explicit decoupling of content and labels in the training loop.
[0048] In one embodiment, updating the matrix parameters corresponding to the visual encoder includes: Discrete singular value decomposition is performed on the matrix parameters based on the left singular orthogonal basis, the right singular orthogonal basis, and the singular value diagonal matrix to obtain the updated matrix parameters.
[0049] Specifically, discrete singular value decomposition (SVD) can be performed on the matrix parameters based on the left singular orthogonal basis, the right singular orthogonal basis, and the singular value diagonal matrix to obtain the updated matrix parameters. The left and right singular orthogonal bases are pre-set, and this embodiment does not impose specific limitations on them.
[0050] Specifically, the steps for performing singular value decomposition and low-rank fine-tuning on the weight matrix in the attention mechanism are as follows: Attention mechanisms include query weight matrices Key weight matrix and value weight matrix ;in, The dimension of the hidden layer representing the attention mechanism.
[0051] With any target weight matrix For example, the formula for performing singular value decomposition (SVD) on it is expressed as follows: . Describe a left orthogonal matrix (left singular orthogonal basis) that satisfies ,in express An identity matrix of order 1.
[0052] This represents a right orthogonal matrix (right singular orthogonal basis). express The transpose of the matrix satisfies . Denotes a singular valued diagonal matrix, which is expressed by the diagonal operator as follows: .
[0053] Represents a diagonal operator used to construct parameters. A matrix whose main diagonal elements are all zero, and whose other off-diagonal elements are all zero. (in ) represents the target weight matrix The i-th singular value, satisfying the descending order constraint. .
[0054] During the fine-tuning training phase, based on the preset truncation low-rank order r (where r is the lowest rank order)... Extract the first r largest singular values and their corresponding eigenvectors to construct a low-rank approximation matrix, which is expressed by the following formula: In the formula This indicates extracting a left orthogonal matrix. The left submatrix is composed of the first r columns.
[0055] This indicates extracting a right orthogonal matrix. The right submatrix consisting of the first r columns, express The transpose of . Let r be the diagonal submatrix formed by the first r largest singular values, satisfying .
[0056] By fixing the target weight matrix Injecting a low-order trainable adaptive adjustment matrix ΔW makes the updated actual working weight matrix... Satisfy the following calculation formula: This allows for fine-tuning of the attention matrix weights.
[0057] In the above embodiments, this low-rank approximate fine-tuning mechanism based on intrinsic feature space projection not only compresses the parameter calculation dimension during the training phase, ensuring the energy efficiency of model optimization, but also strictly constrains the incremental adjustment of gradients within the intrinsic main semantic manifold space of the original pre-trained model. This fundamentally suppresses excessive perturbation and catastrophic forgetting caused by full-parameter fine-tuning to the original robust semantic prior. By preserving the generalized cognition of global macro-semantics such as identity and environmental background based on singular value decomposition, and extracting forgery features through the trainable parameter matrix, the perturbation or forgetting caused by full-parameter fine-tuning or random low-rank fine-tuning to the original robust semantic prior is reduced.
[0058] The detection apparatus for deepfake images provided by the present invention will be described below. The detection apparatus for deepfake images described below can be referred to in correspondence with the detection method for deepfake images described above.
[0059] Figure 3This is a schematic diagram of the structure of the deepfake image detection device provided by the present invention, as shown below. Figure 3 As shown, the detection device 300 for deepfake images includes the following modules: The detection module 310 is used to input the image to be detected into a multi-task bias-reduction detection network model to obtain the true / false detection result output by the multi-task bias-reduction detection network model; the multi-task bias-reduction detection network model includes at least a plurality of transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; the multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images.
[0060] In one embodiment, the deepfake image detection device 300 further includes a training module for training the multi-task bias-reduction detection network model based on the following method: A spatial positioning mask is determined based on the sample image and a preset face analysis model; the spatial positioning mask is used to constrain the location of forgery regions. The visual encoder is iteratively updated based on the sample image and the spatial localization mask to obtain the multi-task bias-reduction detection network model.
[0061] In one embodiment, the training module is specifically used for: The sample image is input into the face analysis model to obtain the face region mask output by the face analysis model; The face region mask is subjected to pooling processing to obtain the spatial positioning mask.
[0062] In one embodiment, the training module is further used for: For each iteration in the iterative update, the sample image is input into the first transformer in the visual encoder to obtain the position self-attention feature map and the prediction scalar value output by the last transformer in the visual encoder; The position loss function is determined based on the position self-attention feature map and the spatial positioning mask; If the position loss function fails to converge, the spatial attention module and position attention module in the visual encoder, as well as the matrix parameters corresponding to the visual encoder, are updated. The true / false classification loss function is determined based on the predicted scalar value and the true / false labels; If the true / false classification loss function fails to converge, the matrix parameters and binary classification branch corresponding to the visual encoder are updated. The corresponding visual encoder is determined as the multi-task bias-reduction detection network model when both the position loss function and the true / false classification loss function converge.
[0063] In one embodiment, the training module is further used for: in, This represents the location loss function. Indicates the first The location self-attention feature map of each sample, Represents the set of real numbers. and This represents the pixel length and pixel width of the self-attention feature map at the location. This represents the activation function. Represents pixel coordinates in two-dimensional space. Indicates the first The spatial positioning mask for each sample.
[0064] In one embodiment, the training module is further used for: in, This represents the true / false classification loss function. Indicates the first The true label of each sample, Indicates the first The predicted scalar value for each sample.
[0065] In one embodiment, the training module is further used for: Discrete singular value decomposition is performed on the matrix parameters based on the left singular orthogonal basis, the right singular orthogonal basis, and the singular value diagonal matrix to obtain the updated matrix parameters.
[0066] The deepfake image detection device provided by this invention inputs the image to be detected into a multi-task bias-reduction detection network model, and obtains the authenticity detection result output by the multi-task bias-reduction detection network model. The multi-task bias-reduction detection network model includes at least multiple transformers. Each transformer includes a spatial attention module and a positional attention module. The multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images. The technical solution of this invention, through the multiple transformers and the spatial and positional attention modules within the transformers, enables the attention of the multi-task bias-reduction detection network model to focus on fine-grained facial regions, suppressing bias information in the training data, and thus obtaining more generalized fake detection results.
[0067] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for detecting deepfake images, the method including: The image to be detected is input into a multi-task bias-reduction detection network model to obtain the true / false detection result output by the multi-task bias-reduction detection network model; the multi-task bias-reduction detection network model includes at least multiple transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; the multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images.
[0068] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0069] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of performing the depth-spoofing image detection method provided by the above methods, the method comprising: The image to be detected is input into a multi-task bias-reduction detection network model to obtain the true / false detection result output by the multi-task bias-reduction detection network model; the multi-task bias-reduction detection network model includes at least multiple transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; the multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images.
[0070] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for detecting deepfake images provided by the methods described above, the method comprising: The image to be detected is input into a multi-task bias-reduction detection network model to obtain the true / false detection result output by the multi-task bias-reduction detection network model; the multi-task bias-reduction detection network model includes at least multiple transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; the multi-task bias-reduction detection network model is trained on a pre-trained visual encoder based on sample images.
[0071] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0072] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting deepfake images, characterized in that, include: The image to be detected is input into the multi-task bias-removing detection network model to obtain the true and false detection results output by the multi-task bias-removing detection network model. The multi-task bias removal detection network model includes at least a plurality of transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; The multi-task bias removal detection network model is trained on a pre-trained visual encoder based on sample images.
2. The method for detecting deepfake images according to claim 1, characterized in that, The multi-task bias removal detection network model was trained in the following manner: A spatial positioning mask is determined based on the sample image and a preset face analysis model; the spatial positioning mask is used to constrain the location of forgery regions. The visual encoder is iteratively updated based on the sample image and the spatial localization mask to obtain the multi-task bias-reduction detection network model.
3. The method for detecting deepfake images according to claim 2, characterized in that, The step of determining the spatial localization mask based on the sample image and the preset face parsing model includes: The sample image is input into the face analysis model to obtain the face region mask output by the face analysis model; The face region mask is subjected to pooling processing to obtain the spatial positioning mask.
4. The method for detecting deepfake images according to claim 2, characterized in that, The step of iteratively updating the visual encoder based on the sample image and the spatial localization mask to obtain the multi-task bias-reducing detection network model includes: For each iteration in the iterative update, the sample image is input into the first transformer in the visual encoder to obtain the position self-attention feature map and the prediction scalar value output by the last transformer in the visual encoder; The position loss function is determined based on the position self-attention feature map and the spatial positioning mask; If the position loss function fails to converge, the spatial attention module and position attention module in the visual encoder, as well as the matrix parameters corresponding to the visual encoder, are updated. The true / false classification loss function is determined based on the predicted scalar value and the true / false labels; If the true / false classification loss function fails to converge, the matrix parameters and binary classification branch corresponding to the visual encoder are updated. The corresponding visual encoder is determined as the multi-task bias-reduction detection network model when both the position loss function and the true / false classification loss function converge.
5. The method for detecting deepfake images according to claim 4, characterized in that, Determining the position loss function based on the position self-attention feature map and the spatial localization mask includes: in, This represents the location loss function. Indicates the first The location self-attention feature map of each sample, Represents the set of real numbers. and This represents the pixel length and pixel width of the self-attention feature map at the location. This represents the activation function. Represents two-dimensional pixel coordinates. Indicates the first The spatial positioning mask for each sample.
6. The method for detecting deepfake images according to claim 4, characterized in that, The step of determining the true / false classification loss function based on the predicted scalar value and the true / false labels includes: in, This represents the true / false classification loss function. Indicates the first The true / false labels of each sample, Indicates the first The predicted scalar value for each sample.
7. The method for detecting deepfake images according to claim 4, characterized in that, Updating the matrix parameters corresponding to the visual encoder includes: Discrete singular value decomposition is performed on the matrix parameters based on the left singular orthogonal basis, the right singular orthogonal basis, and the singular value diagonal matrix to obtain the updated matrix parameters.
8. A device for detecting deepfake images, characterized in that, include: The detection module is used to input the image to be detected into the multi-task bias-removing detection network model and obtain the true and false detection results output by the multi-task bias-removing detection network model. The multi-task bias removal detection network model includes at least a plurality of transformers; for each transformer, the transformer includes a spatial attention module and a positional attention module; The multi-task bias removal detection network model is trained on a pre-trained visual encoder based on sample images.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for detecting deepfake images as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for detecting deepfake images as described in any one of claims 1 to 7.
11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for detecting deepfake images as described in any one of claims 1 to 7.
