Deep pseudo detection method and device based on feature decoupling and double-branch enhancement

Abstract

The embodiment of the invention provides a deep false detection method and device based on feature decoupling and double-branch enhancement, and the method comprises the steps: extracting a real identity feature of a to-be-detected face image through a pre-training face recognition model of freezing parameters, inputting a to-be-detected face image into a trainable deep learning network, extracting an identity correlation component and a counterfeit correlation component at the same time, constraining the identity correlation component to approach a real identity feature, and constraining the counterfeit correlation component to be orthogonal or low-correlated with the identity correlation component in a feature space at the same time; and inputting the constrained counterfeited related component into a double-branch counterfeited enhancement module for feature enhancement, and performing counterfeited detection through the enhanced counterfeited enhancement feature to obtain a detection result, thereby improving the precision of deep-false detection.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a deep pseudo-detection method and apparatus based on feature decoupling and dual-branch enhancement. Background Technology

[0002] In the fields of computer vision and artificial intelligence security, deepfake technology has become a significant challenge. Through advanced deep learning algorithms, deepfakes can generate highly realistic and indistinguishable facial videos or images, posing a serious threat to personal privacy and social trust. Therefore, developing efficient and accurate deepfake detection technologies is of paramount importance.

[0003] However, current deepfake detection technology still faces many challenges and shortcomings, mainly in the following aspects: Identity overfitting problem: Existing deep forgery detection models often tend to learn the accidental correlation between specific identities and forgery traces, rather than capturing general forgery features. This overfitting phenomenon causes the model's detection performance to drop significantly when faced with identities not present in the training set, severely affecting the generalization ability and practicality of the detection technology.

[0004] Feature obfuscation is a problem where traditional methods often include both identity information and forgery information when extracting features. These two types of information interfere with each other in the feature space, making it difficult for the model to effectively distinguish between identity-related features and forgery-related features. This feature obfuscation not only reduces detection accuracy but also increases the risk of false positives and false negatives.

[0005] Poor generalization ability: Existing models overfit to common forgery methods in the training data, resulting in limited detection capabilities against novel forgery techniques. With the continuous development of deepfake technology, new forgery methods are emerging constantly, placing higher demands on the adaptability and flexibility of detection technologies.

[0006] Insufficient interpretability: Many deepfake detection models operate on a black-box decision-making process, failing to explicitly indicate which features led to the forgery judgment. This lack of interpretability not only limits the model's reliability in practical applications but also hinders further research and improvement.

[0007] To address the aforementioned issues, there is an urgent need for a deep fake detection method based on feature decoupling and bi-branch enhancement to improve the accuracy of deep fake detection and effectively overcome the current challenges in deep fake detection. Summary of the Invention

[0008] To address the problems in the prior art, this application provides a deep fake detection method and apparatus based on feature decoupling and dual-branch enhancement, which can improve the accuracy of deep fake detection.

[0009] To solve at least one of the above problems, this application provides the following technical solution: Firstly, this application provides a deep pseudo-detection method based on feature decoupling and dual-branch enhancement, including: The face image to be detected is input into a pre-trained face recognition model with frozen parameters, and an identity reference vector is extracted. The identity reference vector is related to identity information and is used to provide real identity information. The face image to be detected is input into a preset deep learning network to extract original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and forgery information. The original deep features are input into an adversarial feature decoupling module. Through adversarial learning, the original deep features are decomposed into identity-related components and forgery-related components. The identity-related components are constrained to be close to the identity reference vector, while the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained forgery-related components are input into the dual-branch forgery enhancement module. The high-resolution branch receives high-resolution image input and determines the corresponding high-resolution forgery enhancement features. The low-resolution branch receives low-resolution image input and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially according to a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Forgery detection is then performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

[0010] Further, before inputting the original deep features into the adversarial feature decoupling module, the following steps are included: Based on the shared encoder, identity feature encoder, and forgery feature encoder, construct the corresponding adversarial feature decoupling module; The shared encoder is used to perform preliminary encoding on the original depth features to obtain a shared feature representation; The identity feature encoder is used to extract identity-related components from the shared feature representation and constrain the similarity between the identity-related components and the identity reference vector through a first loss function; The forgery feature encoder is used to extract forgery-related components from the shared feature representation, and constrains the orthogonality or low correlation between the forgery-related components and the identity-related components through a second loss function.

[0011] Furthermore, the training process of the forgery feature encoder includes: Construct a forgery feature detector, which is used to receive forgery-related components and determine whether the components originate from real face images or forged face images; By training the forgery feature encoder so that the forgery feature discriminator cannot accurately determine the source of the forgery-related components, the corresponding trained forgery feature encoder is determined and used to extract feature components that are unrelated to identity information and only related to forgery traces.

[0012] Furthermore, it also includes: The first loss function is a distance loss function based on cosine similarity, used to maximize the cosine similarity between the identity-related component and the identity reference vector; The second loss function is a regularized loss function based on matrix orthogonality, used to minimize the inner product of the forgery-related component and the identity-related component.

[0013] Further, the step of receiving high-resolution image input through the high-resolution branch and determining the corresponding high-resolution forgery enhancement features, and receiving low-resolution image input through the low-resolution branch and determining the corresponding low-resolution forgery features, includes: The high-resolution branch receives an m×m pixel image block or a pre-processed m×m pixel feature map to determine the corresponding high-resolution forgery enhancement features. The high-resolution branch is used to extract more forgery traces through high resolution. By receiving an n×n pixel image block or a pre-processed n×n pixel feature map through a low-resolution branch, the corresponding low-resolution forgery features are determined, where pixel m > pixel n.

[0014] Further, the step of sequentially performing feature enhancement on the low-resolution forgery features according to a preset upsampling compensation algorithm and a preset multilayer perceptron, and determining the corresponding low-resolution forgery enhancement features, includes: Bilinear interpolation or transposed convolution is used to improve the spatial resolution of the low-resolution forgery features to match the high-resolution forgery enhancement features, and the corresponding compensated low-resolution forgery features are determined. The compensated low-resolution forgery features are output in a fixed dimension using a fixed-dimensional multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. The low-resolution forgery enhancement features have the same number of channels as the high-resolution forgery enhancement features.

[0015] Further, the step of performing forgery detection based on the low-resolution forgery enhancement features and determining the corresponding forgery result includes: The low-resolution forgery enhancement features are classified into two categories to determine their authenticity using a preset classifier, thereby identifying the corresponding forgery result. And / or, by using a preset classifier to perform multi-classification forgery type identification on the low-resolution forgery enhancement features, the corresponding forgery result can be determined.

[0016] Secondly, this application provides a deep fake detection device based on feature decoupling and dual-branch enhancement, comprising: The identity feature extraction module is used to input the face image to be detected into a pre-trained and parameter-frozen face recognition model to extract the identity reference vector, which is related to identity information and is used to provide real identity information; An adversarial deep fake detection module is used to input the face image to be detected into a preset deep learning network to extract original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and fake information. The original deep features are input into an adversarial feature decoupling module, which decomposes the original deep features into identity-related components and fake-related components through adversarial learning. The identity-related components are constrained to be close to the identity reference vector, while the fake-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The forgery category determination module is used to input the constrained forgery-related components into the dual-branch forgery enhancement module. It receives high-resolution image input through the high-resolution branch and determines the corresponding high-resolution forgery enhancement features. It also receives low-resolution image input through the low-resolution branch and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially using a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Finally, forgery detection is performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

[0017] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement.

[0018] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement.

[0019] Fifthly, this application provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement.

[0020] As can be seen from the above technical solution, this application provides a deep fake detection method and apparatus based on feature decoupling and dual-branch enhancement. It extracts the true identity features of the face image to be detected by a pre-trained face recognition model with frozen parameters. The face image to be detected is then input into a trainable deep learning network to simultaneously extract identity-related components and fakeness-related components. The identity-related components are constrained to approximate the true identity features, while the fakeness-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained fakeness-related components are then input into a dual-branch fakeness enhancement module for feature enhancement, and fakeness detection is performed using the enhanced fakeness features to obtain the detection result. This improves the accuracy of deep fake detection. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is one of the flowcharts illustrating the deep pseudo-detection method based on feature decoupling and dual-branch enhancement in the embodiments of this application; Figure 2 This is a structural diagram of the deep fake detection device based on feature decoupling and dual-branch enhancement in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device in the embodiments of this application.

[0023] Figure label: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The acquisition, storage, use, and processing of data in this application all comply with relevant laws and regulations.

[0026] Considering the problems of feature confusion and poor generalization ability faced by deepfake technology in computer vision, this application provides a deepfake detection method and apparatus based on feature decoupling and bi-branch enhancement. The method extracts the true identity features of the face image to be detected by a pre-trained face recognition model with frozen parameters. The face image to be detected is then input into a trainable deep learning network to simultaneously extract identity-related components and forgery-related components. The identity-related components are constrained to approximate the true identity features, while the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained forgery-related components are then input into a bi-branch forgery enhancement module for feature enhancement. Forgery detection is then performed using the enhanced forgery features to obtain the detection result, thereby improving the accuracy of deepfake detection.

[0027] To improve the accuracy of deep fake detection, this application provides an embodiment of a deep fake detection method based on feature decoupling and dual-branch enhancement, see [link to relevant documentation]. Figure 1 The deep pseudo-detection method based on feature decoupling and dual-branch enhancement specifically includes the following: Step S101: Input the face image to be detected into a pre-trained and parameter-frozen face recognition model, and extract the identity reference vector. The identity reference vector is related to the identity information and is used to provide real identity information. Optionally, in this embodiment, this step uses a face recognition model with frozen (unupdated) parameters to extract the identity feature vector of the input face image. This is because, to prevent this identity extractor from being "contaminated" to learn forged information during subsequent training, we must ensure that it serves as a stable and unchanging identity reference benchmark.

[0028] Specifically, the process begins by using a pre-trained face recognition model with all its network parameters set to a non-updateable state to extract features from the input face image data. This face recognition model has been thoroughly pre-trained on a large-scale identity recognition dataset containing massive amounts of real face images before being deployed to this detection process, giving it good generalization ability in identity feature encoding. Preferably, the model architecture is based on deep convolutional neural networks, such as ResNet, ArcFace, or Vision Transformer, which can extract highly abstract and discriminative identity representations from face images.

[0029] In practice, the face image to be detected is fed into a pre-trained model with the frozen parameters. A fixed-dimensional real-number vector (identity baseline vector) is generated. This vector is a mathematical and dense representation of the identity attributes inherent in the input face. The pre-trained model has learned to map the faces of different individuals to different and relatively separate regions in the feature space. Therefore, this vector essentially encodes core identity information about "who this person is," such as relatively stable and strongly individual-related biometric features like facial bone structure, the relative positions and shapes of facial features.

[0030] In this embodiment, freezing the parameters of the pre-trained model means that during the entire training and optimization process of the deep fake detection model, the weights and bias parameters of the face recognition sub-module always maintain their initial values ​​when the pre-training is completed, and are not updated with the backpropagation algorithm. This ensures that the module's function is pure and stable, and its output is not affected by the downstream fake detection task objective, thus providing a pure and reliable reference anchor point for "real identity information".

[0031] Step S102: Input the face image to be detected into a preset deep learning network to extract the original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and forgery information. Input the original deep features into an adversarial feature decoupling module. Through adversarial learning, the original deep features are decomposed into identity-related components and forgery-related components. The identity-related components are constrained to be close to the identity reference vector. At the same time, the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. Optionally, traditional methods do not separate deep pseudo-features, which can easily lead to model confusion.

[0032] In this embodiment, this step uses mathematical constraints to separate real and fake information, allowing subsequent modules to focus on processing purely fake signals. Specifically, the identity component is guided to reconstruct the identity features extracted in the first step; if the reconstruction is successful, it indicates that it does indeed contain identity information. The fake component is guided to exclude identity information. Adversarial training specifically attempts to identify identity information from the fake component. If the discriminator fails, it means that the identity information in the fake component has been successfully removed.

[0033] Specifically, first, the image of the face to be detected is fed into a trainable deep learning network, preferably a convolutional neural network or a visual Transformer structure. This network is responsible for extracting the raw depth features from the image. The raw depth features are high-dimensional vectors or feature maps that contain various information from the face image, such as: Identity information: Characteristics related to an individual's identity, such as facial features, skin color, face shape, etc. Forged information: traces introduced by the forgery generation model, such as inconsistent textures, abnormal lighting, edge artifacts, etc. Other semantic information: such as facial expressions, posture, background, etc.

[0034] Because this information is mixed together in the feature space, traditional detection methods have difficulty effectively distinguishing which features are related to identity and which are related to forgery, which leads to the model being prone to overfitting to specific identities or forgery types in the training set.

[0035] To separate identity and forged information from the mixed features, this step employs an adversarial feature decoupling module, the core of which is a three-way encoder structure: The shared encoder performs preliminary encoding on the original deep features and extracts a shared feature representation.

[0036] The identity feature encoder extracts identity-related components from shared feature representations.

[0037] The forgery feature encoder extracts forgery-related components from the same shared feature representation.

[0038] During the model training phase, the above components are optimized collaboratively through the following loss function: Identity consistency loss: ensures that identity-related components are aligned with pre-extracted identity features; Feature orthogonal loss: the independence of the forced identity and forgery components in the feature space; Adversarial loss: Improves the discriminative power of forged features through a game between the discriminator and the encoder; Reconstruction loss (optional): Recombining the two components should reconstruct the original features, ensuring that no important information is lost during the decoupling process.

[0039] For the trained feature decoupling module, two decoupled feature components are output: Identity-related components: Clean representations of identity features, which, due to optimization through a loss function, converge with real identity features and can be used for subsequent identity verification or retrieval tasks (if needed). Forgery-related components: The purified forgery trace features will be sent to the subsequent dual-branch forgery enhancement module for further processing.

[0040] Understandably, this step achieves effective separation of identity and forged information at the feature level through orthogonal constraints and adversarial learning, avoiding the problem of mutual interference between the two types of information in traditional methods. At the same time, the model no longer relies on the accidental correlation between identity and forgery traces for judgment, alleviating the identity overfitting problem. Even when encountering unknown identities, it can make reliable detection based on purified forgery features. In addition, the decoupled forgery features focus more on general forgery traces rather than specific forgery methods, and have better generalization ability.

[0041] Step S103: The constrained forgery-related components are input into the dual-branch forgery enhancement module. The high-resolution branch receives the high-resolution image input and determines the corresponding high-resolution forgery enhancement features. The low-resolution branch receives the low-resolution image input and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially according to a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Forgery detection is then performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

[0042] Optionally, since different forgery methods leave traces at different scales—some at the microscopic texture, others at the macroscopic structure—this embodiment performs bi-branch feature enhancement on deepfake features to complete the forgery determination of the input face. The bi-branch design uses two magnifying glasses of different magnifications to examine simultaneously, ensuring that no forgery clues at any scale are missed.

[0043] Specifically, it involves two-branch input and feature extraction.

[0044] The module receives two images of the same face at different resolutions as input: the high-resolution branch receives the high-resolution image (e.g., 512×512 pixels), and the low-resolution branch receives the corresponding low-resolution version (e.g., 224×224 pixels). The high-resolution image retains richer detail information, especially subtle texture anomalies, lighting inconsistencies, edge artifacts, and other local forgery traces; the low-resolution image provides global structural information and overall semantic consistency features, which helps to capture cross-regional coordination anomalies. In this embodiment, the high-resolution and low-resolution data are only illustrative examples; using other resolutions does not affect the implementation of this embodiment.

[0045] Each branch is forward-propagated through a trainable convolutional neural network (such as a lightweight ResNet or EfficientNet variant) to extract depth-related features at its respective resolution. The high-resolution branch outputs "high-resolution fake enhancement features," and the low-resolution branch outputs "low-resolution fake features." These two features typically have different structures in both spatial and channel dimensions.

[0046] Because the low-resolution branch has a lower input resolution, the "low-resolution fake features" extracted from it are coarser in spatial detail compared to the high-resolution branch. To effectively integrate these features with those of the high-resolution branch, resolution and semantic alignment are required.

[0047] Upsampling compensation: First, a preset upsampling algorithm (such as bilinear interpolation, transposed convolution, or subpixel convolution) is used to spatially upscale the "low-resolution fake features" to make them consistent with the "high-resolution fake enhanced features" in height and width. This process not only enlarges the size but also attempts to recover some of the detailed structural details lost during downsampling through learnable upsampling kernels or interpolation weights.

[0048] Multilayer Perceptron (MLP) Mapping: Subsequently, the upsampled features are input into a pre-defined multilayer perceptron. Typically composed of several fully connected layers or 1×1 convolutions, its core function is non-linear transformation and alignment of channel dimensions. MLPs can learn the mapping relationship from low-resolution semantic features to high-resolution detail features, further compensating for any blurring or distortion that may be introduced during the upsampling process, ensuring that the "low-resolution fake enhancement features" are semantically consistent with and complementary to the "high-resolution fake enhancement features."

[0049] Specifically, the high-resolution branch and low-resolution branch structures mentioned above can be the same or different.

[0050] In implementations where high-resolution and low-resolution branches differ in structure, the high-resolution branch requires stronger representational capabilities. Therefore, a teacher network is employed. Compared to the student network, the teacher network has deeper layers, a wider number of channels, or a more advanced architecture, enabling it to extract more refined and robust forgery trace features from high-resolution input. This strong network acts as a "teacher" throughout the training process, not only learning its own detection task but also refining its learned knowledge about high-quality forgery features.

[0051] The low-resolution branch uses a relatively lightweight subnetwork. Then, in order to make up for the information loss of the low-resolution branch and enable it to work together with the high-resolution branch, the features extracted by the low-resolution branch are enhanced and aligned.

[0052] The "low-resolution forgery enhancement features" after enhancement and alignment are input into the classifier for binary classification of true and false and / or multi-class classification of forgery methods, and the final "forgery result" is output.

[0053] Understandably, by processing inputs at different resolutions through a dual-branch approach, the system can simultaneously analyze images from both macroscopic semantic and microscopic texture levels. Through supervised learning of high-resolution forged features, low-resolution forged features can gradually learn complex feature relationships. Furthermore, the features extracted by the two branches overlap while each having its own emphasis; the learning process is essentially a process of information complementarity and redundancy verification. For example, the high-resolution branch might misjudge due to local noise, while the global consistency judgment of the low-resolution branch can correct this.

[0054] In practical applications, forged videos or images are often compressed, transcoded, or have their resolution reduced, resulting in the loss of high-frequency details. In this step, the low-resolution branch specifically handles low-resolution input and performs feature recovery and enhancement through upsampling and MLP, enabling the model to make effective judgments based on the preserved mid-to-low-frequency information even when faced with media of poor image quality, thus enhancing the system's practicality and generalization ability.

[0055] This example demonstrates how this embodiment separates identity and forgery information through feature decoupling and enhances forgery trace features by combining two-branch learning, thereby improving the accuracy and robustness of deep fake detection.

[0056] As described above, the deep fake detection method based on feature decoupling and dual-branch enhancement provided in this application can extract the real identity features of the face image to be detected through a pre-trained face recognition model with frozen parameters. The face image to be detected is input into a trainable deep learning network to simultaneously extract identity-related components and fakeness-related components. The identity-related components are constrained to be close to the real identity features, while the fakeness-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained fakeness-related components are input into a dual-branch fakeness enhancement module for feature enhancement, and fakeness detection is performed through the enhanced fakeness-enhanced features to obtain the detection result, thereby improving the accuracy of deep fake detection.

[0057] In one embodiment of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement in this application, the method may further include the following: Step S201: Construct the corresponding adversarial feature decoupling module based on the shared encoder, identity feature encoder, and forgery feature encoder; Step S202: The shared encoder is used to perform preliminary encoding on the original depth features to obtain a shared feature representation; Step S203: The identity feature encoder is used to extract identity-related components from the shared feature representation and constrain the similarity between the identity-related components and the identity reference vector through a first loss function; Step S204: The forgery feature encoder is used to extract forgery-related components from the shared feature representation, and constrains the orthogonality or low correlation between the forgery-related components and the identity-related components through a second loss function.

[0058] Optionally, in this embodiment, a three-channel encoder structure involving an adversarial feature decoupling module is described.

[0059] Specifically, shared encoders.

[0060] First, the system receives raw deep features extracted by a trainable network. These features are mixed with individual facial identity information and potential traces of forgery. The core function of the shared encoder is to encode and transform these raw features, generating shared feature representations. This provides sufficient basic feature representations for subsequent specialized decoupling, preventing the loss of crucial information from identity and forgery features due to differing encoding objectives at the initial stage of decoupling, and ensuring a complete input source for the subsequent decoupling process.

[0061] Specifically, identity feature encoder.

[0062] The identity feature encoder specifically extracts identity-related components from shared feature representations. Its design goal focuses on capturing relatively stable feature patterns strongly correlated with facial identity, such as facial skeletal structure and the inherent geometric relationships of facial features. To ensure that the extracted components truly represent identity, they are constrained by a loss function. The core of the loss function is to calculate the similarity between the identity-related components and the true identity feature vector extracted by the pre-trained face recognition model, and to maximize this similarity through an optimization process. This forces the output of the identity feature encoder to align with a known, pure identity feature space, thereby ensuring that the extracted components effectively represent identity information and remain consistent with the source identity.

[0063] Specifically, a forged feature encoder.

[0064] The forgery feature encoder extracts forgery-related components in parallel from the same shared feature representation. The goal of the forgery feature encoder is to capture unnatural traces, inconsistencies, or statistical anomalies introduced by forgery operations such as tampering and synthesis. To ensure effective separation of forgery components from identity components, a loss function is used to constrain them. The core of this function is to measure the correlation between forgery-related and identity-related components, such as calculating their inner product or correlation coefficient, and minimizing this value through an optimization process to pursue orthogonality or extremely low correlation between the two. The effect is to forcibly "separate" identity information from forgery information in the feature space, making the forgery component contain as few identity clues as possible, thus becoming a purer feature representation more focused on tampering traces. This orthogonality constraint is key to successful decoupling, laying a clear and interference-free foundation for subsequent enhancement and classification of forgery features.

[0065] Through step S204, this embodiment successfully implemented the three-channel encoder design of the adversarial feature decoupling module, laying the foundation for the subsequent successful separation of real features and forged features.

[0066] In one embodiment of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement in this application, the method may further include the following: Step S301: Construct a forgery feature discriminator, which is used to receive forgery-related components and determine whether the components originate from real face images or forged face images; Step S302: By training the forgery feature encoder so that the forgery feature discriminator cannot accurately determine the source of the forgery-related components, the corresponding trained forgery feature encoder is determined to extract feature components that are unrelated to identity information and only related to forgery traces.

[0067] Optionally, in this embodiment, this step involves adversarial training of the spoofed feature encoder.

[0068] First, a forgery feature discriminator is constructed. This discriminator is a binary classification module; forgery-related components are input into it, and the output is a probability judgment of whether the component originates from a real or forged face image. In the early stages of training, this discriminator undergoes supervised learning based on existing labels (real / forged), with the goal of accurately distinguishing the source category of the input features.

[0069] In the adversarial training framework, the optimization objective of the spoofing feature encoder is opposite to that of the discriminator. The encoder's training no longer relies solely on reconstruction or decoupling losses, but introduces an adversarial loss: maximizing the discriminator's error in identifying the source of spoofing-related components. Specifically, in each training iteration, the encoder strives to generate feature representations that make it impossible for the discriminator to reliably distinguish between real and fake images; that is, making the spoofing-related components extracted from real face images approximate the spoofing-related components extracted from fake face images in terms of feature distribution, and vice versa.

[0070] During the process, we introduce a gradient inversion layer or adversarial loss function. During backpropagation, the discriminator's gradient is inverted or negated when it is fed into the encoder, thus driving the encoder to update weights in a direction that weakens the discriminator's discriminative ability. Through this adversarial game, the encoder gradually learns to filter out identity-related information from the features—because identity information is irrelevant to the authenticity of the image and cannot help deceive the discriminator—and strengthen the feature representations that are highly correlated with forgery traces, such as image synthesis traces, frequency domain anomalies, texture inconsistencies, and other forgery signals that exist across identities.

[0071] After sufficient adversarial training, the forgery-related components extracted by the forgery feature encoder will have the following characteristics: First, their distribution is highly confused among images from different sources (real / forged), causing the discriminator to tend to guess randomly; Second, the correlation between this component and the identity feature vector is significantly reduced, achieving the stripping of identity information; Third, the component retains and highlights common traces related to forgery techniques, and these traces have a certain generalization ability in different identities and different forgery methods.

[0072] Through step S302, this embodiment successfully trained a forgery feature encoder under an adversarial framework, ensuring the thoroughness of feature decoupling from an optimization mechanism perspective, and also enhancing the generalization recognition capability of the subsequent forgery detection module for unknown identities and novel forgery methods.

[0073] In one embodiment of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement in this application, the method may further include the following: Step S401: The first loss function is a distance loss function based on cosine similarity, used to maximize the cosine similarity between the identity-related component and the identity reference vector; Step S402: The second loss function is a regularized loss function based on matrix orthogonality, used to minimize the inner product of the forgery-related component and the identity-related component.

[0074] Optionally, in this embodiment, this step involves designing the loss functions for the identity feature encoder and the spoofing feature encoder. The loss functions for the identity feature encoder and the spoofing feature encoder include identity alignment loss and feature orthogonality loss, which work together to ensure the effectiveness and directionality of feature decoupling.

[0075] Specifically, the identity alignment loss is constructed based on cosine similarity. During training, the identity-related components extracted by the identity feature encoder and the identity reference vector from the pre-trained and frozen face recognition model are both normalized, and then the cosine similarity between them is calculated. This similarity value is between -1 and 1, with a larger value indicating closer identity information. The identity alignment loss is defined as the negative or inverse form of this similarity. During training, this loss is minimized through gradient descent, essentially pushing the direction of the identity-related components in the feature space to continuously approach the identity reference vector. This ensures that the identity components separated from the original deep features accurately reflect the inherent identity information of the input face, regardless of whether it is forged. Its role is to "anchor" the identity information to a stable and interpretable reference direction, thereby providing a clear identity reference for subsequent separation of forged features.

[0076] Feature orthogonality loss is built upon the principle of matrix orthogonality, aiming to minimize the inner product of the forgery-related component and the identity-related component. In its implementation, each component is treated as a feature vector, and their dot product (inner product) is calculated. Given normalized vectors, the absolute value of the inner product directly reflects the correlation between the two. Feature orthogonality loss is defined as the square or absolute value of this inner product. By minimizing this loss, the model forces the angle between the forgery-related component and the identity-related component in the vector space to approach 90 degrees during optimization, i.e., to become orthogonal. Orthogonality implies linear independence between the two feature vectors, mathematically ensuring that the forgery component contains as little identity information as possible, and the identity component is as free from forgery traces as possible. Its function is to construct a decoupled representation structure in the feature space, allowing subsequent forgery feature enhancement and classification to rely solely on pure, identity-independent forgery trace features, fundamentally avoiding interference from identity information in forgery determination.

[0077] Two loss functions are optimized simultaneously during training, constraining the feature decoupling process from the two dimensions of "alignment" and "orthogonality," respectively. The identity alignment loss ensures that the decoupled identity components have semantic consistency and stability, while the feature orthogonality loss ensures that the decoupled forgery components have independence and purity.

[0078] Through step S402, this embodiment successfully designed a loss function that enables the model to effectively separate feature subspaces from the mixed original deep features, each corresponding to different semantics (identity vs. forgery) and with minimal mutual interference, providing a high-quality, interpretable feature foundation for subsequent forgery detection.

[0079] In one embodiment of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement in this application, the method may further include the following: Step S501: Receive an m×m pixel image block or a pre-processed m×m pixel feature map through the high-resolution branch, and determine the corresponding high-resolution forgery enhancement features. The high-resolution branch is used to extract more forgery traces through high resolution. Step S502: Receive an n×n pixel image block or a pre-processed n×n pixel feature map through the low-resolution branch, and determine the corresponding low-resolution forgery features, where pixel m > pixel n.

[0080] Optionally, in this embodiment, this step is a dual-branch deep pseudo-feature extraction and enhancement step.

[0081] Specifically, the input data must first be ensured to meet specific specifications. The original face region to be detected is cropped, aligned and resampled to generate a standardized 512×512 pixel RGB image block; or in the preceding feature extraction network, the intermediate layer feature map is adjusted to a spatial resolution of 512×512 through a spatial transformation layer.

[0082] Preferably, in this embodiment, the high-resolution pixel m uses a 512×512 pixel RGB image block. Using images with other high-resolution pixels will not affect the implementation of this application embodiment.

[0083] The high-resolution image / feature map is then fed into a dedicated feature extraction network in the high-resolution branch for processing. Preferably, the high-resolution branch consists of multiple convolutional layers, attention modules, or hierarchical feature aggregation structures.

[0084] The core of the high-resolution branch design lies in preserving and refining high-frequency details and subtle anomalies in the image, such as unnatural transitions in facial textures, inconsistencies in lighting reflections, and synthetic artifacts at the edges of facial features. The network uses deep nonlinear transformations to parse feature tensors rich in spatial details and semantic information from the high-resolution input—the high-resolution forgery enhancement features. These features are not only high-dimensional, but their feature vectors at each spatial location also carry subtle forgery traces from the corresponding local image region, thus achieving dense and precise capture of forgery clues.

[0085] Specifically, a parallel low-resolution processing path is constructed. The system prepares low-resolution input simultaneously by directly downsampling the same face region to 224×224 pixels, or by extracting the corresponding resolution dimensionality-reduced feature map from the early layers of the high-resolution processing path.

[0086] Preferably, in this embodiment, the low-resolution pixel n is a 224×224 pixel RGB image block. Using images with other low-resolution pixels will not affect the implementation of this embodiment.

[0087] The low-resolution data is fed into a low-resolution branch, which has a lighter network structure with fewer convolutional layers or channels. This branch focuses on quickly capturing, at the global or regional level, forgery patterns and statistical anomalies that remain significant despite the reduced resolution. Examples include abnormal overall skin color distribution, facial symmetry violations, and cross-regional consistency errors inherited from certain forgery algorithms. The low-resolution forgery features output in this step, while having relatively coarse spatial details, are computationally efficient and exhibit complementary sensitivity to certain types of forgery traces.

[0088] The two steps are executed in parallel and work together. The high-resolution branch (S501) ensures the model's "microscopic" ability to detect fine local forgery traces, while the low-resolution branch (S502) provides a "wide-angle" rapid detection of global and intermediate forgery anomalies.

[0089] Through step S502, this embodiment successfully improved the robust detection capability of forgery traces at different scales by using dual-branch feature extraction, especially for complex deepfake content that contains both local fine artifacts and overall statistical anomalies.

[0090] In one embodiment of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement in this application, the method may further include the following: Step S601: Use bilinear interpolation or transposed convolution to improve the spatial resolution of the low-resolution forgery feature to match the high-resolution forgery enhancement feature, and determine the corresponding compensated low-resolution forgery feature. Step S602: The compensated low-resolution forgery features are output in a fixed dimension using a fixed-dimensional multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. The low-resolution forgery enhancement features have the same number of channels as the high-resolution forgery enhancement features.

[0091] Optionally, in this embodiment, this step further enhances the forged features obtained from the low-resolution branch by supervising the enhanced forged features of the high-resolution branch, so that the information content of the obtained low-resolution forged enhanced features is sufficiently rich.

[0092] Specifically, the low-resolution forgery features are first subjected to a spatial resolution upscaling operation to align them spatially with the high-resolution forgery enhancement features.

[0093] Preferably, the enhancement process can be implemented using bilinear interpolation or transposed convolution. If bilinear interpolation is used, the system generates a feature map with higher spatial resolution by weighted averaging of adjacent pixel values, based on the positional relationship between each pixel in the low-resolution feature map and the target high-resolution grid. If transposed convolution is used, a learnable convolution kernel is used to perform inverse convolution operations on the low-resolution feature map, gradually expanding its feature map size and introducing a certain feature reconstruction capability in the process.

[0094] Specifically, the compensated features are further processed with channel dimension alignment and feature enhancement. A fixed-dimensional multilayer perceptron, consisting of several fully connected layers, is used. The number of neurons in the final output layer is pre-set to match the number of channels in the high-resolution forged enhanced features. After the compensated low-resolution forged features undergo layer-by-layer nonlinear transformation and dimension mapping by the multilayer perceptron, the number of channels is uniformly adjusted to the target dimension. Simultaneously, forgery traces are extracted and enhanced, achieving strict alignment of the two branch outputs in the channel dimension. Furthermore, the nonlinear fitting capability of the multilayer perceptron is used to deeply reconstruct and enhance the forged features extracted from the low-resolution branch, ensuring that even with lower-resolution input, they achieve a level of feature expressiveness consistent with those extracted from the high-resolution branch.

[0095] Through step S602, this embodiment effectively ensures that the low-resolution input branch can output enhanced features that match the high-resolution branch in both spatial and channel dimensions and are comparable and complementary in expressing forged information, thereby providing a structurally standardized and information-rich input for subsequent forgery identification.

[0096] In one embodiment of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement in this application, the method may further include the following: Step S701: Use a preset classifier to perform binary classification to determine the authenticity of the low-resolution forgery enhancement features and determine the corresponding forgery result; Step S702: and / or, perform multi-classification forgery type identification on the low-resolution forgery enhancement features using a preset classifier to determine the corresponding forgery result.

[0097] Optionally, in this embodiment, the detection of forgery results can be authenticity classification and / or forgery type identification.

[0098] If the focus is on the task of authenticity identification, the system inputs the decoupled and enhanced low-resolution forgery enhancement features into a pre-trained binary classifier. Preferably, the binary classifier can be a network structure with a fully connected layer followed by a Sigmoid activation function. The output of the classifier is a probability value or a binary label (such as "0" for real and "1" for fake), which directly makes a final judgment on the authenticity of the input face media content.

[0099] If focusing on source tracing analysis tasks, the system inputs the same forgery feature representation into a pre-trained multi-classifier. Preferably, the multi-classifier can be a Softmax classification layer with multiple neuron outputs. The output of this classifier is a probability distribution that indicates which specific forgery generation method or tool the input content is most likely to belong to.

[0100] Understandably, different deepfake techniques, due to differences in their algorithmic principles, model architectures, and training data, will leave method-specific traces in the generated media. The dual-branch enhancement module amplifies general forgery traces while preserving these method-specific feature details. The role of the multi-classifier is to identify and distinguish these detailed patterns, thereby enabling technical tracing of the forged content.

[0101] Optionally, in this embodiment, the two forgery detection tasks mentioned above can be selected for application or applied simultaneously without affecting the implementation of this embodiment.

[0102] Through step S702, this embodiment successfully implemented two different levels of detection functions by configuring different classifiers, which can flexibly adapt to multi-level application needs from rapid screening to in-depth analysis.

[0103] To improve the accuracy of deep fake detection, this application provides an embodiment of a deep fake detection device based on feature decoupling and dual-branch enhancement, which implements all or part of the aforementioned deep fake detection method based on feature decoupling and dual-branch enhancement. See [link to embodiment]. Figure 2 The deep fake detection device based on feature decoupling and dual-branch enhancement specifically includes the following components: The identity feature extraction module 10 is used to input the face image to be detected into a pre-trained and parameter-frozen face recognition model to extract the identity reference vector, which is related to identity information and is used to provide real identity information. The adversarial deep fake detection module 20 is used to input the face image to be detected into a preset deep learning network to extract the original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and fake information. The original deep features are input into the adversarial feature decoupling module, and the original deep features are decomposed into identity-related components and fake-related components through adversarial learning. The identity-related components are constrained to be close to the identity reference vector, while the fake-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The forgery category determination module 30 is used to input the constrained forgery-related components into the dual-branch forgery enhancement module. It receives high-resolution image input through the high-resolution branch to determine the corresponding high-resolution forgery enhancement features, and receives low-resolution image input through the low-resolution branch to determine the corresponding low-resolution forgery features. It then sequentially enhances the low-resolution forgery features using a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Finally, it performs forgery detection based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

[0104] As can be seen from the above description, the deep fake detection device based on feature decoupling and dual-branch enhancement provided in this application embodiment can extract the real identity features of the face image to be detected through a pre-trained face recognition model with frozen parameters. The face image to be detected is input into a trainable deep learning network to simultaneously extract identity-related components and fakeness-related components. The identity-related components are constrained to be close to the real identity features, while the fakeness-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained fakeness-related components are input into a dual-branch fakeness enhancement module for feature enhancement, and fakeness detection is performed through the enhanced fakeness enhancement features to obtain the detection result, thereby improving the accuracy of deep fakeness detection.

[0105] From a hardware perspective, in order to improve the accuracy of deep fake detection, this application provides an embodiment of an electronic device for implementing all or part of the deep fake detection method based on feature decoupling and dual-branch enhancement. The electronic device specifically includes the following components: The system comprises a processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to realize information transmission between the deep fake detection method based on feature decoupling and dual-branch enhancement and core business systems, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the logic controller can be implemented with reference to the embodiments of the deep fake detection method based on feature decoupling and dual-branch enhancement in the previous embodiments, and the contents of these embodiments are incorporated herein, and repeated details will not be described again.

[0106] It is understood that the user terminal may include smartphones, tablet computers, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), in-vehicle devices, smart wearable devices, etc. Among these, the smart wearable devices may include smart glasses, smartwatches, smart bracelets, etc.

[0107] In practical applications, parts of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement can be executed on the electronic device side as described above, or all operations can be completed in the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed in the client device, the client device may further include a processor.

[0108] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0109] Figure 3 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 3 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 3 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0110] In one embodiment, the deep pseudo-detection method based on feature decoupling and dual-branch enhancement can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control: Step S101: Input the face image to be detected into a pre-trained and parameter-frozen face recognition model, and extract the identity reference vector. The identity reference vector is related to the identity information and is used to provide real identity information. Step S102: Input the face image to be detected into a preset deep learning network to extract the original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and forgery information. Input the original deep features into an adversarial feature decoupling module. Through adversarial learning, the original deep features are decomposed into identity-related components and forgery-related components. The identity-related components are constrained to be close to the identity reference vector. At the same time, the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. Step S103: The constrained forgery-related components are input into the dual-branch forgery enhancement module. The high-resolution branch receives the high-resolution image input and determines the corresponding high-resolution forgery enhancement features. The low-resolution branch receives the low-resolution image input and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially according to a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Forgery detection is then performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

[0111] As described above, the electronic device provided in this application embodiment extracts the real identity features of the face image to be detected through a pre-trained face recognition model with frozen parameters. The face image to be detected is input into a trainable deep learning network to simultaneously extract identity-related components and forgery-related components. The identity-related components are constrained to be close to the real identity features, while the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained forgery-related components are input into a dual-branch forgery enhancement module for feature enhancement, and forgery detection is performed through the enhanced forgery enhancement features to obtain the detection result. This can improve the accuracy of deep forgery detection.

[0112] In another implementation, the deep fake detection method based on feature decoupling and dual-branch enhancement can be configured separately from the central processing unit 9100. For example, the deep fake detection method based on feature decoupling and dual-branch enhancement can be configured as a chip connected to the central processing unit 9100, and the function of the deep fake detection method based on feature decoupling and dual-branch enhancement can be realized through the control of the central processing unit.

[0113] like Figure 3 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 3 All components shown; in addition, the electronic device 9600 may also include Figure 3 For components not shown, please refer to existing technologies.

[0114] like Figure 3 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device 9600.

[0115] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0116] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0117] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0118] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0119] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which is the same as in a conventional mobile communication terminal.

[0120] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 9130 is also coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0121] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the deep fake detection method based on feature decoupling and dual-branch enhancement, where the execution subject is a server or client, as described in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the deep fake detection method based on feature decoupling and dual-branch enhancement, where the execution subject is a server or client, as described in the above embodiments. For example, when the processor executes the computer program, it implements the following steps: Step S101: Input the face image to be detected into a pre-trained and parameter-frozen face recognition model, and extract the identity reference vector. The identity reference vector is related to the identity information and is used to provide real identity information. Step S102: Input the face image to be detected into a preset deep learning network to extract the original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and forgery information. Input the original deep features into an adversarial feature decoupling module. Through adversarial learning, the original deep features are decomposed into identity-related components and forgery-related components. The identity-related components are constrained to be close to the identity reference vector. At the same time, the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. Step S103: The constrained forgery-related components are input into the dual-branch forgery enhancement module. The high-resolution branch receives the high-resolution image input and determines the corresponding high-resolution forgery enhancement features. The low-resolution branch receives the low-resolution image input and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially according to a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Forgery detection is then performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

[0122] As described above, the computer-readable storage medium provided in this application embodiment extracts the real identity features of the face image to be detected through a pre-trained face recognition model with frozen parameters. The face image to be detected is input into a trainable deep learning network to simultaneously extract identity-related components and forgery-related components. The identity-related components are constrained to be close to the real identity features, while the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained forgery-related components are input into a dual-branch forgery enhancement module for feature enhancement, and forgery detection is performed through the enhanced forgery features to obtain the detection result. This can improve the accuracy of deep forgery detection.

[0123] Embodiments of this application also provide a computer program product capable of implementing all steps of the deep fake detection method based on feature decoupling and dual-branch enhancement, where the execution subject is a server or client, as described in the above embodiments. When executed by a processor, this computer program / instruction implements the steps of the deep fake detection method based on feature decoupling and dual-branch enhancement. For example, the computer program / instruction implements the following steps: Step S101: Input the face image to be detected into a pre-trained and parameter-frozen face recognition model, and extract the identity reference vector. The identity reference vector is related to the identity information and is used to provide real identity information. Step S102: Input the face image to be detected into a preset deep learning network to extract the original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and forgery information. Input the original deep features into an adversarial feature decoupling module. Through adversarial learning, the original deep features are decomposed into identity-related components and forgery-related components. The identity-related components are constrained to be close to the identity reference vector. At the same time, the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. Step S103: The constrained forgery-related components are input into the dual-branch forgery enhancement module. The high-resolution branch receives the high-resolution image input and determines the corresponding high-resolution forgery enhancement features. The low-resolution branch receives the low-resolution image input and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially according to a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Forgery detection is then performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

[0124] As described above, the computer program product provided in this application extracts the real identity features of the face image to be detected through a pre-trained face recognition model with frozen parameters. The face image to be detected is input into a trainable deep learning network to simultaneously extract identity-related components and forgery-related components. The identity-related components are constrained to be close to the real identity features, while the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained forgery-related components are input into a dual-branch forgery enhancement module for feature enhancement, and forgery detection is performed through the enhanced forgery features to obtain the detection result. This can improve the accuracy of deep forgery detection.

[0125] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0129] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A deep fake detection method based on feature decoupling and dual-branch enhancement, characterized in that, The method includes: The face image to be detected is input into a pre-trained face recognition model with frozen parameters, and an identity reference vector is extracted. The identity reference vector is related to identity information and is used to provide real identity information. The face image to be detected is input into a preset deep learning network to extract original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and forgery information. The original deep features are input into an adversarial feature decoupling module. Through adversarial learning, the original deep features are decomposed into identity-related components and forgery-related components. The identity-related components are constrained to be close to the identity reference vector, while the forgery-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The constrained forgery-related components are input into the dual-branch forgery enhancement module. The high-resolution branch receives high-resolution image input and determines the corresponding high-resolution forgery enhancement features. The low-resolution branch receives low-resolution image input and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially according to a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Forgery detection is then performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

2. The deep fake detection method based on feature decoupling and dual-branch enhancement according to claim 1, characterized in that, Before inputting the original deep features into the adversarial feature decoupling module, the following steps are included: Based on the shared encoder, identity feature encoder, and forgery feature encoder, construct the corresponding adversarial feature decoupling module; The shared encoder is used to perform preliminary encoding on the original depth features to obtain a shared feature representation; The identity feature encoder is used to extract identity-related components from the shared feature representation and constrain the similarity between the identity-related components and the identity reference vector through a first loss function; The forgery feature encoder is used to extract forgery-related components from the shared feature representation, and constrains the orthogonality or low correlation between the forgery-related components and the identity-related components through a second loss function.

3. The deep fake detection method based on feature decoupling and dual-branch enhancement according to claim 2, characterized in that, The training process of the forgery feature encoder includes: Construct a forgery feature detector, which is used to receive forgery-related components and determine whether the components originate from real face images or forged face images; By training the forgery feature encoder so that the forgery feature discriminator cannot accurately determine the source of the forgery-related components, the corresponding trained forgery feature encoder is determined and used to extract feature components that are unrelated to identity information and only related to forgery traces.

4. The deep fake detection method based on feature decoupling and dual-branch enhancement according to claim 2, characterized in that, Also includes: The first loss function is a distance loss function based on cosine similarity, used to maximize the cosine similarity between the identity-related component and the identity reference vector; The second loss function is a regularized loss function based on matrix orthogonality, used to minimize the inner product of the forgery-related component and the identity-related component.

5. The deep fake detection method based on feature decoupling and dual-branch enhancement according to claim 1, characterized in that, The process of receiving high-resolution image input through the high-resolution branch and determining the corresponding high-resolution forgery enhancement features, and receiving low-resolution image input through the low-resolution branch and determining the corresponding low-resolution forgery features, includes: The high-resolution branch receives an m×m pixel image block or a pre-processed m×m pixel feature map to determine the corresponding high-resolution forgery enhancement features. The high-resolution branch is used to extract more forgery traces through high resolution. By receiving an n×n pixel image block or a pre-processed n×n pixel feature map through a low-resolution branch, the corresponding low-resolution forgery features are determined, where pixel m > pixel n.

6. The deep fake detection method based on feature decoupling and dual-branch enhancement according to claim 5, characterized in that, The step of sequentially performing feature enhancement on the low-resolution forgery features according to a preset upsampling compensation algorithm and a preset multilayer perceptron, and determining the corresponding low-resolution forgery enhancement features, includes: Bilinear interpolation or transposed convolution is used to improve the spatial resolution of the low-resolution forgery features to match the high-resolution forgery enhancement features, and the corresponding compensated low-resolution forgery features are determined. The compensated low-resolution forgery features are output in a fixed dimension using a fixed-dimensional multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. The low-resolution forgery enhancement features have the same number of channels as the high-resolution forgery enhancement features.

7. The deep pseudo-detection method based on feature decoupling and dual-branch enhancement according to claim 1, characterized in that, The step of performing forgery detection based on the low-resolution forgery enhancement features and determining the corresponding forgery result includes: The low-resolution forgery enhancement features are classified into two categories to determine their authenticity using a preset classifier, thereby identifying the corresponding forgery result. And / or, by using a preset classifier to perform multi-classification forgery type identification on the low-resolution forgery enhancement features, the corresponding forgery result can be determined.

8. A deep fake detection device based on feature decoupling and dual-branch enhancement, characterized in that, The device includes: The identity feature extraction module is used to input the face image to be detected into a pre-trained and parameter-frozen face recognition model to extract the identity reference vector, which is related to identity information and is used to provide real identity information; An adversarial deep fake detection module is used to input the face image to be detected into a preset deep learning network to extract original deep features. The deep learning network is a trainable backbone network including convolutional layers and feature fusion structures. The original deep features contain mixed information of identity information and fake information. The original deep features are input into an adversarial feature decoupling module, which decomposes the original deep features into identity-related components and fake-related components through adversarial learning. The identity-related components are constrained to be close to the identity reference vector, while the fake-related components are constrained to be orthogonal or low-correlated with the identity-related components in the feature space. The forgery category determination module is used to input the constrained forgery-related components into the dual-branch forgery enhancement module. It receives high-resolution image input through the high-resolution branch and determines the corresponding high-resolution forgery enhancement features. It also receives low-resolution image input through the low-resolution branch and determines the corresponding low-resolution forgery features. The low-resolution forgery features are then enhanced sequentially using a preset upsampling compensation algorithm and a preset multilayer perceptron to determine the corresponding low-resolution forgery enhancement features. Finally, forgery detection is performed based on the low-resolution forgery enhancement features to determine the corresponding forgery result. The high-resolution branch and the low-resolution branch may have the same or different structures. When the structures are different, the high-resolution branch uses a teacher network.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the deep pseudo-detection method based on feature decoupling and dual-branch enhancement as described in any one of claims 1 to 7.

Images