Depth counterfeit image identification method based on differential feature search

Abstract

The invention discloses a deep counterfeit image identification method based on differentiable feature search, which breaks through the limitation of artificial feature selection through automatic feature mining and dynamic fusion, and comprises the following steps: S1, data collection and generation; s2, multi-dimensional feature construction is carried out; s3, constructing a feature structure search space; s4, carrying out differential feature structure search, and determining an optimal feature combination by adopting a'feature search space-classification model 'alternate training strategy; and S4, dynamic feature fusion and authentic identification model training: screening two optimal feature extraction paths with the highest weight, compressing and non-linearly fusing feature vectors and cascade classification models through a pre-training fusion model, training an end-to-end authentic identification model by using a public data set, and outputting an authenticity judgment result and confidence. According to the method, through an automatic feature search mechanism, counterfeit feature distribution of different generation models is adaptively matched, the detection robustness of cross-model and cross-technology iteration is remarkably improved, an efficient and universal technical scheme is provided for digital image authentic identification, the method can be widely applied to scenes such as real-time content auditing, legal evidence verification and copyright protection, and the method has wide application prospects. And false information management and digital content security are facilitated.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of deepfake content identification, and particularly relates to a deepfake image identification method based on differentiable feature search, and belongs to the fields of internet spread image forgery detection, multimedia forensics, intellectual property protection, portrait protection and the like. Specifically relates to image feature extraction, feature structure search and feature fusion identification method. BACKGROUND

[0002] With the rapid evolution of generative artificial intelligence technology, open-source image generation models represented by Stable Diffusion and Flux have the ability to generate high-quality images, and their generated results have significantly improved in visual fidelity and naturalness. Such models have lowered the technical threshold for creative expression, providing non-professional users with efficient image creation tools, but at the same time, they have also been maliciously used to generate highly realistic fake images. The proliferation of deepfake images poses a serious threat to information authenticity verification, copyright ownership determination and social trust systems, and high-precision and high-generalization identification technologies are urgently needed to support image forensics and false information governance. For this reason, various methods have been proposed to distinguish between fake images and real images.

[0003] Existing technologies for identifying deepfake images, such as CN202510282109.7, propose a deepfake detection method based on spatial-frequency feature integration and dynamic edge optimization, which combines spatial and frequency domain features through a spatial-frequency feature integration module to detect deepfakes. CN202510902331.2 proposes a deepfake detection method based on multi-modal features, further considering the expansion and optimization of feature dimensions, detection mechanisms, and data application scenarios, and by extracting multi-modal feature vectors of the data to be detected, and searching and matching with a vector index library constructed based on preset forgery conditions, to determine whether the data to be detected is deepfake data.

[0004] The above technical solutions have improved the forgery identification technology to some extent, however, these methods generally rely on manual preset in the feature selection process, mainly using single or specific dimensional features (such as spatial-frequency features, noise features, multi-modal features, etc.), which may not be the optimal choice for different generation model forgery traces, making it difficult to fully cover the specific forgery features of GAN, diffusion models, Transformers and other types of generation models; at the same time, the randomness of manually designed features results in high sensitivity of the model to changes in the distribution of fake image features, making it difficult to dynamically adapt to the rapid iteration of generation technology, and the generalization detection effect significantly decreases when facing new generation algorithms or cross-model fake images.

[0005] To address the aforementioned issues, this invention proposes a deep forgery image identification technology based on differentiable feature search. This technology, through differentiable feature structure search and feature fusion methods, can automatically search and select the optimal feature combination for the specific inherent features contained in forgery images generated by different generative models. This overcomes the limitations of manual feature selection and improves the comprehensive detection capability and generalization performance of forgery images generated by various generative models. Summary of the Invention

[0006] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a method for identifying deepfake images based on differentiable feature search.

[0007] This invention obtains the optimal feature combination by constructing a differentiable feature structure search module and a feature fusion module. Specifically, firstly, for the image to be detected and the corresponding generation model type, a gradient descent-based differentiable feature structure search algorithm is used to traverse a multi-dimensional feature space (including but not limited to spatial domain features, frequency domain features, color domain features, texture distribution features, noise distribution features, and semantic association features) to automatically mine a subset of innate features highly correlated with the forgery patterns of the generation model. Subsequently, the feature fusion module dynamically assigns weights and nonlinearly fuses the multi-dimensional features obtained through the search to form a comprehensive discriminative feature vector with model specificity. Finally, the vector is input into a classifier to achieve accurate recognition of deepfake images.

[0008] This invention overcomes the randomness and limitations of manual feature selection through an automated feature search mechanism. It can adaptively match the distribution of forged features from different generative techniques such as GANs, diffusion models, and Transformers, significantly improving the robustness of detecting forged images across different models. Furthermore, through dynamic feature adaptation capabilities, it effectively addresses the detection challenges brought about by the rapid iteration of generative AI technologies, providing an efficient and universal technical solution for digital image forgery detection. The technical solution adopted by this invention to achieve the above-mentioned objectives is as follows: A method for identifying deepfake images based on differentiable feature search includes the following steps: S1: Data Collection and Generation: Download publicly available fake image detection datasets, classify and organize the fake images in each dataset according to the generation method type, and use them as training and testing sets for classification models. Simultaneously, deploy various mainstream generation models to generate labeled data for feature structure search in S4, covering the distribution of fake features from different generation techniques; S2: Feature Construction: Implement a multi-dimensional feature extraction method, unify the input and output dimensions between features, and perform targeted performance optimization to improve feature extraction efficiency and ensure the computational efficiency of the feature extraction network in the subsequent feature structure search process; S3: Feature Search Space Construction: The multi-dimensional features in S2 are classified according to feature type to construct a search space composed of multiple feature layers. Each feature layer contains feature extraction nodes of the corresponding type. Feature interaction between layers is achieved through full connectivity. Learnable weight parameters are assigned to each node and its contained feature extraction methods to support the automatic traversal of the multi-dimensional feature space and the mining of talent feature subsets by the gradient descent-based differentiable feature structure search algorithm. S4: Feature Structure Search: The feature search space constructed in S3 is concatenated with the classification model, and the model is optimized using the labeled data generated in step S1 through alternating training. First, the weight parameters of the classification model are frozen. The weight parameters of each node and feature in the feature search space are iteratively updated based on the gradient descent algorithm to achieve automated mining of subsets of innate features in the multi-dimensional feature space. After completing one round of feature search space parameter update, the weight parameters in the feature search space are fixed, and the classification model is trained to optimize its classification decision-making ability. The process of updating the feature search space parameters and training the classification model is performed alternately until the iterative change of the weight parameters in the feature search space is less than a preset threshold, thereby determining the optimal feature combination structure. S5: Feature Fusion: For each layer of the network in the feature search space, the two nodes with the largest weight parameters are selected and retained. At the same time, the two feature extraction operations with the highest probability values ​​among the two nodes are extracted to determine the two optimal feature extraction structures. The feature vectors obtained by the image to be detected through the two optimal feature extraction structures are respectively input into the pre-trained feature fusion model. The model performs feature compression and nonlinear fusion operations to finally generate a comprehensive feature vector for training the classification model. S6: Counterfeit Detection Model Training: Based on the performance requirements of the task application scenario, such as real-time performance, detection accuracy, and computational resource limitations, a target classification model for classifying real and counterfeit images is selected. This model is then cascaded with the optimal feature extraction architecture and feature fusion module determined in step S5 to construct a complete counterfeit detection model. The publicly available dataset collected and categorized in step S1 is used as training data to iteratively train the cascaded counterfeit detection model. After training, the fused feature vector obtained by extracting features from the image to be detected using the optimal feature extraction architecture and processing it through the feature fusion module is input into the trained classification model, which then outputs the result determining the authenticity of the image to be detected.

[0009] Furthermore, in step S1: S1.1: Find the most commonly used fake datasets in the industry, and classify the fake images according to their image labels. Based on the type of generation method, they are divided into three main categories. This yields the dataset used to train the classification model. ; S1.2: Generate a dataset for feature structure search by deploying Stable Diffsion v2, Stable Diffsion v3, Flux.1, StyleGAN, BigGAN, and utilizing the APIs of commercial models Midjourney, DALL·E, and openAI 4o. This ensures that the dataset used for feature structure search and the training set of the fake detection model in S6 have no overlap, guaranteeing the fairness and generalization of training.

[0010] Furthermore, in step S2: S2.1: For images, 22 feature extraction methods are designed, including Gaussian filtering, median filtering, bilateral filtering, wavelet transform, etc., to extract statistical and noise features; edge and texture features are extracted using Sobel operator, Canny operator, Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG); color features are calculated using color histogram and local color mean / variance; frequency domain feature analysis is achieved using Discrete Cosine Transform (DCT) and its high- and low-frequency component separation; semantic features are obtained through pre-trained CLIP; and data augmentation operations such as random shuffling of image patches are supported. Finally, a feature extraction function set is obtained. , representing 22 feature extraction methods, corresponding to the extracted features , in It can be either the image to be detected or a feature vector; S2.2: For the extracted features Mapping it to a unified dimension, specifically expressed as ,Right now .in For feature dimension mapping model, These represent the specified batch size, the number of channels in the feature vector, the height of the feature vector, and the width of the feature vector, respectively.

[0011] Furthermore, in step S3: S3.1: Classification and Division of Multi-Dimensional Feature Types: Based on feature properties and extraction logic, classify the multi-dimensional features obtained in S2. The features are divided into six major categories: statistical and noise features, edge and texture features, color features, frequency domain features, semantic features, and data augmentation features. Specifically, they are represented as follows:

[0012] S3.2: Multi-layer feature layer architecture design: Based on the classification set in S3.1, a three-layer feature layer is constructed. Each floor Contains 6 nodes , respectively corresponding The six feature categories, nodes The output is .

[0013] S3.3: Fully Connected Feature Interaction Mechanism: The output of each node within a layer is obtained by weighted summation of the outputs of similar features within the node. The weights within each node are... ,in The number of feature functions in each feature class is specified, and cross-type feature interaction is achieved through learnable fully connected layers between layers. arrive Through the weight matrix Connected. Specifically, regarding the input... , No. The first layer The output of each node is:

[0014] The final output of the first layer is:

[0015] and The six weighted outputs will be used as... The input will eventually become the output of the third layer. The matrices in the matrix are concatenated to obtain Using this feature structure to search the space Set up.

[0016] Furthermore, in step S4: S4.1 Classification Model Cascade Architecture Construction: Generated from S3 As input to the classification model, an end-to-end cascaded architecture of "feature search space - classification model" is constructed. The specific design is as follows: Classification model selection: Supports Convolutional Neural Networks (CNN), Fully Connected Networks (FCN), pre-trained models (such as ResNet-50, ViT-B / 16), etc., and the input dimension needs to be consistent with... The matching output is the classification probability distribution of the target task (such as the sigmoid output for binary classification or the softmax output for multi-class classification).

[0017] The collaborative goal of the architecture is to automatically select the optimal feature combination through the feature search space, and to make decisions based on the selected features. The two are optimized together through labeled data to form a closed loop of "feature mining - classification decision".

[0018] S4.2 Alternating Training Method for Model Optimization: A strategy of alternating between "feature search space parameter updates" and "classification model training" is adopted to achieve synergistic optimization of feature selection and classification capabilities. This is specifically divided into two stages: Phase 1: Freeze the classification model and update the feature search space parameters. Fix all weight parameters of the classification model (such as convolutional kernels, fully connected layer weights, and bias terms) to prevent them from interfering with gradient propagation of feature weights during the feature search phase. Only iteratively update the learnable parameters in the feature search space, including: the weights within each node of each layer. and adjacent layers arrive Through the weight matrix Classification loss based on labeled data

[0019] To optimize the objective, gradient descent using the Adam optimizer is employed. The dataset used is... During training, when the loss function value of the feature search space decreases by less than [value] for 5 consecutive rounds... When this point is reached, the update for this phase will be stopped.

[0020] Phase 2: Fix the feature search space and train the classification model. All parameters in the fixed feature search space remain unchanged, preserving the current feature combination structure. Use the dataset... Training is performed, and the weight parameters of the classification model (such as the convolutional layer weights of CNN and the attention matrix of ViT) are updated iteratively until the accuracy on the validation set no longer improves.

[0021] S4.3 Iteration Termination Condition Judgment: Alternately execute phases 1-2 of S4.2. After each round of alternation, calculate the iterative change of all weight parameters in the feature search space:

[0022] These are the feature search space parameter sets for the current round and the previous round, respectively. and At that point, the feature search space parameters were determined to have converged, and alternating training was stopped.

[0023] Furthermore, in step S5: S5.1 Determination of the optimal feature combination structure: After the iteration terminates, the weight parameters of the feature search space are determined. and Convergence occurs, and the feature combination structure at this point represents the optimal solution. The weight parameters within each node are retained. The eigenfunctions corresponding to the two largest values ​​in the middle are, according to Through the weight matrix The element distribution determines the final selected feature type for each layer, thereby obtaining the ranking of feature paths. The two best paths are then selected, specifically represented as follows: The output features are and ,in The image to be detected.

[0024] S5.2 Feature fusion strategy: [This refers to a strategy involving the fusion of features in the image to be detected.] The feature vectors obtained through the two optimal feature extraction structures are respectively input into the pre-trained feature fusion model. The model performs feature compression and fusion operations to ultimately generate a comprehensive feature vector for training the classification model.

[0025] Furthermore, in step S6: S6.1 Define task performance requirements: Based on the actual application scenario, determine the core performance indicators and constraints of the model, including: Real-time performance: If deployed on mobile devices or in real-time detection scenarios, such as live streaming or video streams, priority should be given to model inference speed; Detection accuracy: If used in high-security scenarios, such as identity authentication or legal evidence, classification accuracy should be the primary focus. Determine the number of model parameters and the upper limit of computational complexity based on the computing power of the deployed devices.

[0026] S6.2 Selecting the target classification model: Based on the performance requirements in step 1, select the classification model that is ultimately suitable for the binary classification task of distinguishing between real and fake images. Here, the classic classification model ResNet-50 is used as an example.

[0027] The selected classification model is cascaded with the previously determined modules to form an end-to-end anti-counterfeiting model, with the following specific structure: S6.2 Training the model: Call the public dataset that has been collected and categorized in step S1. The data is preprocessed, including: data partitioning (dividing the data proportionally into training, validation, and test sets), data augmentation (randomly transforming the training set images, such as rotating, cropping, flipping, and color jittering, to improve the model's generalization ability), and format unification (resizing the images to the input size required by the feature extraction architecture and converting them to a model-compatible data format for training). The cascaded anti-counterfeiting model is then trained using the training data. After training, the following process is executed on the images to be detected to output the authenticity judgment result: Feature extraction: Input the image to be detected into the optimal feature extraction architecture to extract multi-dimensional features; Feature fusion: The feature fusion module converts multi-dimensional features into a fused feature vector; Classification prediction: Input the fused feature vector into the trained target classification model and output the binary classification probability; Output: Returns the final judgment result and the corresponding confidence level.

[0028] This invention utilizes an automated feature search mechanism to adaptively match the distribution of forgery features from different generation models, significantly improving detection robustness across models and technology iterations. It provides an efficient and universal technical solution for digital image forgery detection and can be widely applied in scenarios such as real-time content review, legal evidence verification, and copyright protection, thus contributing to the governance of false information and digital content security.

[0029] The beneficial effects of this invention are as follows: It solves the problem that existing forgery detection methods generally rely on manual presets during feature selection, mainly using single or specific dimensional features, which leads to high sensitivity of the model to changes in the distribution of forged image features. This makes it unable to dynamically adapt to the rapid iteration of generation technologies, and significantly reduces the generalization detection performance when facing new generation algorithms or cross-model forged images. This invention provides a systematic, end-to-end forgery image detection method, from feature space generation to feature search. The forgery detection features obtained by this method can make optimal selections for forgery traces from different generation models, comprehensively covering the specific forgery features of various generation models such as GAN, Transformer, and diffusion models. This improves the accuracy and generalization performance of forged image identification, thereby more efficiently empowering fields such as internet image forgery detection, multimedia forensics, intellectual property protection, and portrait protection. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings involved in the implementation process will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and do not limit the scope of the invention. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram illustrating the specific implementation process of the method of the present invention.

[0032] Figure 2 This is a schematic diagram of the feature structure search space construction in this invention.

[0033] Figure 3 This is a schematic diagram of the cascaded modules during the training and detection process in this invention. Detailed Implementation

[0034] Reference Appendix Figure 1 Appendix Figure 2 and attached Figure 3 The technical solution of the present invention will be described in detail below.

[0035] This implementation provides a method for identifying deepfake images based on differentiable feature search, including the following steps: S1: Data Collection and Generation: Download publicly available fake data datasets. Simultaneously deploy various mainstream generative models to generate labeled data for feature structure search in S4, covering the distribution of fake features from different generative techniques; S2: Feature Construction: Implement a multi-dimensional feature extraction method, unify the input and output dimensions between features, and perform targeted performance optimization to improve feature extraction efficiency and ensure the computational efficiency of the feature extraction network in the subsequent feature structure search process; S3: Feature Structure Search Space Construction: The multi-dimensional features in S2 are categorized according to feature type, constructing a search space composed of multiple feature layers, such as... Figure 2 As shown, each feature layer contains feature extraction nodes of the corresponding type. The layers interact with each other through a fully connected method. Learnable weight parameters are assigned to each node and its contained feature extraction method to support the automatic traversal of multi-dimensional feature space and the mining of talent feature subsets by the gradient descent-based differentiable feature structure search algorithm. S4: Feature Structure Search: The feature search space constructed in S3 is concatenated with the classification model, and the model is optimized using the labeled data generated in step S1 through alternating training. First, the weight parameters of the classification model are frozen. The weight parameters of each node and feature in the feature search space are iteratively updated based on the gradient descent algorithm to achieve automated mining of subsets of innate features in the multi-dimensional feature space. After completing one round of feature search space parameter update, the weight parameters in the feature search space are fixed, and the classification model is trained to optimize its classification decision-making ability. The process of updating the feature search space parameters and training the classification model is performed alternately until the iterative change of the weight parameters in the feature search space is less than a preset threshold, thereby determining the optimal feature combination structure. S5: Feature Fusion: For each layer of the network in the feature search space, the two nodes with the largest weight parameters are selected and retained. At the same time, the two feature extraction operations with the highest probability values ​​among the two nodes are extracted to determine the two optimal feature extraction structures. The feature vectors obtained by the image to be detected through the two optimal feature extraction structures are respectively input into the pre-trained feature fusion model. The model performs feature compression and nonlinear fusion operations to finally generate a comprehensive feature vector for training the classification model. S6: Counterfeit Detection Model Training: Based on the performance requirements of the task application scenario, such as real-time performance, detection accuracy, and computational resource limitations, a target classification model for classifying real and counterfeit images is selected. This model is then cascaded with the optimal feature extraction architecture and feature fusion module determined in step S5 to construct a complete counterfeit detection model. The publicly available dataset collected and categorized in step S1 is used as training data to iteratively train the cascaded counterfeit detection model. After training, the fused feature vector obtained by extracting features from the image to be detected using the optimal feature extraction architecture and processing it through the feature fusion module is input into the trained classification model, which then outputs the result determining the authenticity of the image to be detected.

[0036] Furthermore, in step S1: S1.1: Find the most commonly used fake datasets in the industry, and classify the fake images according to their image labels. Based on the type of generation method, they are divided into three main categories. This yields the dataset used to train the classification model. ; S1.2: Generate a dataset for feature structure search by deploying Stable Diffsion v2, Stable Diffsion v3, Flux.1, StyleGAN, BigGAN, and utilizing the APIs of commercial models Midjourney, DALL·E, and openAI 4o. This ensures that the dataset used for feature structure search and the training set of the fake detection model in S6 have no overlap, guaranteeing the fairness and generalization of training.

[0037] Furthermore, in step S2: S2.1: For images, 22 feature extraction methods are designed, including Gaussian filtering, median filtering, bilateral filtering, wavelet transform, etc., to extract statistical and noise features; edge and texture features are extracted using Sobel operator, Canny operator, Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG); color features are calculated using color histogram and local color mean / variance; frequency domain feature analysis is achieved using Discrete Cosine Transform (DCT) and its high- and low-frequency component separation; semantic features are obtained through pre-trained CLIP; and data augmentation operations such as random shuffling of image patches are supported. Finally, a feature extraction function set is obtained. , representing 22 feature extraction methods, corresponding to the extracted features , in It can be either the image to be detected or a feature vector; S2.2: For the extracted features Mapping it to a unified dimension, specifically expressed as ,Right now .in For feature dimension mapping model, These represent the specified batch size, the number of channels in the feature vector, the height of the feature vector, and the width of the feature vector, respectively.

[0038] Furthermore, in step S3: S3.1: Classification and Division of Multi-Dimensional Feature Types: Based on feature properties and extraction logic, classify the multi-dimensional features obtained in S2. The features are divided into six major categories: statistical and noise features, edge and texture features, color features, frequency domain features, semantic features, and data augmentation features. Specifically, they are represented as follows:

[0039] S3.2: Multi-layer feature layer architecture design: Based on the classification set in S3.1, a three-layer feature layer is constructed. Each floor Contains 6 nodes , respectively corresponding The six feature categories, nodes The output is .

[0040] S3.3: Fully Connected Feature Interaction Mechanism: The output of each node within a layer is obtained by weighted summation of the outputs of similar features within the node. The weights within each node are... ,in The number of feature functions in each feature class is specified, and cross-type feature interaction is achieved through learnable fully connected layers between layers. arrive Through the weight matrix Connected. Specifically, regarding the input... , No. The first layer The output of each node is:

[0041] The final output of the first layer is:

[0042] and The six weighted outputs will be used as... The input will eventually become the output of the third layer. The matrices in the matrix are concatenated to obtain Using this feature structure to search the space Set up.

[0043] Furthermore, in step S4: S4.1 Classification Model Cascade Architecture Construction: Generated from S3 As input to the classification model, an end-to-end cascaded architecture of "feature search space - classification model" is constructed. The specific design is as follows: Classification model selection: Supports Convolutional Neural Networks (CNN), Fully Connected Networks (FCN), pre-trained models (such as ResNet-50, ViT-B / 16), etc., and the input dimension needs to be consistent with... The matching output is the classification probability distribution of the target task (such as the sigmoid output for binary classification or the softmax output for multi-class classification).

[0044] The collaborative goal of the architecture is to automatically select the optimal feature combination through the feature search space, and to make decisions based on the selected features. The two are optimized together through labeled data to form a closed loop of "feature mining - classification decision".

[0045] S4.2 Alternating Training Method for Model Optimization: A strategy of alternating between "feature search space parameter updates" and "classification model training" is adopted to achieve synergistic optimization of feature selection and classification capabilities. This is specifically divided into two stages: Phase 1: Freeze the classification model and update the feature search space parameters. Fix all weight parameters of the classification model (such as convolutional kernels, fully connected layer weights, and bias terms) to prevent them from interfering with gradient propagation of feature weights during the feature search phase. Only iteratively update the learnable parameters in the feature search space, including: the weights within each node of each layer. and adjacent layers arrive Through the weight matrix Classification loss based on labeled data

[0046] To optimize the objective, gradient descent using the Adam optimizer is employed. The dataset used is... During training, when the loss function value of the feature search space decreases by less than [value] for 5 consecutive rounds... When this point is reached, the update for this phase will be stopped.

[0047] Phase 2: Fix the feature search space and train the classification model. All parameters in the fixed feature search space remain unchanged, preserving the current feature combination structure. Use the dataset... Training is performed, and the weight parameters of the classification model (such as the convolutional layer weights of CNN and the attention matrix of ViT) are updated iteratively until the accuracy on the validation set no longer improves.

[0048] S4.3 Iteration Termination Condition Judgment: Alternately execute phases 1-2 of S4.2. After each round of alternation, calculate the iterative change of all weight parameters in the feature search space:

[0049] These are the feature search space parameter sets for the current round and the previous round, respectively. and At that point, the feature search space parameters were determined to have converged, and alternating training was stopped.

[0050] Furthermore, in step S5: S5.1 Determination of the optimal feature combination structure: After the iteration terminates, the weight parameters of the feature search space are determined. and Convergence occurs, and the feature combination structure at this point represents the optimal solution. The weight parameters within each node are retained. The eigenfunctions corresponding to the two largest values ​​in the middle are, according to Through the weight matrix The element distribution determines the final selected feature type for each layer, thereby obtaining the ranking of feature paths. The two best paths are then selected, specifically represented as follows: The output features are and ,in The image to be detected.

[0051] S5.2 Feature fusion strategy: [This refers to a strategy involving the fusion of features in the image to be detected.] The feature vectors obtained through the two optimal feature extraction structures are respectively input into the pre-trained feature fusion model. The model performs feature compression and fusion operations to ultimately generate a comprehensive feature vector for training the classification model.

[0052] Furthermore, in step S6: S6.1 Define task performance requirements: Based on the actual application scenario, determine the core performance indicators and constraints of the model, including: Real-time performance: If deployed on mobile devices or in real-time detection scenarios, such as live streaming or video streams, priority should be given to model inference speed; Detection accuracy: If used in high-security scenarios, such as identity authentication or legal evidence, classification accuracy should be the primary focus. Determine the number of model parameters and the upper limit of computational complexity based on the computing power of the deployed devices.

[0053] S6.2 Selecting the target classification model: Based on the performance requirements in step 1, select the classification model that is ultimately suitable for the binary classification task of distinguishing between real and fake images. Here, the classic classification model ResNet-50 is used as an example.

[0054] The selected classification model is cascaded with the previously determined modules to form an end-to-end anti-counterfeiting model, with the following specific structure: S6.2 Training the model: Call the public dataset that has been collected and categorized in step S1. The data is preprocessed, including: data partitioning (dividing the data proportionally into training, validation, and test sets), data augmentation (randomly transforming the training set images, such as rotating, cropping, flipping, and color jittering, to improve the model's generalization ability), and format standardization (resizing the images to the input size required by the feature extraction architecture and converting them to a model-compatible data format for training). The cascaded anti-counterfeiting model is then trained using the training data. After training, the following process is executed on the images to be detected to output the authenticity judgment result.

[0055] The above embodiments are merely illustrative and not restrictive of the invention. Those skilled in the art will understand that many changes, modifications, and even equivalents can be made within the spirit and scope defined by the claims, all of which will fall within the protection scope of this invention.

Claims

1. A method for identifying deepfake images based on differentiable feature search, characterized in that: Includes the following steps: S1: Data Collection and Generation: Download publicly available fake data datasets for use as training and testing sets for classification models; simultaneously deploy mainstream generative models to generate labeled data covering the distribution of fake features from different generative techniques. S2: Feature Construction: Implement a multi-dimensional feature extraction method, unify the input and output dimensions between features, and perform targeted performance optimization; S3: Feature Search Space Construction: Multi-dimensional features are classified according to feature type, and a feature search space composed of multiple feature layers is constructed. Each feature layer contains feature extraction nodes of the corresponding type. Feature interaction between layers is achieved through full connection. Learnable weight parameters are assigned to each node and its contained feature extraction methods to support the automatic traversal of multi-dimensional feature space and the mining of talent feature subsets by gradient descent-based differentiable feature structure search algorithm. S4: Feature Structure Search: The feature search space is concatenated with the classification model, and the model is optimized using alternating training with step-annotated data. First, the weight parameters of the classification model are frozen, and the weight parameters of each node and feature in the feature search space are iteratively updated based on the gradient descent algorithm to achieve automated mining of subsets of innate features in the multi-dimensional feature space. After completing one round of feature search space parameter update, the weight parameters in the feature search space are fixed, and the classification model is trained to optimize its classification decision-making ability. The process of updating the feature search space parameters and training the classification model is performed alternately until the iterative change of the weight parameters in the feature search space is less than a preset threshold, thereby determining the optimal feature combination structure. S5: Feature Fusion: For each layer of the network in the feature search space, the two nodes with the largest weight parameters are selected and retained. At the same time, the two feature extraction operations with the highest probability values ​​among the two nodes are extracted to determine the two optimal feature extraction structures. The feature vectors obtained by the image to be detected through the two optimal feature extraction structures are respectively input into the pre-trained feature fusion model. The model performs feature compression and nonlinear fusion operations to finally generate a comprehensive feature vector for training the classification model. S6: Counterfeit Detection Model Training: Based on the performance requirements of the task application scenario, such as real-time performance, detection accuracy, and computational resource limitations, select a target classification model for classifying real and counterfeit images, and cascade it with the optimal feature extraction architecture and feature fusion module determined in step S5 to construct a complete counterfeit detection model; use the public dataset collected and classified in step S1 as training data to iteratively train the cascaded counterfeit detection model. After training, the image to be detected is processed by the optimal feature extraction architecture to extract features and the fused feature vector obtained by the feature fusion module is input into the trained classification model, which can then output the result of judging the authenticity of the image to be detected.

2. The deepfake image identification technology based on differentiable feature search as described in claim 1, characterized in that: Step S1 specifically includes: Dataset used to train classification models Datasets for feature structure search were generated by deploying Stable Diffsion v2, Stable Diffsion v3, Flux.1, StyleGAN, BigGAN, and utilizing the APIs of commercial models Midjourney, DALL·E, and openAI 4o. .

3. The deepfake image identification technology based on differentiable feature search as described in claim 1, characterized in that: Step S2 specifically includes: S2.1: For images, multiple feature extraction methods are designed, including Gaussian filtering, median filtering, bilateral filtering, and wavelet transform to extract statistical and noise features; edge and texture features are extracted using the Sobel operator, Canny operator, Local Binary Pattern (LBP), and Histogram of Oriented Gradients (HOG); color features are calculated using color histograms and local color mean / variance; frequency domain feature analysis is achieved using Discrete Cosine Transform (DCT) and its high- and low-frequency component separation; semantic features are obtained through pre-trained CLIP; and data augmentation operations such as random shuffling of image patches are supported; finally, a set of feature extraction functions is obtained. , representing k feature extraction methods, corresponding to the extracted features , in It can be either the image to be detected or a feature vector; S2.2: For the extracted features Mapping it to a unified dimension, specifically expressed as ,Right now ;in For feature dimension mapping model, These represent the specified batch size, the number of channels in the feature vector, the height of the feature vector, and the width of the feature vector, respectively.

4. The deepfake image identification technology based on differentiable feature search as described in claim 1, characterized in that: Step S3 specifically includes: S3.1: Classification and Division of Multi-Dimensional Feature Types: Based on feature properties and extraction logic, classify the multi-dimensional features obtained in S2. The features are divided into six major categories: statistical and noise features, edge and texture features, color features, frequency domain features, semantic features, and data augmentation features; specifically represented as follows: S3.2: Multi-layer feature layer architecture design: Based on the classification set in S3.1, a three-layer feature layer is constructed. Each floor Contains 6 nodes , respectively corresponding The six feature categories, nodes The output is ; S3.3: Fully Connected Feature Interaction Mechanism: The output of each node within a layer is obtained by weighted summation of the outputs of similar features within the node. The weights within each node are... ,in The number of feature functions in each feature class is specified, and cross-type feature interaction is achieved through learnable fully connected layers between layers. arrive Through the weight matrix Connected; specifically, for input , No. The first layer The output of each node is: The final output of the first layer is: and The six weighted outputs will be used as... The input will eventually become the output of the third layer. The matrices in the matrix are concatenated to obtain Search space based on this feature structure Set up.

5. The deepfake image identification technology based on differentiable feature search as described in claim 1, characterized in that: Step S4 specifically includes: S4.1 Classification Model Cascade Architecture Construction: Generated from S3 As input to the classification model, an end-to-end cascaded architecture of "feature search space - classification model" is constructed; the specific design is as follows: Classification model selection: Supports Convolutional Neural Networks (CNN), Fully Connected Networks (FCN), pre-trained models (such as ResNet-50, ViT-B / 16), etc., and the input dimension needs to be consistent with... The matching function outputs the classification probability distribution of the target task (such as the sigmoid output for binary classification or the softmax output for multi-class classification). Architecture Collaboration Goal: The feature search space is responsible for automatically selecting the optimal feature combination, the classification model makes decisions based on the selected features, and the two are optimized together through labeled data to form a closed loop of "feature mining - classification decision". S4.2 Alternating Training Method for Model Optimization: A strategy of alternating "feature search space parameter update" and "classification model training" is adopted to achieve synergistic optimization of feature selection and classification capabilities. This is specifically divided into two stages: Phase 1: Freeze the classification model and update the feature search space parameters; fix all weight parameters of the classification model (such as convolutional kernels, fully connected layer weights, and bias terms) to prevent them from interfering with the gradient propagation of feature weights during the feature search phase; only iteratively update the learnable parameters in the feature search space, including: the weights within each node of each layer. and adjacent layers arrive Through the weight matrix Classification loss based on labeled data To optimize the objective; gradient descent is performed using the Adam optimizer; the dataset is used. During training, when the loss function value of the feature search space decreases by less than [value] for 5 consecutive rounds... Stop updating at this stage; Phase 2: Fix the feature search space and train the classification model; keep all parameters of the fixed feature search space unchanged while maintaining the current feature combination structure; use the dataset. Training is performed, and the weight parameters of the classification model (such as the convolutional layer weights of CNN and the attention matrix of ViT) are updated iteratively until the accuracy on the validation set no longer improves. S4.3 Iteration Termination Condition Judgment: Alternately execute phases 1-2 of S4.

2. After each round of alternation, calculate the iterative change of all weight parameters in the feature search space: These are the feature search space parameter sets for the current round and the previous round, respectively. and At that point, the feature search space parameters were determined to have converged, and alternating training was stopped.

6. The deepfake image identification technique based on differentiable feature search as described in claim 1, characterized in that: Step S5 specifically includes: S5.1 Determination of the optimal feature combination structure: After the iteration terminates, the weight parameters of the feature search space are determined. and Convergence occurs, and the feature combination structure at this point is the optimal solution; the weight parameters within each node are preserved. The eigenfunctions corresponding to the two largest values ​​in the middle are, according to Through the weight matrix The element distribution determines the final selected feature type for each layer, thereby obtaining the ranking of feature paths. The two best paths are then selected, specifically represented as follows: The output features are and ,in The image to be detected; S5.2 Feature fusion strategy: [This refers to a strategy involving the fusion of features in the image to be detected.] The feature vectors obtained through the two optimal feature extraction structures are respectively input into the pre-trained feature fusion model. The model performs feature compression and fusion operations to ultimately generate a comprehensive feature vector for training the classification model.

7. The deepfake image identification technology based on differentiable feature search as described in claim 1, characterized in that: Step S6 specifically includes: S6.1 Define task performance requirements: Based on the actual application scenario, determine the core performance indicators and constraints of the model; S6.2 Selecting the target classification model: Based on the performance requirements in step 1, select the classification model that is ultimately suitable for the binary classification task of distinguishing between real and fake images. Here, the classic classification model ResNet-50 is used as an example. The selected classification model is cascaded with the previously determined modules to form an end-to-end anti-counterfeiting model, with the following specific structure: S6.2 Training the model: Call the public dataset that has been collected and categorized in step S1. The data is preprocessed, including data partitioning, data augmentation, and format standardization; the cascaded anti-counterfeiting model is trained using the training data. The data collection and generation module is responsible for collecting the training and test sets of the classification model and generating labeled data for feature structure search. The feature construction module is responsible for implementing multi-dimensional feature extraction methods, unifying the input and output dimensions between features, performing targeted performance optimization, improving feature extraction efficiency, and ensuring the computational efficiency of the feature extraction network during the subsequent feature structure search process. The feature structure search space construction module is responsible for constructing a feature search space to support gradient descent-based differentiable feature structure search algorithms. The feature structure search module is responsible for concatenating the constructed feature search space with the classification model, and using an alternating training method to optimize the model with labeled data, thereby determining the optimal feature combination structure. The feature fusion module is responsible for extracting the two nodes with the largest weight parameters in each layer of the network in the feature search space, and simultaneously extracting the two feature extraction operations with the highest probability values ​​from the two nodes, thereby determining two optimal feature extraction structures. The feature vectors obtained by the image to be detected through the two optimal feature extraction structures are then input into the pre-trained feature fusion model. The model performs feature compression and nonlinear fusion operations to finally generate a comprehensive feature vector for training the classification model. The aforementioned anti-counterfeiting model training module is responsible for the sequential cascading of the optimal feature extraction architecture and feature fusion module to construct a complete anti-counterfeiting model.

Images