Deep fake detection method based on local-global self-supervised contrastive learning

Abstract

The present application relates to the technical field of deep fake detection, in particular to a deep fake detection method based on local-global self-supervised contrastive learning; the method comprises the following steps: firstly, collecting unlabelled face images and labelled real-fake data and uniformly preprocessing, including face detection, extension cropping and key point positioning; secondly, dividing two global perspectives and seven local perspectives according to the key points, and constructing multi-view contrastive learning samples; further, designing a self-supervised pre-training process based on a teacher-student framework, learning robust face representation unsupervisedly by comparing the consistency of local and global features and reconstructing mask image blocks; finally, supervisedly fine-tuning the pre-trained model, and optimizing the classification structure by using neural network architecture search. The present application realizes high-precision and strong-generalization deep fake detection.

Description

Technical Field

[0001] This invention relates to the field of deepfake detection technology, specifically to a deepfake detection method based on local and global self-supervised comparative learning. Background Technology

[0002] With the rapid development of deep generative models (such as Generative Adversarial Networks (GANs), Diffusion Models, and Autoregressive Large Models), deepfake technology has been widely used. This technology can generate highly realistic face replacements, expression transfers, and face generation in images and videos. Although deepfakes have positive value in film and television entertainment, virtual humans, education, and other fields, their abuse in areas such as online rumor dissemination, identity forgery, public opinion manipulation, and fraud has also brought serious security risks. Therefore, developing effective deepfake detection methods has become an urgent problem to be solved.

[0003] Existing deepfake detection methods are mainly based on supervised learning, using manually labeled real and fake samples for model training. These methods have the following problems: 1) Insufficient generalization ability: The distribution of fake samples generated by different deepfake techniques varies, and the model often cannot be generalized to unknown fake types after being trained on one dataset; 2) High labeling cost: High-quality deepfake detection datasets require a large amount of manual labeling, which is costly to obtain and has a lag in updates, making it difficult to cover new fake techniques in a timely manner; 3) Insufficient sensitivity to local features: Deepfakes usually have fake traces in local areas (such as the edge of the lips, eyes, and skin texture), while traditional detection methods rely too much on global classification features and are difficult to capture fine-grained fake clues; 4) Lack of global consistency: Although some detection methods can focus on local areas, they lack modeling of global spatiotemporal consistency and are easily evaded by high-quality end-to-end generative models.

[0004] Therefore, how to improve the robustness and generalization of deep fake detection methods by simultaneously taking into account local detailed features and global consistency modeling in the absence of large-scale labeled data has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to address the problems existing in the background technology by proposing a deep fake detection method based on local and global self-supervised contrastive learning.

[0006] The technical solution of this invention: a deep fake detection method based on local and global self-supervised contrastive learning, comprising the following specific implementation steps: S1. Construct and preprocess the face data required for model training. Collect diverse unlabeled face images to form a pre-training set, and integrate real and fake images from multiple sources to form a labeled fine-tuning set. Then, perform face detection, extended cropping, and 68-point key point localization on all images to provide standardized input data for subsequent viewpoint segmentation and model training. S2. Based on the preprocessed face images and key points, several face viewpoints are systematically divided. By generating two global viewpoints containing the whole and random background sub-regions, as well as local viewpoints covering seven key regions including the whole face, forehead, eyes, nose, mouth and jaw, rich multi-view data pairs are constructed for self-supervised contrastive learning. S3. Design a self-supervised contrastive pre-training method based on a teacher-student framework. By having the student network learn to align local viewpoints with global viewpoints provided by the teacher network and reconstruct randomly masked global face blocks, the model learns robust face representations that combine local detail sensitivity with global consistency in an unsupervised manner. S4. Supervised fine-tuning of the pre-trained base model is performed, and a neural network architecture is used to search for and optimize the fine-tuning structure. By systematically searching for multi-scale feature fusion methods, feature token selection, pooling operations, and learning rate parameter combinations, the optimal fine-tuning configuration for deep pseudo-detection tasks is automatically found, thereby improving the accuracy and generalization ability of the final model.

[0007] Preferably, in step S1, face detection and extended cropping specifically involve: The YOLO-Face face detection model is used to detect and locate the face region in the input image. The bounding box of the region is then expanded to 2.5 times its original size before cropping to obtain the cropped face image. The key point localization specifically involves using the dlib face key point detector to accurately locate sixty-eight feature points in the cropped face image.

[0008] Preferably, in step S2, the two global perspectives include: The first global perspective is the cropped face image itself; The second global perspective is obtained by randomly cropping a sub-region from the cropped face image; The scaling factor during random cropping ranges from 1.5 to 2.5, ensuring that the sub-region includes part of the face and part of the random background.

[0009] Preferably, in step S2, the division of the local viewpoints of the seven key regions is based on the spatial coordinates of sixty-eight key points, and the images of the whole face, forehead, left eye, right eye, nose, mouth and jaw are cropped out respectively. When cropping each region, the minimum bounding rectangle determined by the key points of the region is randomly expanded outward by a preset multiple to increase the data diversity.

[0010] Preferably, in step S3, the teacher-student framework uses the Vision Transformer Base / 14 architecture as the feature encoder, and initially the student encoder and the teacher encoder have the same structure and parameters. During training, the student encoder receives a sequence of image patches from local perspectives and a sequence of global perspectives from image patches that are randomly occluded by 50%, while the teacher encoder receives a complete sequence of global perspective image patches as input.

[0011] Preferably, in step S3, self-supervised contrastive pre-training is achieved by minimizing a joint loss function, which includes global-local contrastive loss and mask reconstruction loss; The global-local contrast loss is the similarity loss between the classification token features extracted by the student encoder from a local perspective and the classification token features extracted by the teacher encoder from a global perspective; the mask reconstruction loss is the feature difference loss between the predicted features of the student encoder for the occluded global image patch and the corresponding real image patch features provided by the teacher encoder.

[0012] Preferably, in step S3, the joint loss function is a weighted sum of the global-local contrast loss and the mask reconstruction loss, with weight coefficients of 0.6 and 0.4, respectively. During training, the student encoder parameters are updated using gradient descent, while the teacher encoder parameters are smoothly updated from the student encoder using an exponential moving average strategy.

[0013] Preferably, in step S4, the supervised fine-tuning specifically involves: freezing the pre-trained student encoder as a fixed feature extractor, and converting the image to be detected into an image patch sequence for input into the encoder; The search space for neural network architecture search includes four optional dimensions: multi-scale feature sources, type of feature tokens used, whether to add pooling layers, and the learning rate of the classification head.

[0014] Preferably, in step S4, the neural network architecture search adopts a sequential search strategy, trying different feature fusion schemes, structural configurations and learning rate combinations in turn. For each combination, only the classification head is trained and the performance is evaluated on the validation set. Finally, the configuration with the highest accuracy on the validation set is selected as the optimal model architecture, and the configuration is used to perform final training on complete labeled data to obtain a deep forgery detection model.

[0015] Compared with the prior art, the above-mentioned technical solution of the present invention has the following beneficial technical effects: This invention designs a deep fake detection method based on local and global self-supervised contrastive learning. First, it significantly reduces the reliance on large-scale labeled data by learning general and robust facial representations from massive amounts of unlabeled face images through self-supervised contrastive learning, alleviating the bottlenecks of high labeling costs and slow updates in traditional supervised methods. Second, the proposed local and global multi-view partitioning mechanism enables the model to simultaneously capture subtle forgery traces in local facial regions and the global consistency of the overall structure, thereby improving the ability to identify highly realistic forgeries. Third, the use of a teacher-student framework combined with mask reconstruction and contrastive loss allows the model to establish semantic connections between local features and global context during the pre-training stage, enhancing the discriminative power and generalization of the representation. Furthermore, the introduction of neural network architecture search during the fine-tuning stage adaptively fuses multi-scale features and optimizes the classification structure, further improving the model's accuracy and stability in specific deep fake detection tasks. Overall, this invention achieves more refined, robust, and generalized detection of deep fake content without requiring extensive labeling, providing a sustainable and efficient solution to address the ever-evolving forgery techniques. Attached Figure Description

[0016] Figure 1 This is a flowchart of a deep fake detection method based on local and global self-supervised contrastive learning proposed in this invention; Figure 2 This is a schematic diagram of the local-to-global self-supervised contrastive learning pre-training framework proposed in this invention; Figure labels: n-fold crop face refers to the face region image obtained by cropping outwards by n times after face detection; m-fold crop face refers to a sub-region containing part of the background that is randomly cropped again from the crop face by a multiple of m, used as another global viewpoint; EMA represents exponential moving average, used to update teacher model parameters; patch feature corresponds to the token feature of the image patch transformation; class feature refers to the global summary feature represented by the [CLS] token; E s The student encoder is a trainable network that is directly updated via gradient descent during training, responsible for extracting features from both local and masked global perspectives; E t The teacher encoder represents the target network, whose parameters are smoothly updated from the student encoder through an exponential moving average, providing stable and consistent target features for contrastive learning. It processes the complete global view image; Token represents the basic data unit used to represent the image in the Vision Transformer architecture. Detailed Implementation

[0017] Example 1, as Figure 1 As shown, the deep fake detection method based on local and global self-supervised contrastive learning proposed in this invention includes the following specific implementation steps: S1. Constructing and preprocessing the face data required for model training: A pre-training set is formed by collecting diverse unlabeled face images, and a labeled fine-tuning set is constructed by integrating real and fake images from multiple sources; subsequently, face detection, extended cropping, and 68-point keypoint localization are performed on all images to provide standardized input data for subsequent viewpoint segmentation and model training, specifically: S11. A wide variety of facial images are collected extensively through web crawling. After automatic and manual cleaning, a large-scale, high-quality unlabeled face dataset is obtained, providing a data foundation for self-supervised learning. To train a feature extraction base model with strong generalization ability, it is first necessary to build a large-scale and diverse pre-training dataset of face images; this dataset does not contain any real or fake labels; Data collection: Using web crawling technology, face images and video frames were collected extensively from several publicly available internet image libraries, video platforms, and academic open-source face datasets (such as FFHQ, CelebA, etc.). This process aims to cover different races, ages, genders, lighting conditions, poses, expressions, and complex backgrounds to ensure the richness and diversity of the data. Data cleaning: The collected raw data is cleaned by a combination of automated and manual screening. First, a basic face detector is used to filter out images that do not contain any faces. Second, low-quality images that are excessively blurry, severely occluded, or have inappropriate content are removed manually or through a quality assessment model. Finally, a massive collection of high-quality face images is formed as a data source for self-supervised pre-training. S12. By combining images from web scraping, open-source deepfake datasets, and generated synthetic images, and after cleaning, labeling them as "real" or "fake," a real / fake face dataset is constructed for supervised fine-tuning, namely: To optimize the pre-trained model and enable it to distinguish between real and fake data, a fine-tuning dataset labeled "real" and "fake" needs to be constructed. This dataset is sourced from three sources: Real and known fake data scraped from the web: Collecting celebrity photos that have been verified as real from the internet, as well as deepfake content that has been publicly exposed or flagged; Open source deep fake detection dataset: Integrates existing academic benchmark datasets, such as FaceForensics++, Celeb-DF, DeepFakeDetection, etc., and utilizes their accurate real and fake labels; Generate synthetic data: Use current advanced deep generative models (such as StyleGAN3, Stable Diffusion, etc.) to synthesize highly realistic fake face images as a supplement to "fake" samples to enhance the model's potential to identify emerging forgery techniques; All collected data are cleaned uniformly to ensure that the faces are clear and usable, and strict binary classification annotation is performed to form a dataset for final supervised training and testing; S13. Use YOLO-Face to detect faces and crop them by 2.5 times to obtain "crop faces"; then use the dlib detector to locate 68 facial landmarks and save their coordinate information, i.e.: For all images in the two datasets mentioned above, a unified and precise preprocessing procedure is performed to lay the foundation for subsequent viewpoint segmentation and model input, including: Face detection and cropping: The YOLO-Face face detection model is used to detect faces in the input image and locate the bounding boxes of the face regions. To preserve the complete facial structure and some contextual background information, the size of the bounding box is expanded to 2.5 times its original size (i.e., parameter n=2.5). Subsequently, the face region image is cropped from the original image based on the expanded bounding box, and the image and the corresponding bounding box coordinate information are saved. This cropping result is denoted as crop face. Facial landmark localization: For each crop face image, the dlib facial landmark detector is used to accurately locate 68 feature points; these landmarks outline the structure of the face contour, eyebrows, eyes, nose, lips and other parts; the two-dimensional coordinate information of all landmarks is saved, which is the direct basis for subsequent fine local view segmentation. It should be noted that, in this embodiment, "crop face" refers to the face region image obtained by expanding and cropping the bounding box of the face after face detection and localization by a fixed ratio (e.g., n=2.5 times). It is not a simple face detection box. Its core meaning includes two points: first, it is accurately located by detectors such as YOLO-Face; second, it deliberately expands the cropping range to preserve the background context around the face and a more complete facial contour, providing standardized input rich in contextual information for subsequent global view segmentation and model learning. It should be noted that the dlib detector is a facial landmark detector in the open-source machine learning tool library widely used in the field of computer vision. It adopts a classic algorithm based on histogram of oriented gradient features and ensemble of regression trees, which can stably and efficiently locate 68 predefined landmark positions on a human face, including eyebrows, eyes, nose, mouth and facial contour; in this embodiment, it is used in the preprocessing stage to perform fine landmark localization on the cropped face region image, and the obtained landmark coordinates are the fundamental basis for subsequent accurate division of seven local view regions including the forehead, eyes, nose, and mouth, and provide a structured facial geometry information foundation for self-supervised learning.

[0018] S2. Based on the preprocessed face image and landmarks, a plurality of face views are systematically divided; by generating two global views including the whole and random background sub-regions, and local views covering seven key regions of the whole face, forehead, binocular, nose, mouth and jaw, rich multi-view data pairs are constructed for self-supervised contrastive learning, specifically as follows: S21. The cropped face is directly used as one global view, and a sub-region containing part of the face and background is randomly cropped therefrom as the second global view to introduce background diversity, that is: The global view is intended to provide overall context information including the face and part of the background; View G1 (whole cropped face): the cropped face image itself is directly used as a global view; View G2 (random background sub-region): a sub-region is randomly cropped from the cropped face image; during cropping, a scaling factor m is randomly selected, and its value range is set as 1.5<m<2.5; this operation ensures that the cropped sub-region (G2) must contain a part of the complete face and a part of random background, thereby introducing view variation and enhancing the model's robustness to background interference; in this embodiment, only one G2 view is generated for each face, so the total number of global views is 2; S22. According to the coordinates of 68 landmarks, seven local region images of the whole face, forehead, left eye, right eye, nose, mouth and jaw are cropped respectively, and each region is slightly expanded randomly to increase data variation, that is: Local views focus on the key anatomical regions of the face, aiming to capture possible subtle forgery traces; according to the spatial distribution of the 68 landmarks, the following 7 local region views are defined and cropped: L1 (whole face view): calculate the minimum bounding rectangle based on all internal facial landmarks (removing jaw contour points), and randomly expand outward by k times (1<k<1.2) before cropping, so as to obtain a pure frontal face image with background substantially removed; L2 (Forehead View): Determine the forehead area based on key points related to the upper edge of the eyebrows and the hairline, and then crop it outward by a factor of k. L3 (left eye view) and L4 (right eye view): The eye area is determined based on the outline points of the left eye (key points 36-41) and the right eye (key points 42-47) respectively, and then cropped after being enlarged; L5 (Nose View): Determine the nose area based on key points of the bridge and wings of the nose, then expand and crop it; L6 (Mouth View): Determine the mouth area based on the key points of the inner and outer contours of the lips, then expand and crop it; L7 (Jaw View): Determine the jaw area based on the key points of the chin contour, then expand and crop it; Each local viewpoint is cropped with a random, small expansion (k) to increase the slight diversity of the data; thus, each original input image is transformed into 2 global views and 7 local views, for a total of 9 viewpoint images.

[0019] S3. Design a self-supervised contrastive pre-training method based on a teacher-student framework. This method allows the student network to learn and align local viewpoints with global viewpoint features provided by the teacher network, and then reconstructs randomly masked global face blocks. This enables the model to learn robust face representations with both local detail sensitivity and global consistency in an unsupervised manner. Figure 2 As shown, specifically: S31. Initialize the student and teacher encoders using the ViT-B / 14 architecture. The teacher network parameters are initially the same as the student's, i.e.: Vision Transformer-Base / 14 (ViT-B / 14) was chosen as the backbone network of the feature encoder. Instantiate two encoders with identical structures: a student encoder and a teacher encoder; initially, all weight parameters of the teacher encoder are copied from the student encoder. It should be noted that ViT-B / 14 is a classic configuration of the Vision Transformer model. "B" stands for Base, indicating that the model has a moderate size and number of parameters, achieving a balance between computational efficiency and expressive power. "14" represents that the patch size of the input image segmentation process is 14x14 pixels. This architecture completely abandons traditional convolution operations and relies entirely on the Transformer's self-attention mechanism to learn the global relationships of image patch sequences, thereby capturing long-distance semantic features. In this embodiment, ViT-B / 14 is chosen as the encoder backbone network mainly because it has strong global context modeling capabilities and the potential for hierarchical understanding of image details, making it very suitable for completing self-supervised contrastive learning tasks that require simultaneous consideration of local anomaly perception and global consistency judgment. S32. Convert the global view image into a token sequence and mask it randomly by 50%. Also convert the local view image into a token sequence, i.e.: For a batch of training data, perform the following procedure: Global viewpoint processing: The global viewpoint images (G1, G2) are uniformly scaled to 224×224 pixels; each image is divided into 16×16 14×14 image patches; each patch is converted into a feature vector (called a token) through a linear projection layer; all tokens are arranged in spatial order to form the token sequence of the global viewpoint. ; Global mask generation: Randomly select a token sequence from a global perspective. 50% of the tokens are masked, meaning they are replaced with specific [MASK] tokens, generating a masked global sequence. ; Local viewpoint processing: The seven local viewpoint images (L1-L7) are uniformly scaled to 98×98 pixels; following the same logic, they are divided into 7×7 14×14 blocks and converted into tokens, forming a local viewpoint token sequence. ; S33. The student network processes the local sequence and the masked global sequence, while the teacher network processes the original global sequence and extracts [CLS] token features ([CLS] is a special token in the Transformer architecture, short for ClassificationToken) and image patch features, namely: The processed sequences are then input into the two networks respectively: The student network receives two inputs: a local viewpoint token sequence. and the occluded global sequence ; The teacher network receives one input: a global sequence of tokens. ; After processing by the encoder network (ViT), the following key features are extracted: Extracting the [CLS] token feature from the student network output: the local perspective (Global summary of a local image), and features of all image patches in the occluded global view. (Including predictions of shaded areas); Extract the [CLS] token feature from the teacher network output: the global perspective. (A global summary representing the entire image), and features of all image patches. (as the true goal of reconstruction); S34. Calculate the contrast loss between local and global [CLS] features, and the reconstruction loss of the student network for the mask block, and sum them by weight as the total loss, i.e.: Training is driven by minimizing a joint loss function: Global-local contrast loss : Calculate the [CLS] token features corresponding to the local viewpoint [CLS] token features corresponding to the global perspective Similarity loss between them (such as image-level cross-entropy loss); this loss forces the model to understand that the representation of the same face should remain consistent under different perspectives (local and global), thereby learning identity consistency and structural consistency; Mask Reconstruction Loss : Calculate the features of all image patches in the occluded global view. Features of all image patches Feature difference loss at the corresponding image patch location of the occluded area (such as patch-level cross-entropy loss); this task forces the student network to use visible contextual information (including knowledge from local viewpoints) to reason and reconstruct the occluded face parts, thereby enhancing the model's understanding of fine-grained facial structure and texture; Total loss The weighted sum of the two: ; in, and This represents the weighting coefficient, which in this embodiment takes values ​​of 0.6 and 0.4 respectively; S35. The student network is updated using gradient descent, while the teacher network parameters are smoothly updated from the student network using an exponential moving average, i.e.: Student network update: Calculating total loss The weights of the student network are updated using a gradient descent optimizer (such as AdamW) relative to the gradient of the student network parameters. Teacher network update: The parameters of the teacher network are not updated directly through gradient descent, but rather through an exponential moving average (EMA) strategy, smoothly evolving from the student network; after each iteration, the following is executed: ; in, Indicates the teacher's network parameters; Indicates student network parameters; Indicates the momentum coefficient; The above process is iteratively executed until the model converges. Finally, the student encoder is saved as a powerful, pre-trained base feature extractor (base model) for subsequent deep pseudo-detection tasks.

[0020] S4. Supervised fine-tuning of the pre-trained base model is performed, and a neural network architecture search is used to optimize the fine-tuning structure. By systematically searching for combinations of hyperparameters such as multi-scale feature fusion methods, feature token selection, pooling operations, and learning rates, the optimal fine-tuning configuration for deep fake detection tasks is automatically found, thereby improving the accuracy and generalization ability of the final model. Specifically: S41. Freeze the pre-trained student encoder and use it as a fixed feature extractor to process the input image, that is: Freeze all parameters of the pre-trained student encoder (base model) and use it as a fixed feature extractor; The cropped face images from the fine-tuning dataset (from step S13) are scaled to 224×224 pixels and converted into token sequences as in step S32, and then input into the frozen base model. S42. Clearly define the four dimensions of the search space: multi-scale feature sources, type of feature tokens used, whether to add pooling layers, and learning rate. Construct a structured search space containing four dimensions of options (“pools”): Multi-scale feature pool: [stage1, stage2, stage3]; This corresponds to the output feature maps of the last three Transformer blocks in the ViT-B encoder; stage1 is the deepest and most semantically rich feature, while stage3 is a relatively shallower feature containing more details; Token selection pool: [class_token, patch_token]; Choose to use the [CLS] token feature, which represents global information, or the average of the token features of all image patches; Average Pooling: [Yes, No]; Determines whether to apply a global average pooling operation after feature fusion; Learning rate pool: [1e-5, 2e-5, 5e-5, 1e-4, 2e-4, 5e-4, 1e-3, 2e-3, 5e-3]; This is used to select an appropriate learning rate for fine-tuning the classification head. S43. Sequentially try different feature fusions, structure configurations, and learning rate combinations to train the classification head and evaluate its performance on the validation set, i.e.: A systematic sequential search strategy is employed to train and evaluate possible combinations: Feature fusion: From the multi-scale feature pool, different feature combination methods are tried sequentially (e.g., using stage1 alone, fusing stage1+stage2, fusing stage1+stage3, fusing stage1+stage2+stage3); the fusion method is feature addition. Structure configuration: For each multi-scale fusion scheme, select one option from the token selection pool and the average pooling pool to construct the feature processing path; Training and Evaluation: At the end of the feature processing path defined above, add a brand new, trainable fully connected layer (classification head); select a learning rate from the learning rate pool, train only the parameters of the classification head and the average pooling layer that may be involved, and do not update the frozen base model parameters; train a small number of epochs on the fine-tuning training set using binary cross-entropy loss, and evaluate the deep pseudo detection accuracy of the model on an independent validation set. Traversal and Recording: Repeat the above process to traverse all meaningful strategy combinations and record the performance of each combination on the validation set; S44. Select the configuration with the highest accuracy on the validation set as the optimal model architecture, and use this configuration for final fine-tuning on the complete data, that is: Among all the strategy combinations tried, the combination that achieves the highest classification accuracy on the validation set is selected as the optimal configuration; Based on the actual search results of this embodiment, the optimal configuration is: Multi-scale features: fusing stage 1 (deep semantic features) and stage 3 (mid-level detailed features); Token selection: Use class_token ([CLS] token) as the representative of the merged feature; Average pooling: No additional average pooling layer is required; Learning rate: 5e-4; Using this optimal configuration, the model is fine-tuned on complete (training + validation) labeled data to obtain the final deep forgery detection model proposed in this embodiment. This model integrates the strong representational ability obtained by self-supervised pre-training with the classification structure optimized for the discrimination task, achieving high-precision, strong generalization and robust detection performance.

[0021] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A deep fake detection method based on local and global self-supervised contrastive learning, characterized in that, The specific implementation steps include the following: S1. Construct and preprocess the face data required for model training. Collect diverse unlabeled face images to form a pre-training set, and integrate real and fake images from multiple sources to form a labeled fine-tuning set. Then, perform face detection, extended cropping, and 68-point key point localization on all images to provide standardized input data for subsequent viewpoint segmentation and model training. S2. Based on the preprocessed face images and key points, several face viewpoints are systematically divided. By generating two global viewpoints containing the whole and random background sub-regions, as well as local viewpoints covering seven key regions including the whole face, forehead, left eye, right eye, nose, mouth and jaw, rich multi-view data pairs are constructed for self-supervised contrastive learning. Among them, the two global perspectives include: The first global perspective is the cropped face image itself; The second global perspective is obtained by randomly cropping a sub-region from the cropped face image; The scaling factor during random cropping ranges from 1.5 to 2.5, ensuring that the sub-region includes part of the face and part of the random background. The division of the local perspective of the seven key regions is based on the spatial coordinates of sixty-eight key points. The images of the whole face, forehead, left eye, right eye, nose, mouth and jaw are cropped out respectively. When cropping each region, the minimum bounding rectangle determined by the key points of the region is randomly expanded outward by a preset multiple to increase the diversity of data. Among them, "overall" refers to the first global view, which is the complete cropped image obtained after cropping; "full face" refers to the first local view among the seven local region images, which is the image obtained after cropping based on the internal facial key points of 68 facial key points, removing the chin contour points, and calculating the minimum bounding rectangle. S3. Design a self-supervised contrastive pre-training method based on a teacher-student framework. By having the student network learn to align local viewpoints with global viewpoints provided by the teacher network and reconstruct randomly masked global face blocks, the model learns robust face representations that combine local detail sensitivity with global consistency in an unsupervised manner. The teacher-student framework uses the Vision Transformer Base / 14 architecture as the feature encoder. Initially, the student encoder and the teacher encoder have the same structure and parameters. During training, the student encoder receives a sequence of image patches from local perspectives and a sequence of global perspectives from image patches that are randomly occluded by 50 percent as input, while the teacher encoder receives a complete sequence of global perspective image patches as input. Self-supervised contrastive pre-training is achieved by minimizing a joint loss function, which includes global-local contrastive loss and mask reconstruction loss. The global-local contrast loss is the similarity loss between the classification token features extracted by the student encoder from a local perspective and the classification token features extracted by the teacher encoder from a global perspective; the mask reconstruction loss is the feature difference loss between the predicted features of the student encoder for the occluded global image patch and the corresponding real image patch features provided by the teacher encoder. The joint loss function is a weighted sum of the global-local contrast loss and the mask reconstruction loss, with weight coefficients of 0.6 and 0.4, respectively. During training, the student encoder parameters are updated using gradient descent, while the teacher encoder parameters are updated smoothly from the student encoder using an exponential moving average strategy. S4. Supervised fine-tuning of the pre-trained base model is performed, and a neural network architecture is used to search for and optimize the fine-tuning structure. By systematically searching for multi-scale feature fusion methods, feature token selection, pooling operations, and learning rate parameter combinations, the optimal fine-tuning configuration for deep pseudo-detection tasks is automatically found, thereby improving the accuracy and generalization ability of the final model.

2. The deep fake detection method based on local and global self-supervised contrastive learning according to claim 1, characterized in that, In step S1, face detection and extended cropping specifically involve: The YOLO-Face face detection model is used to detect and locate the face region in the input image. The bounding box of the region is then expanded to 2.5 times its original size before cropping to obtain the cropped face image. The key point localization specifically involves using the dlib face key point detector to accurately locate sixty-eight feature points in the cropped face image.

3. The deep fake detection method based on local and global self-supervised contrastive learning according to claim 2, characterized in that, In step S4, the supervised fine-tuning specifically involves: freezing the pre-trained student encoder as a fixed feature extractor, and converting the image to be detected into a sequence of image patches to be input into the encoder; The search space for neural network architecture search includes four optional dimensions: multi-scale feature sources, type of feature tokens used, whether to add pooling layers, and the learning rate of the classification head.

4. The deep fake detection method based on local and global self-supervised contrastive learning according to claim 3, characterized in that, In step S4, the neural network architecture search adopts a sequential search strategy, trying different feature fusion schemes, structural configurations and learning rate combinations in turn. For each combination, only the classification head is trained and the performance is evaluated on the validation set. Finally, the configuration with the highest accuracy on the validation set is selected as the optimal model architecture, and the configuration is used to perform final training on complete labeled data to obtain the deep forgery detection model.

Images