Face-changing detection method, device and equipment based on multi-granularity feature fusion

Abstract

This invention discloses a face-swap detection method, apparatus, and device based on multi-granularity feature fusion. The method comprises: preprocessing a dataset containing real face data and face-swap data to obtain a preprocessed dataset; using the Xception model as a baseline network, improving the feature extraction layer based on a global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework comprising a coarse-grained global feature branch and a fine-grained local feature branch, introducing a convolutional attention mechanism to allocate attention in spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-swap detection model that fuses global and local multi-granularity features; training the artificial intelligence face-swap detection model based on the preprocessed dataset; and deploying the trained artificial intelligence face-swap detection model to perform face-swap detection on input facial videos or images to obtain face-swap recognition results. This invention can effectively improve the robustness and accuracy of face-swap detection.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a face-changing detection method, device and equipment based on multi-granularity feature fusion. Background Art

[0002] As a carrier of personal information, the human face plays a crucial role in authentication and authorization in modern society. As the importance of the human face in communication, security control, and financial transactions in digital urban environments becomes increasingly prominent, the remarkable development of deep learning technology has posed severe challenges to the reliability and security of facial recognition. In particular, the widespread application of AI-powered face-swapping technology, based on advanced image processing and deep learning methods such as generative adversarial networks, replaces or alters faces in videos with target faces without visual defects, thereby generating realistic and high-quality AI-powered face-swapping videos. However, videos that have undergone AI-powered face-swapping can lead to a large amount of false information and uncertainty, exposing society to greater security risks. Therefore, research on the effective detection and recognition of AI-powered face-swapping videos is crucial for maintaining security and trust in the digital world.

[0003] AI-powered face-swap detection has evolved through three stages: from methods based on traditional image processing and computer vision techniques, to methods based on machine learning techniques like deep learning, and finally to methods based on adversarial learning techniques like generative adversarial networks. Traditional image processing and computer vision methods fail or become inaccurate when used in complex scenarios, such as those involving varying lighting conditions, changes in expression, and posture, to detect high-quality AI-powered face-swap videos. Machine learning methods like deep learning face insufficient training data, overfitting, and poor generalization. Adversarial learning methods like generative adversarial networks also suffer from unstable training, difficulty evaluating, and vulnerability to adversarial attacks.

[0004] Therefore, how to improve the robustness and accuracy of deep classifier technology based on visual artifacts within the AI ​​face-changing video frame and be able to cope with different types of AI face-changing technologies has become the key to solving the above problems. Summary of the Invention

[0005] The embodiments of the present invention provide a face-swap detection method, apparatus, and device based on multi-granularity feature fusion, which can effectively improve the robustness and accuracy of face-swap detection.

[0006] An embodiment of the present invention provides a face-swap detection method based on multi-granularity feature fusion, comprising:

[0007] Step S1: preprocessing the dataset containing real face data and face-swapped data to obtain a preprocessed dataset;

[0008] Step S2: Using the Xception model based on depthwise separable convolution and residual connection as the baseline network, the feature extraction layer is improved based on the global-local multi-granularity feature fusion mechanism. A multi-granularity feature fusion framework is constructed, which includes a coarse-grained global feature branch and a fine-grained local feature branch. The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize them, thereby constructing an AI face-swap detection model that integrates global and local multi-granularity features.

[0009] Step S3: training the artificial intelligence face-swap detection model based on the preprocessed data set;

[0010] Step S4: deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain a face-changing recognition result.

[0011] As an improvement to the above solution, step S2: using the Xception model based on depthwise separable convolution and residual connection as the baseline network, improving the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introducing a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-swap detection model that integrates global and local multi-granularity features, including:

[0012] A global and local multi-granularity feature fusion framework is built between the intermediate and exit streams of the Xception model. By performing segmentation operations on the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multiple levels of granularity, forming a multi-granularity structure with coarse-grained global feature branches and fine-grained local feature branches.

[0013] The fine-grained local feature branches are constructed and processed. The construction and processing process includes: dividing the facial feature map output from the Xception backbone network into four fine-grained local feature sub-blocks of equal size, and inputting the four feature sub-blocks into four local feature branches respectively; each fine-grained local feature branch is sequentially convolved through a residual-connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling. The feature map size is output to 1x1x2048 for input to the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0014] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the facial feature map is processed through the three-stage convolution of the Xception network's inlet, intermediate, and outlet streams, gradually extracting global features and abstracting and optimizing them into higher-level feature expressions; by performing an average pooling operation on the entire image or feature map, the feature values ​​on each feature channel are averaged to obtain the overall feature representation, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0015] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Among them S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0016] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0017] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the middle stream stage of the Xception network using residual connections;

[0018] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and spatial context information of two different dimensions is obtained. It is then passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is M c (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multi-layer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0019] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, which is calculated as M s (F)=σ(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0020] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0021] Determine the loss function: A loss function consisting of global feature classification loss and fine-grained local feature classification loss is used. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L globol = -[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 indicates the real face category, y=1 indicates the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the predicted probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0022] As an improvement to the above solution, step S3: training the artificial intelligence face-swap detection model based on the preprocessed data set includes:

[0023] 3.1 Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0024]

[0025] θ t =θt-1 -ηυ t

[0026] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-swap detection model;

[0027] 3.2 Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement. The specific steps include:

[0028] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are grouped together. In the experimental dataset, the number of frames extracted from each real video is greater than the number of frames in the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0029] Sampling operation: The probability of random erasing operation is set to 50%, and n frames are sampled from the 20 frames of face images in each real video, and 1 frame is sampled from the 3 frames of images in each face-swapped video;

[0030] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-swapped images and frames of real face images to achieve a balance in the frequency of data enhancement processing between the true and false categories;

[0031] 3.3 Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The accuracy ACC calculation formula is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate,

[0032] As an improvement to the above solution, step S4: deploying the trained artificial intelligence face-swap detection model to perform face-swap detection on the input face video or image to obtain a face-swap recognition result includes:

[0033] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0034] Detection process: The user selects the face video or image to be detected and inputs it into the artificial intelligence face-changing detection model for detection;

[0035] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has undergone artificial intelligence face-changing processing. The detection system displays this result on the screen.

[0036] Another embodiment of the present invention provides a face-swap detection device based on multi-granularity feature fusion, including:

[0037] A preprocessing module is used to preprocess the data set containing real face data and face-swapped data to obtain a preprocessed data set;

[0038] A model building module uses the Xception model based on depthwise separable convolutions and residual connections as the baseline network. It improves the feature extraction layer based on a global-local multi-granularity feature fusion mechanism, constructs a multi-granularity feature fusion framework consisting of a coarse-grained global feature branch and a fine-grained local feature branch, and introduces a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize them, thereby building an AI face-swap detection model that integrates global and local multi-granularity features.

[0039] A model training module, used to train the artificial intelligence face-swap detection model based on the preprocessed data set;

[0040] The model deployment module is used to deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

[0041] As an improvement to the above solution, the model building module is specifically used to:

[0042] A global and local multi-granularity feature fusion framework is built between the intermediate and exit streams of the Xception model. By performing segmentation operations on the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multiple levels of granularity, forming a multi-granularity structure with coarse-grained global feature branches and fine-grained local feature branches.

[0043] The fine-grained local feature branches are constructed and processed. The construction and processing process includes: dividing the facial feature map output from the Xception backbone network into four fine-grained local feature sub-blocks of equal size, and inputting the four feature sub-blocks into four local feature branches respectively; each fine-grained local feature branch is sequentially convolved through a residual-connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling. The feature map size is output to 1x1x2048 for input to the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0044] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the facial feature map is processed through the three-stage convolution of the Xception network's inlet, intermediate, and outlet streams, gradually extracting global features and abstracting and optimizing them into higher-level feature expressions; by performing an average pooling operation on the entire image or feature map, the feature values ​​on each feature channel are averaged to obtain the overall feature representation, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0045] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Among them S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0046] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0047] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the middle stream stage of the Xception network using residual connections;

[0048] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and spatial context information of two different dimensions is obtained. It is then passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is Mc (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multi-layer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0049] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, which is calculated as M s (F)=σ(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0050] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0051] Determine the loss function: A loss function consisting of global feature classification loss and fine-grained local feature classification loss is used. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global = -[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 indicates the real face category, y=1 indicates the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the predicted probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0052] As an improvement to the above solution, the model training module is specifically used to:

[0053] Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0054]

[0055] θ t =θ t-1 -ηυ t

[0056] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-swap detection model;

[0057] Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement, including:

[0058] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are grouped together. In the experimental dataset, the number of frames extracted from each real video is greater than the number of frames in the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0059] Sampling operation: The probability of random erasing operation is set to 50%, and n frames are sampled from the 20 frames of face images in each real video, and 1 frame is sampled from the 3 frames of images in each face-swapped video;

[0060] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-swapped images and frames of real face images to achieve a balance in the frequency of data enhancement processing between the true and false categories;

[0061] Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The accuracy ACC calculation formula is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate,

[0062] As an improvement to the above solution, the model deployment module is specifically used to:

[0063] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0064] Detection process: The user selects the face video or image to be detected and inputs it into the artificial intelligence face-changing detection model for detection;

[0065] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has undergone artificial intelligence face-changing processing. The detection system displays this result on the screen.

[0066] Another embodiment of the present invention provides a face-changing detection device based on multi-granularity feature fusion, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the face-changing detection method based on multi-granularity feature fusion described in the above-mentioned embodiment of the invention.

[0067] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0068] By preprocessing the dataset containing facial data to reduce computational complexity and improve training efficiency; then, the Xception model was constructed and optimized. By introducing the global-local multi-granularity feature fusion mechanism and the convolutional attention mechanism, the ability to focus on key areas in the image was enhanced, and the feature expression was refined and adaptively optimized in the spatial and channel dimensions, thereby constructing an artificial intelligence face-swap detection model; then, the improved artificial intelligence face-swap detection model was trained using the preprocessed dataset; finally, the trained artificial intelligence face-swap detection model was deployed into the detection system to achieve efficient face-swap recognition of input face videos or images. In the embodiment of the present invention, the multi-granularity feature fusion framework overcomes the insufficient feature expression caused by single-scale feature extraction by simultaneously extracting and utilizing coarse-grained global features and fine-grained local features, while the convolutional attention mechanism improves the model's attention to subtle and important features in the image, thereby significantly improving the robustness and accuracy of the deep classifier technology for visual artifacts within the video frame based on artificial intelligence face-changing. Through the combined effect of the above technical processes, it is ensured that the artificial intelligence face-changing detection model can not only maintain high accuracy in complex scenarios such as different lighting conditions, expression changes, and posture changes, but also effectively cope with different types of artificial intelligence face-changing technologies, providing a more reliable means of identity verification. Therefore, the embodiment of the present invention can effectively improve the robustness and accuracy of face-changing detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 11 is a flow chart of a face-swap detection method based on multi-granularity feature fusion provided by one embodiment of the present invention;

[0070] Figure 2 This is a structural diagram of an artificial intelligence face-swap detection network that integrates global and local multi-granularity features, provided by one embodiment of the present invention;

[0071] Figure 3 Schematic diagram of a feature segmentation method provided by an embodiment of the present invention;

[0072] Figure 4 2 is a schematic diagram of introducing a convolutional attention module according to an embodiment of the present invention;

[0073] Figure 5 1 is a schematic structural diagram of a face-changing detection device based on multi-granularity feature fusion provided by an embodiment of the present invention;

[0074] Figure 6 This is a structural diagram of a face-changing detection device based on multi-granularity feature fusion provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0075] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0076] See also Figure 1 , is a flow chart of a face-swap detection method based on multi-granularity feature fusion provided by one embodiment of the present invention. The face-swap detection method based on multi-granularity feature fusion includes the following steps:

[0077] Step S1: preprocessing the dataset containing real face data and face-swapped data to obtain a preprocessed dataset;

[0078] Step S2: Using the Xception model based on depthwise separable convolution and residual connection as the baseline network, the feature extraction layer is improved based on the global-local multi-granularity feature fusion mechanism. A multi-granularity feature fusion framework is constructed, which includes a coarse-grained global feature branch and a fine-grained local feature branch. The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize them, thereby constructing an AI face-swap detection model that integrates global and local multi-granularity features.

[0079] Step S3: training the artificial intelligence face-swap detection model based on the preprocessed data set;

[0080] Step S4: deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain a face-changing recognition result.

[0081] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0082] By preprocessing the dataset containing facial data to reduce computational complexity and improve training efficiency; then, the Xception model was constructed and optimized. By introducing the global-local multi-granularity feature fusion mechanism and the convolutional attention mechanism, the ability to focus on key areas in the image was enhanced, and the feature expression was refined and adaptively optimized in the spatial and channel dimensions, thereby constructing an artificial intelligence face-swap detection model; then, the improved artificial intelligence face-swap detection model was trained using the preprocessed dataset; finally, the trained artificial intelligence face-swap detection model was deployed into the detection system to achieve efficient face-swap recognition of input face videos or images. In the embodiment of the present invention, the multi-granularity feature fusion framework overcomes the insufficient feature expression caused by single-scale feature extraction by simultaneously extracting and utilizing coarse-grained global features and fine-grained local features, while the convolutional attention mechanism improves the model's attention to subtle and important features in the image, thereby significantly improving the robustness and accuracy of the deep classifier technology for visual artifacts within the video frame based on artificial intelligence face-changing. Through the combined effect of the above technical processes, it is ensured that the artificial intelligence face-changing detection model can not only maintain high accuracy in complex scenarios such as different lighting conditions, expression changes, and posture changes, but also effectively cope with different types of artificial intelligence face-changing technologies, providing a more reliable means of identity verification. Therefore, the embodiment of the present invention can effectively improve the robustness and accuracy of face-changing detection.

[0083] As one embodiment, step S1, data set preprocessing: a series of data preprocessing operations are performed on the data set images to better highlight the key features and visual artifact details of the AI ​​face-changing data set, reduce the model computational complexity, and improve training efficiency. Specifically, the data set preprocessing described in step S1 can be performed as follows:

[0084] Given that datasets used in AI face-swap detection tasks are typically in the form of video frames, directly using the entire video dataset would result in a massive amount of data, and would also increase the time and complexity required to train the model. Therefore, the dataset can be evenly sampled, ensuring that the number of AI face-swap images and real face images in the training and test sets is equal, and the training and test sets are then divided proportionally.

[0085] To better highlight the key features and visual artifact details of AI face-swapping datasets, reduce model computational complexity, and improve training efficiency, a preprocessing step called face location and cropping should be performed first. This removes irrelevant parts like the background and body, retaining only the image containing the face. Specifically, for each frame, the Dlib toolkit's face detector and alignment algorithm are used to detect 68 facial key points. Then, a rectangular frame is generated based on 51 key points for areas like eyebrows, eyes, nose, and lips. This resulting rectangular frame is then magnified 1.2x for cropping.

[0086] However, AI face-swapped image datasets still exhibit significant variations in lighting conditions, angular positions, facial expressions, and modal differences. This inevitably results in a certain amount of abnormal noise interference in the images, which in turn affects the classification and recognition performance of AI face-swapped detection. Therefore, a series of data preprocessing operations are required after cropping the faces. These include image resizing to convert these images into tensor form and input them into the network model for iterative training; data normalization and standardization to reduce computational complexity and accelerate convergence; and data augmentation through random horizontal flipping, random erasing, and edge padding to increase data diversity, improve the model's generalization ability, and reduce reliance on specific locations and pixel values, making the model more robust.

[0087] Specifically, considering that the facial images in the dataset may have background interference, different sizes, and small samples, a series of image preprocessing operations such as face positioning and cropping, de-impurity, uniform size scaling, and richness expansion are required to facilitate better training results when the subsequent dataset is input into the network model, thereby improving model performance from the perspective of data processing.

[0088] Face localization and cropping is a key step in the preprocessing of deepfake face detection. This step uses a face detector to accurately locate the face area in the input image and crop it out to obtain an image containing only the face.

[0089] In the task of deepfake face detection, the primary goal of this research is to determine whether a deepfake face exists in the input image. The facial region is the primary focus, while other parts do not contribute to the task. By locating and cropping the face in the image, the background, body, and other irrelevant areas are removed, reducing the model's processing complexity and improving computational efficiency. Furthermore, the cropped face region is more concentrated and prominent, highlighting key facial features and minor visual artifacts while also directing the model's neural network to focus on the face. This helps the model better learn and distinguish the subtle differences between real and fake faces, thereby improving detection accuracy.

[0090] The specific steps for face positioning and cropping are as follows:

[0091] (1) Load the Dlib library face detector for detecting faces in images and use the OpenCV library to read the dataset images to be processed.

[0092] (2) If no face is detected, skip the image; if multiple faces are detected, select the face with the largest area as the target face.

[0093] (3) Obtain the bounding box coordinates (upper left corner and lower right corner coordinates) of the target face and make fine adjustments as needed, such as increasing a certain proportion of the boundary to include more key information.

[0094] (4) Based on the calculated facial region coordinates, the facial image is cropped from the original image. In this step, cropped images that are too small can be filtered out to improve data quality.

[0095] (5) According to the label of the face data (real face or fake face), the cropped face images are classified and stored.

[0096] Furthermore, image resizing is the process of normalizing the image data fed into a deep learning model. Since cropped facial images in a dataset often vary in size, and deep learning models have specific requirements for input image dimensions, image resizing must be normalized before training. For deepfake face detection, image data must be converted into tensor data for input into the network model for training or testing. Since the tensor input dimensions of network models are typically fixed, normalizing the image resizing is essential.

[0097] To improve the model's training performance and generalization capabilities, data augmentation and data normalization are also required. Data augmentation aims to increase the diversity of the dataset and includes random horizontal flipping, random cropping after padding, and random erasing. Data normalization transforms and normalizes the image data to improve the model's training performance and generalization capabilities.

[0098] Data normalization is the process of mapping image pixel values ​​to a uniform range [0,1], ensuring that the minimum pixel value is 0, the maximum pixel value is 1, and other pixel values ​​are proportionally converted to decimals between 0 and 1. The normalization formula is as follows:

[0099]

[0100] Among them, X normalizedis the normalized pixel value, X is the original pixel value, and Max and Min represent the maximum and minimum pixel values ​​of the original image, respectively. Normalization allows pixel values ​​in different images to exhibit corresponding gradients, eliminating image differences caused by external factors such as lighting and modality, enabling the model to better learn the characteristics of facial images.

[0101] Data normalization is based on normalization, which transforms the pixel value range of each channel of the image from [0, 1] to [-1, 1] and makes it conform to the standard normal distribution with a specific mean and standard deviation. The formula for this data processing is expressed as:

[0102]

[0103] Among them, X standardized Represents the normalized pixel value, Mean represents the mean of the dataset, and Std represents the standard deviation of the dataset. The purpose of standardization is to speed up the convergence of the model and reduce oscillations. Through standardization, the image data is converted into a standard normal distribution with a mean of 0 and a variance of 1, which helps the model learn the statistical characteristics of the data more quickly and enables the model to focus more directly on the characteristics of the features during the learning process, thereby achieving faster convergence. In addition, standardization also helps to reduce the impact of areas with large local changes on the model, allowing the model to pay more attention to changes in the entire image. By removing the standard deviation of the image data, the model can reduce oscillations and increase sensitivity to artifacts or inconsistencies that may exist in deep fake faces. This standardization process enables the model to more accurately capture the details and features in deep fake faces, thereby improving overall detection performance and robustness.

[0104] As one of the embodiments, step S2: using the Xception model based on depthwise separable convolution and residual connection as the baseline network, improving the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introducing a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features, including:

[0105] A global and local multi-granularity feature fusion framework is built between the intermediate and exit streams of the Xception model. By performing segmentation operations on the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multiple levels of granularity, forming a multi-granularity structure with coarse-grained global feature branches and fine-grained local feature branches.

[0106] The fine-grained local feature branches are constructed and processed. The construction and processing process includes: dividing the facial feature map output from the Xception backbone network into four fine-grained local feature sub-blocks of equal size, and inputting the four feature sub-blocks into four local feature branches respectively; each fine-grained local feature branch is sequentially convolved through a residual-connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling. The feature map size is output to 1x1x2048 for input to the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0107] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the facial feature map is processed through the three-stage convolution of the Xception network's inlet, intermediate, and outlet streams, gradually extracting global features and abstracting and optimizing them into higher-level feature expressions; by performing an average pooling operation on the entire image or feature map, the feature values ​​on each feature channel are averaged to obtain the overall feature representation, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0108] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Among them S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0109] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0110] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the middle stream stage of the Xception network using residual connections;

[0111] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and spatial context information of two different dimensions is obtained. It is then passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is Mc (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multi-layer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0112] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, which is calculated as M s (F)=o(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0113] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0114] Determine the loss function: A loss function consisting of global feature classification loss and fine-grained local feature classification loss is used. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global = -[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 indicates the real face category, y=1 indicates the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the predicted probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0115] In this embodiment, specifically, the network structure of the present application is improved based on the Xception network, and a multi-granularity feature fusion framework of coarse-grained global feature branches and fine-grained local feature branches is constructed. After introducing the convolutional attention mechanism module, the final deep artificial intelligence face-changing detection network is obtained.

[0116] To address the lack of multi-scale and feature view information in traditional AI face-swap detection methods and to leverage the complementarity between global and local features, this application constructs a framework for fusion of global and local multi-granular features between the intermediate and exit streams of the Xception network. This framework divides the feature data into sub-regions or sub-blocks of different scales at multiple levels of granularity by segmenting the image or feature map data along its width and height.

[0117] The fine-grained features obtained by this segmentation method are not only the representation of local features in spatial position, but also include the representation of various features within the local area. These features can be extracted from the global semantic layer information through different calculation methods or operators. They are not only local spatial position information, but may also involve diverse information within the local area, such as color distribution, texture pattern, gradient change, etc. Therefore, each cut sub-block represents the layer information under a specific semantics in the entire image set, and each local layer emphasizes features at different levels, thus forming a logically defined local feature. Each sub-block can contain rich information about the semantic feature, and these features can be further analyzed and utilized in subsequent processing.

[0118] Meanwhile, the coarse global feature branch is used to extract overall attributes and contextual information. Finally, by fusing local and global features, the content of the facial image can be described from a more comprehensive feature perspective. Multiple fine-grained local feature branches and coarse-grained global feature branches can have receptive fields of varying scales, allowing for better processing of facial features at different scales. This effectively addresses the feature loss problem in traditional methods and improves the accuracy and robustness of AI-powered face-swap detection tasks.

[0119] This framework can obtain a richer and more detailed image representation that includes local features and features at other semantic levels. This segmentation method can improve the expressiveness and flexibility of the model and help better understand the semantic information in the image.

[0120] The framework adopts a loss function consisting of global feature classification loss and fine-grained local feature classification loss, and both losses are calculated using cross entropy (CE) loss.

[0121] The global feature classification loss calculation formula can be expressed as:

[0122] L global = -(y log(p) + (1-y) log(1-p))

[0123] Among them, y is the true label (y=0 represents the real face category; y=1 represents the artificial intelligence face-changing category) and p is the predicted probability.

[0124] The calculation formula for fine-grained local feature classification loss can be expressed as:

[0125]

[0126] Where y is the true label, pi is the predicted probability of the i-th fine-grained local feature branch, and N = 4 is the number of fine-grained local feature branch classifiers. The loss weight of each fine-grained local feature classifier should be the same.

[0127] The final loss function is a combination of global feature classification loss and fine-grained local feature classification loss, and the formula is as follows:

[0128] Loss = αL global +(1-α)L local

[0129] Among them, α is the weight of the global feature classification loss.

[0130] In the fine-grained local feature branch, after the facial feature map is output from the Xcepition backbone network, it is divided into four fine-grained local feature sub-blocks of equal size. The four feature sub-blocks are input into the four local feature branches respectively. Each fine-grained local feature branch then passes through a residual-connected depthwise separable convolution block and two depthwise separable convolution layers for convolution operations, as well as global average pooling. The feature map size is then output as 1x1x2048 to facilitate dimensionality reduction classification input to the fully connected layer classifier, resulting in a predicted category (true / false) output value for each input face image.

[0131] In the global feature branch, the facial feature map undergoes three stages of convolution processing in the Xception network: inlet, intermediate, and outlet streams. Global features are gradually extracted and abstracted into higher-level feature representations. The global feature branch performs average pooling on the entire image or feature map, averaging the feature values ​​across each feature channel to obtain a holistic feature representation. This operation reduces the dimensionality of the image or feature map to a fixed size, allowing the network to focus on global feature information across the entire image, regardless of specific locations. Finally, a fully connected layer outputs a predicted face classification value.

[0132] In the stage of outputting the final prediction value, this study assigns different weight values ​​to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is as follows:

[0133]

[0134] Among them, S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively. The weight value is the same as the weight value of the classification loss.

[0135] In addition, the attention mechanism is a method that can focus on important areas in an image and suppress responses in irrelevant areas. The attention mechanism is essentially a resource allocation mechanism that can adjust the resource allocation method according to the importance of the focus target, so that more resources are concentrated on the focus target. In convolutional neural networks, these resources are weight parameters. By allocating more weight parameters to the focus target during model training, the model's feature extraction capability for the focus target can be enhanced. This application introduces the attention mechanism into the artificial intelligence face-changing detection and classification task, aiming to enhance the representation capability of the model. By introducing the attention mechanism, the model can improve its perception of key features in the focus target, such as detailed texture and facial structure, thereby improving the classification effect.

[0136] Specifically, this application uses a residual connection method to embed the Convolutional Block Attention Module (CBAM) between the last four depth-wise separable convolution blocks of the middle stream stage of the Xception network. Since the above four depth-wise separable convolution blocks belong to the second half of the main feature extraction layer, they are responsible for capturing more abstract and semantically rich high-order features in the image. Therefore, adding a convolutional attention module here can significantly enhance the model's ability to model high-level features of the image. The calculation method of CBAM can be divided into two stages. In the channel attention stage, the main focus is on what information in the input face image is divided into two categories: true and false. In order to effectively calculate the channel attention, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1 in size to obtain two different dimensions of spatial context information (two descriptors). They are then passed to a shared multi-layer perceptron (MLP) network, which first reduces the dimension through 1*1 convolution and then increases the dimension through 1*1 convolution. The output feature vectors are added element by element, and the weight of each channel of the feature map is normalized through the sigmoid activation function. Finally, the weight is multiplied with the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is:

[0137] M c =σ(MLP(AυgPool(F))+MLP(MaxPool(F)))

[0138] Where F is the input feature map, AvgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multi-layer perceptron, σ is the sigmoid function, and Mc is the channel attention map. In the spatial attention stage, the main focus is on the position information of the input face target. To calculate spatial attention, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map. The calculation formula is:

[0139] M s =σ(f7(AυgPool(F) T +MaxPool(F) T ))

[0140] Among them, F is the input feature map, AvgPool and MaxPool are average pooling and maximum pooling in the channel dimension, f7 is a 7x7 convolution layer, σ is the sigmoid function, and Ms is the spatial attention map.

[0141] Therefore, the convolutional attention mechanism module sequentially derives a 1D channel attention map Mc of size Cx1x1 and a 2D spatial attention map Ms of size 1xHxW. The process is:

[0142]

[0143] in, Formulate element-wise multiplication.

[0144] More specifically, construct Figure 2 The artificial intelligence face-changing detection network with global and local multi-granularity feature fusion shown in the figure: Based on the Xception baseline network, it fully utilizes the complementarity between global and local features to construct a multi-granularity feature fusion framework of coarse-grained global feature branches and fine-grained local feature branches. The framework consists of fine-grained local feature branches and global feature branches.

[0145] The fine-grained local feature branch performs multi-level granularity segmentation of feature map data at the boundary between the Xception network's feature extraction network backbone (intermediate stream) and the feature aggregation and classification portion (egress stream), that is, between the 11th and 12th depthwise separable convolutional blocks. Traditional local feature segmentation methods typically use convolution kernels or pooling layers of varying sizes and strides to extract and segment local features. Local feature segmentation methods that directly slice feature data from the spatial dimension generally use fixed-size sliding windows or fixed-ratio slicing. This paper adopts the latter approach, slicing the feature data in half in both the height and width directions, resulting in four fine-grained local feature sub-blocks and corresponding four local feature branches. These branches independently learn and extract feature representations associated with the corresponding local region sub-blocks.

[0146] Feature segmentation methods such as Figure 3 As shown in the figure, the fine-grained features obtained by this segmentation method are not only the representation of local features in spatial position, but also include the representation of various features in the local area. These features can be extracted from the global semantic layer information through different calculation methods or operators. They are not only local spatial position information, but may also involve diverse information in the local area such as color distribution, texture pattern, gradient change, etc. Therefore, each cut sub-block represents the layer information under a specific semantic in the entire image set, and each local layer emphasizes features at different levels, thus forming a logically defined local feature. Each sub-block can contain rich information about the semantic feature, and these features can be further analyzed and utilized in subsequent processing.

[0147] After the facial feature map is output from the Xcepition backbone network, the global features are divided into four equally sized fine-grained local feature sub-blocks and input into the four local feature branches. Each fine-grained local feature branch then passes through a residual-connected depthwise separable convolution block and two depthwise separable convolutional layers for convolution, followed by global average pooling. The feature map is then output as 1x1x2048, ready for input into the fully connected layer classifier for dimensionality reduction. This results in a predicted category (true / false) output for each input face image.

[0148] The structure of the global feature branch is as follows Figure 6 Specifically, the facial feature map undergoes a three-stage convolutional process: from the Xception inlet stream to the intermediate stream and then to the outlet stream. This process gradually extracts global features and abstracts and optimizes them into higher-level feature representations. The global feature branch performs average pooling on the entire image or feature map, averaging the feature values ​​across each feature channel to obtain a holistic feature representation. This operation reduces the dimensionality of the image or feature map to a fixed size, allowing the network to focus on global feature information across the entire image, regardless of specific locations. Finally, a fully connected layer outputs a predicted face classification value.

[0149] In addition, a convolutional attention module is introduced into the feature extraction network. The convolutional attention mechanism module combines the channel attention module and the spatial attention module, which can focus on important areas in the image and suppress responses in irrelevant areas. This application uses a residual connection to embed the convolutional attention module between the last four depth-separable convolution blocks in the middle stream stage of the Xception network. The embedding method is as follows: Figure 4 shown.

[0150] As one embodiment, step S3: training the artificial intelligence face-swap detection model based on the preprocessed data set includes:

[0151] 3.1 Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0152]

[0153] θ t =θ t-1 -ηυ t

[0154] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-swap detection model;

[0155] 3.2 Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement. The specific steps include:

[0156] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are grouped together. In the experimental dataset, the number of frames extracted from each real video is greater than the number of frames in the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0157] Sampling operation: The probability of random erasing operation is set to 50%, and n frames are sampled from the 20 frames of face images in each real video, and 1 frame is sampled from the 3 frames of images in each face-swapped video;

[0158] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-swapped images and frames of real face images to achieve a balance in the frequency of data enhancement processing between the true and false categories;

[0159] 3.3 Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The accuracy ACC calculation formula is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate,

[0160] In this embodiment, specifically, the SGDM optimizer is selected in step S3. SGDM is an optimization algorithm based on the gradient descent method and an improved version of the stochastic gradient descent algorithm SGD. It introduces the concept of momentum mechanism, which can accelerate the convergence of parameter updates and suppress oscillations, considering not only the current gradient but also the previously accumulated gradient. Momentum can be regarded as the inertia of parameter updates. It takes into account the direction and amplitude of the previous update and plays a certain role in the current update. This can help the optimizer converge faster, and better jump out of the local optimal point or the area with smaller gradient in the parameter space, and better search for the global optimal solution.

[0161] The parameter update formula of SGDM is as follows:

[0162]

[0163] θ t =θt-1 -υ t

[0164] Among them, v t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θt is the parameter value at the tth iteration; η is the learning rate; β is the momentum decay factor, which is generally 0.9. Represents the gradient of the loss function with respect to the parameters.

[0165] Furthermore, to achieve better training results, a lightweight version of the Face-Cutout method is proposed in step S3. First, the real video and the corresponding face-swapped videos are grouped together for data augmentation. In the experimental dataset, the number of frames extracted from each real video is greater than the number of frames in the face-swapped videos. Assuming that one real video can generate n corresponding AI face-swapped videos, we select the first frame from each of the real video and the n face-swapped videos to calculate the difference mask. This allows us to calculate n difference masks for each video.

[0166] Next, the probability of random erasure is set to 50%. This setting allows the network model to learn the overall image properties, such as facial contours, rather than just the incomplete face that has been erased. Therefore, it is not necessary to perform Face-Cutout on all frames in each video set; only a certain number of frames need to be processed. Then, n frames are extracted from the 20 facial images in each real video, and 1 frame is extracted from the 3 frames in each face-swapped video. Using the n masks obtained in the previous step, n frames of the AI-generated face-swapped images and n frames of real face images are processed accordingly to achieve a balance in the frequency of data augmentation processing between the real and fake categories.

[0167] Specifically, during the training process, the SGDM optimizer was selected, and the momentum decay parameter was set to 0.9, and the weight decay parameter was set to 1e-4. SGDM is an optimization algorithm based on the gradient descent method. It is an improved version of the stochastic gradient descent algorithm SGD. It introduces the concept of momentum mechanism, which can accelerate the convergence of parameter updates and suppress oscillations. It considers not only the current gradient, but also the previously accumulated gradient. Momentum can be regarded as the inertia of parameter updates. It takes into account the direction and amplitude of the previous update and plays a certain role in the current update. This can help the optimizer converge faster, and better escape from local optimal points or areas with small gradients in the parameter space, and better search for the global optimal solution. After training is completed, the obtained model is evaluated on the test set to determine its detection effect. The evaluation indicators are AUC and ACC.

[0168] Accuracy (ACC) is an indicator for evaluating the correctness of a classifier on the entire dataset. It refers to the proportion of all correctly predicted samples to all samples, that is:

[0169]

[0170] Where TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively. In deepfake face detection, accuracy measures the model's ability to correctly classify real and fake faces. A higher accuracy means the classifier model is more accurate at distinguishing between real and fake faces during prediction. The closer ACC is to 1, the more accurate the model. However, it is important to note that it does not reflect the detector's ability to distinguish between different categories (real or fake) or its sensitivity to different confidence thresholds.

[0171] AUC measures the classification accuracy of a binary classification model and can effectively measure the accuracy and performance of the model in distinguishing real faces from forged faces. The AUC value is generally defined as the area under the ROC curve. Under this definition, the AUC value can be obtained by integrating the curve. Its calculation formula is:

[0172] AUC = ∫TPR(FPR)dFPR

[0173] Among them, TPR and FPR are true positive rate and false positive rate, respectively.

[0174] As one embodiment, step S4: deploying the trained artificial intelligence face-swap detection model to perform face-swap detection on the input face video or image to obtain a face-swap recognition result includes:

[0175] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0176] Detection process: The user selects the face video or image to be detected and inputs it into the artificial intelligence face-changing detection model for detection;

[0177] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has undergone artificial intelligence face-changing processing. The detection system displays this result on the screen.

[0178] See also Figure 5 , is a schematic diagram of the structure of a face-changing detection device based on multi-granularity feature fusion provided by one embodiment of the present invention. The face-changing detection device based on multi-granularity feature fusion comprises:

[0179] Preprocessing module 1, used to preprocess the data set containing real face data and face-swapped data to obtain a preprocessed data set;

[0180] Model construction module 2 is used to use the Xception model based on depthwise separable convolution and residual connections as the baseline network, improve the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, build a multi-granularity feature fusion framework consisting of coarse-grained global feature branches and fine-grained local feature branches, and introduce a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize them, thereby building an AI face-swap detection model that integrates global and local multi-granularity features;

[0181] Model training module 3, used to train the artificial intelligence face-swap detection model based on the preprocessed data set;

[0182] The model deployment module 4 is used to deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

[0183] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0184] By preprocessing the dataset containing facial data to reduce computational complexity and improve training efficiency; then, the Xception model was constructed and optimized. By introducing the global-local multi-granularity feature fusion mechanism and the convolutional attention mechanism, the ability to focus on key areas in the image was enhanced, and the feature expression was refined and adaptively optimized in the spatial and channel dimensions, thereby constructing an artificial intelligence face-swap detection model; then, the improved artificial intelligence face-swap detection model was trained using the preprocessed dataset; finally, the trained artificial intelligence face-swap detection model was deployed into the detection system to achieve efficient face-swap recognition of input face videos or images. In the embodiment of the present invention, the multi-granularity feature fusion framework overcomes the insufficient feature expression caused by single-scale feature extraction by simultaneously extracting and utilizing coarse-grained global features and fine-grained local features, while the convolutional attention mechanism improves the model's attention to subtle and important features in the image, thereby significantly improving the robustness and accuracy of the deep classifier technology for visual artifacts within the video frame based on artificial intelligence face-changing. Through the combined effect of the above technical processes, it is ensured that the artificial intelligence face-changing detection model can not only maintain high accuracy in complex scenarios such as different lighting conditions, expression changes, and posture changes, but also effectively cope with different types of artificial intelligence face-changing technologies, providing a more reliable means of identity verification. Therefore, the embodiment of the present invention can effectively improve the robustness and accuracy of face-changing detection.

[0185] As an improvement to the above solution, the model building module is specifically used to:

[0186] A global and local multi-granularity feature fusion framework is built between the intermediate and exit streams of the Xception model. By performing segmentation operations on the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multiple levels of granularity, forming a multi-granularity structure with coarse-grained global feature branches and fine-grained local feature branches.

[0187] The fine-grained local feature branches are constructed and processed. The construction and processing process includes: dividing the facial feature map output from the Xception backbone network into four fine-grained local feature sub-blocks of equal size, and inputting the four feature sub-blocks into four local feature branches respectively; each fine-grained local feature branch is sequentially convolved through a residual-connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling. The feature map size is output to 1x1x2048 for input to the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0188] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the facial feature map is processed through the three-stage convolution of the Xception network's inlet, intermediate, and outlet streams, gradually extracting global features and abstracting and optimizing them into higher-level feature expressions; by performing an average pooling operation on the entire image or feature map, the feature values ​​on each feature channel are averaged to obtain the overall feature representation, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0189] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Among them S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0190] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0191] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the middle stream stage of the Xception network using residual connections;

[0192] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and spatial context information of two different dimensions is obtained. It is then passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is M c (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multi-layer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0193] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, which is calculated as M s (F)=o(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0194] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0195] Determine the loss function: A loss function consisting of global feature classification loss and fine-grained local feature classification loss is used. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global= -[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 indicates the real face category, y=1 indicates the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the predicted probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0196] As an improvement to the above solution, the model training module is specifically used to:

[0197] Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0198]

[0199] θ t =θ t-1 -ηυ t

[0200] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-swap detection model;

[0201] Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement, including:

[0202] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are grouped together. In the experimental dataset, the number of frames extracted from each real video is greater than the number of frames in the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0203] Sampling operation: The probability of random erasing operation is set to 50%, and n frames are sampled from the 20 frames of face images in each real video, and 1 frame is sampled from the 3 frames of images in each face-swapped video;

[0204] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-swapped images and frames of real face images to achieve a balance in the frequency of data enhancement processing between the true and false categories;

[0205] Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The accuracy ACC calculation formula is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is AUC = ∫0 1 TPR(FPR)dFPR, where TPR is the true positive rate, TPR FPR is the false positive rate,

[0206] As an improvement to the above solution, the model deployment module is specifically used to:

[0207] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0208] Detection process: The user selects the face video or image to be detected and inputs it into the artificial intelligence face-changing detection model for detection;

[0209] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has undergone artificial intelligence face-changing processing. The detection system displays this result on the screen.

[0210] See also Figure 6 , is a schematic diagram of a face-swap detection device based on multi-granularity feature fusion, provided in one embodiment of the present invention. This embodiment of the face-swap detection device based on multi-granularity feature fusion includes: a processor 100, a memory 101, and a computer program, such as a device program, stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer program, it implements the steps of each of the aforementioned method embodiments. Alternatively, when the processor 100 executes the computer program, it implements the functions of each module / unit in each of the aforementioned apparatus embodiments.

[0211] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the face-swap detection device based on multi-granularity feature fusion.

[0212] The face-changing detection device based on multi-granularity feature fusion can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The face-changing detection device based on multi-granularity feature fusion can include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the schematic diagram is merely an example of a face-changing detection device based on multi-granularity feature fusion and does not constitute a limitation on the face-changing detection device based on multi-granularity feature fusion. It can include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the face-changing detection device based on multi-granularity feature fusion can also include input and output devices, network access devices, buses, etc.

[0213] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the face-changing detection device based on multi-granularity feature fusion, and utilizes various interfaces and lines to connect various parts of the face-changing detection device based on multi-granularity feature fusion.

[0214] The memory can be used to store the computer program and / or module, and the processor realizes the various functions of the face-changing detection device based on multi-granularity feature fusion by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0215] Among them, if the module / unit integrated in the face-changing detection device based on multi-granularity feature fusion is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of each of the above-mentioned method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0216] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0217] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A face-changing detection method based on multi-granularity feature fusion, characterized in that: include: Step S1: preprocessing the dataset containing real face data and face-swapped data to obtain a preprocessed dataset; Step S2: Using the Xception model based on depthwise separable convolution and residual connection as the baseline network, the feature extraction layer is improved based on the global-local multi-granularity feature fusion mechanism. A multi-granularity feature fusion framework is constructed, which includes a coarse-grained global feature branch and a fine-grained local feature branch. The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize them, thereby constructing an AI face-swap detection model that integrates global and local multi-granularity features. Step S3: training the artificial intelligence face-swap detection model based on the preprocessed data set; Step S4: deploying the trained artificial intelligence face-swap detection model to perform face-swap detection on the input face video or image to obtain a face-swap recognition result; Among them, step S2: using the Xception model based on depthwise separable convolution and residual connection as the baseline network, improving the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introducing a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features, including: A global and local multi-granularity feature fusion framework is built between the intermediate and exit streams of the Xception model. By performing segmentation operations on the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multiple levels of granularity, forming a multi-granularity structure with coarse-grained global feature branches and fine-grained local feature branches. The fine-grained local feature branches are constructed and processed. The construction and processing process includes: dividing the facial feature map output from the Xception backbone network into four fine-grained local feature sub-blocks of equal size, and inputting the four feature sub-blocks into four local feature branches respectively; each fine-grained local feature branch is sequentially convolved through a residual-connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling. The feature map size is output to 1x1x2048 for input to the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained; The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the facial feature map is processed through the three-stage convolution of the Xception network's inlet, intermediate, and outlet streams, gradually extracting global features and abstracting and optimizing them into higher-level feature expressions; by performing an average pooling operation on the entire image or feature map, the feature values ​​on each feature channel are averaged to obtain the overall feature representation, and finally the face authenticity classification prediction value is output through the fully connected layer; Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: ,in represents the global feature branch decision score, Indicates the fine-grained feature branch decision scores, and They represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively.

2. The face-changing detection method based on multi-granularity feature fusion according to claim 1, characterized in that: Step S2: Using the Xception model based on depthwise separable convolution and residual connection as the baseline network, improving the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introducing a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-swap detection model that integrates global and local multi-granularity features, and also includes: The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes: The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the middle stream stage of the Xception network using residual connections; In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers, and then compress the feature map from becomes The size of the feature map is obtained by multiplying the spatial context information of two different dimensions, and then passing it to a shared multi-layer perceptron network. The dimensionality is first reduced by 11 convolutions and then increased by 11 convolutions. The output feature vectors are added element by element, and the weight of each channel of the feature map is normalized by the sigmoid activation function. Finally, the weight is multiplied by the input feature map to produce The channel attention feature map of size is calculated as follows: ,in is the input feature map, and It performs average pooling and maximum pooling in the spatial dimension. is a multi-layer perceptron, is the sigmoid function, is the channel attention map; In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map. The calculation formula is ,in is the input feature map, and It performs average pooling and maximum pooling in the channel dimension. It is a 7x7 convolutional layer. is the sigmoid function, is the spatial attention map; The convolutional attention mechanism module sequentially derives a 1D channel attention map of size and a 2D channel attention map of size Spatial attention map , the process is ,in Express element-wise multiplication; Determine the loss function: A loss function consisting of global feature classification loss and fine-grained local feature classification loss is used. Both losses are calculated using cross entropy loss. The global feature classification loss is calculated as ,in is the true label, Indicates the real face category, Indicates the AI ​​face-changing category; is the prediction probability; the calculation formula for fine-grained local feature classification loss is ,in is the true label, It is The prediction probability of a fine-grained local feature branch, is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is ,in is the weight of the global feature classification loss.

3. The face-changing detection method based on multi-granularity feature fusion according to claim 1, characterized in that: The step S3: training the artificial intelligence face-swap detection model based on the preprocessed data set, including: 3.1 Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows: ； in, is The first-order momentum at the iteration, which is the exponential moving average of the current gradient; It is Parameter values ​​of the AI ​​face-changing detection model at the iteration; is the learning rate; is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-swap detection model; 3.2 Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement. The specific steps include: Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are grouped together. In the experimental dataset, the number of frames extracted from each real video is greater than the number of frames in the face-swapped video. The first frame of each face-changing video is selected to calculate the difference mask, so that a set of videos can be calculated difference masks; Sampling operation: The probability of random erasing operation is set to 50%, and 20 frames of face images are extracted from each real video. Frames, and extract 1 frame from the 3 frames of each face-swapped video; Balance processing: Using the previous step Mask, corresponding processing Frames of AI-powered face-swapped images and frames of real face images are used to balance the frequency of data enhancement processing for true and false categories; 3.3 Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The accuracy ACC calculation formula is: ,in are the number of true positive, true negative, false positive, and false negative samples respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is ,in is the true positive rate, ; is the false positive rate, .

4. The face-changing detection method based on multi-granularity feature fusion according to claim 1, characterized in that: The step S4: deploying the trained artificial intelligence face-swap detection model to perform face-swap detection on the input face video or image to obtain a face-swap recognition result, including: Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system. Detection process: The user selects the face video or image to be detected and inputs it into the artificial intelligence face-changing detection model for detection; Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has undergone artificial intelligence face-changing processing. The detection system displays this result on the screen.

5. A face-changing detection device based on multi-granularity feature fusion, characterized in that: include: A preprocessing module is used to preprocess the data set containing real face data and face-swapped data to obtain a preprocessed data set; A model building module uses the Xception model based on depthwise separable convolutions and residual connections as the baseline network. It improves the feature extraction layer based on a global-local multi-granularity feature fusion mechanism, constructs a multi-granularity feature fusion framework consisting of a coarse-grained global feature branch and a fine-grained local feature branch, and introduces a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize them, thereby building an AI face-swap detection model that integrates global and local multi-granularity features. A model training module, used to train the artificial intelligence face-swap detection model based on the preprocessed data set; A model deployment module is used to deploy the trained artificial intelligence face-swap detection model, perform face-swap detection on input face videos or images, and obtain face-swap recognition results; The model building module is specifically used to: A global and local multi-granularity feature fusion framework is built between the intermediate and exit streams of the Xception model. By performing segmentation operations on the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multiple levels of granularity, forming a multi-granularity structure with coarse-grained global feature branches and fine-grained local feature branches. The fine-grained local feature branches are constructed and processed. The construction and processing process includes: dividing the facial feature map output from the Xception backbone network into four fine-grained local feature sub-blocks of equal size, and inputting the four feature sub-blocks into four local feature branches respectively; each fine-grained local feature branch is sequentially convolved through a residual-connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling. The feature map size is output to 1x1x2048 for input to the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained; The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the facial feature map is processed through the three-stage convolution of the Xception network's inlet, intermediate, and outlet streams, gradually extracting global features and abstracting and optimizing them into higher-level feature expressions; by performing an average pooling operation on the entire image or feature map, the feature values ​​on each feature channel are averaged to obtain the overall feature representation, and finally the face authenticity classification prediction value is output through the fully connected layer; Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: ,in represents the global feature branch decision score, Indicates the fine-grained feature branch decision scores, and They represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively.

6. The face-changing detection device based on multi-granularity feature fusion according to claim 5, characterized in that: The model building module is further specifically configured to: The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes: The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the middle stream stage of the Xception network using residual connections; In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers, and then compress the feature map from becomes The size of the feature map is obtained by multiplying the spatial context information of two different dimensions, and then passing it to a shared multi-layer perceptron network. The dimensionality is first reduced by 11 convolutions and then increased by 11 convolutions. The output feature vectors are added element by element, and the weight of each channel of the feature map is normalized by the sigmoid activation function. Finally, the weight is multiplied by the input feature map to produce The channel attention feature map of size is calculated as follows: ,in is the input feature map, and It performs average pooling and maximum pooling in the spatial dimension. is a multi-layer perceptron, is the sigmoid function, is the channel attention map; In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map. The calculation formula is ,in is the input feature map, and It performs average pooling and maximum pooling in the channel dimension. It is a 7x7 convolutional layer. is the sigmoid function, is the spatial attention map; Determine the loss function: A loss function consisting of global feature classification loss and fine-grained local feature classification loss is used. Both losses are calculated using cross entropy loss. The global feature classification loss is calculated as ,in is the true label, Indicates the real face category, Indicates the AI ​​face-changing category; is the prediction probability; the fine-grained local feature classification loss calculation formula is ,in is the true label, It is The prediction probability of a fine-grained local feature branch, is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is ,in is the weight of the global feature classification loss.

7. The face-changing detection device based on multi-granularity feature fusion according to claim 5, characterized in that: The model training module is specifically used for: Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows: ； in, is The first-order momentum at the iteration, which is the exponential moving average of the current gradient; It is Parameter values ​​of the AI ​​face-changing detection model at the iteration; is the learning rate; is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-swap detection model; Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement, including: Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are grouped together. In the experimental dataset, the number of frames extracted from each real video is greater than the number of frames in the face-swapped video. The first frame of each face-changing video is selected to calculate the difference mask, so that a set of videos can be calculated difference masks; Sampling operation: The probability of random erasing operation is set to 50%, and 20 frames of face images are extracted from each real video. Frames, and extract 1 frame from the 3 frames of each face-swapped video; Balance processing: Using the previous step Mask, corresponding processing Frames of AI-powered face-swapped images and frames of real face images are used to balance the frequency of data enhancement processing for true and false categories; Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The accuracy ACC calculation formula is: ,in are the number of true positive, true negative, false positive, and false negative samples respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is ,in is the true positive rate, ; is the false positive rate, .

8. The face-changing detection device based on multi-granularity feature fusion according to claim 5, characterized in that: The model deployment module is specifically used to: Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system. Detection process: The user selects the face video or image to be detected and inputs it into the artificial intelligence face-changing detection model for detection; Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has undergone artificial intelligence face-changing processing. The detection system displays this result on the screen.

9. A face-changing detection device based on multi-granularity feature fusion, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the face-changing detection method based on multi-granularity feature fusion as described in any one of claims 1 to 4.

Images