Face change detection method, device and equipment based on multi-granularity feature fusion

Abstract

The invention discloses a face change detection method, device and equipment based on multi-granularity feature fusion, and the method comprises the steps: carrying out the preprocessing of a data set containing real face data and face change data, and obtaining a preprocessed data set; an Xception model is used as a reference network, a feature extraction layer is improved based on a global-local multi-granularity feature fusion mechanism, a multi-granularity feature fusion framework including coarse-granularity global feature branches and fine-granularity local feature branches is constructed, a convolution attention mechanism is introduced to distribute attention in space and channel dimensions to refine features and perform adaptive optimization, and a multi-granularity feature fusion algorithm is constructed. Therefore, an artificial intelligence face change detection model based on global and local multi-granularity feature fusion is constructed; training an artificial intelligence face change detection model based on the preprocessed data set; and deploying the trained artificial intelligence face change detection model, and performing face change detection on an input face video or image to obtain a face change recognition result. According to the invention, the robustness and accuracy of face change detection can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a face-changing detection method, device and equipment based on multi-granularity feature fusion. Background Art

[0002] As a carrier of personal information, human face plays an important role in identity authentication and authorization in modern society. As the important role of human face in communication, security control, financial transactions, etc. in digital urban environments becomes increasingly prominent, the remarkable development of deep learning technology has brought severe challenges to the reliability and security of human face functions. In particular, the widespread application of artificial intelligence face-changing technology, based on many advanced methods such as image processing and deep learning such as generative adversarial network technology, replaces or tampered with the face in the video into the target face without visual defects, thereby generating realistic and high-quality artificial intelligence face-changing videos. Videos that have been replaced by artificial intelligence will lead to a large amount of false information and uncertainty, which makes society face greater security risks. Therefore, research on the effective detection and recognition of artificial intelligence face-changing videos has become crucial to maintaining the security and trust of the digital society.

[0003] AI face-swapping detection has gone through three stages of development, from methods based on traditional image processing and computer vision technologies, to methods based on machine learning technologies such as deep learning, and then to methods based on adversarial learning technologies such as generative adversarial networks. Methods based on traditional image processing and computer vision technologies will fail or be inaccurate when faced with complex scenes, such as different lighting conditions, expression changes, posture changes, and other high-quality AI face-swapping videos; methods based on machine learning technologies such as deep learning face insufficient training data, overfitting, and poor generalization; methods based on adversarial learning technologies such as generative adversarial networks also have problems such as unstable training, difficult evaluation, and vulnerability to adversarial sample attacks.

[0004] Therefore, how to improve the robustness and accuracy of deep classifier technology based on visual artifacts within the AI ​​face-changing video frame and be able to cope with different types of AI face-changing technologies has become the key to solving the above problems. Summary of the invention

[0005] The embodiments of the present invention provide a face-changing detection method, device and equipment based on multi-granularity feature fusion, which can effectively improve the robustness and accuracy of face-changing detection.

[0006] An embodiment of the present invention provides a face-changing detection method based on multi-granularity feature fusion, comprising:

[0007] Step S1: preprocessing the data set containing real face data and face-swapped data to obtain a preprocessed data set;

[0008] Step S2: Using the Xception model based on deep separable convolution and residual connection as the baseline network, the feature extraction layer is improved based on the global-local multi-granularity feature fusion mechanism, a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches is constructed, and the convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine the features and adaptively optimize them, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features;

[0009] Step S3: training the artificial intelligence face-changing detection model based on the preprocessed data set;

[0010] Step S4: deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

[0011] As an improvement of the above scheme, the step S2: using the Xception model based on deep separable convolution and residual connection as the baseline network, improving the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introducing a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features, including:

[0012] A global and local multi-granularity feature fusion framework is built between the intermediate stream and the outlet stream of the Xception model. By performing segmentation operations in the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multi-level granularity, forming a multi-granularity structure of coarse-grained global feature branches and fine-grained local feature branches.

[0013] The fine-grained local feature branch is constructed and processed. The construction and processing process includes: dividing the face feature map output from the Xception backbone network into 4 fine-grained local feature sub-blocks of the same size, and the 4 feature sub-blocks are respectively input into the 4 local feature branches; each fine-grained local feature branch is sequentially convolved through a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling, and the feature map size is output as 1x1x2048, so as to be input into the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0014] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the face feature map is processed by the three-stage convolution of the Xception network's inlet flow, intermediate flow, and outlet flow, and the global features are gradually extracted and abstracted and optimized into higher-level feature expressions; the whole image or feature map is averaged by performing an average pooling operation, the feature values ​​on each feature channel are averaged, the overall feature representation is obtained, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0015] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Where S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0016] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0017] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the intermediate flow stage of the Xception network using residual connections;

[0018] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and the spatial context information of two different dimensions is obtained. Then it is passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is M c (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multilayer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0019] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, calculated as M s (F) = σ(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0020] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0021] Determine the loss function: Use a loss function consisting of global feature classification loss and fine-grained local feature classification loss. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L globol =-[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 represents the real face category, y=1 represents the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the prediction probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0022] As an improvement of the above solution, the step S3: training the artificial intelligence face-changing detection model based on the preprocessed data set includes:

[0023] 3.1 Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0024]

[0025] θ t =θt-1 -ηυ t

[0026] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-changing detection model;

[0027] 3.2 Data enhancement processing: A lightweight version of the algorithm based on the Face-Cutout method is used for data enhancement. The specific steps include:

[0028] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are taken as a group. In the experimental data set, the number of frames extracted from each real video is greater than the number of frames of the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0029] Sampling operation: Set the probability of random erasing operation to 50%, extract n frames from the 20 frames of face images in each real video, and extract 1 frame from the 3 frames of images in each face-swapped video;

[0030] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-changing images and frames of real face images accordingly, so that the frequency of data enhancement processing for the true and false categories is balanced;

[0031] 3.3 Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The calculation formula of accuracy ACC is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate,

[0032] As an improvement of the above solution, the step S4: deploying the trained artificial intelligence face-changing detection model, performing face-changing detection on the input face video or image, and obtaining the face-changing recognition result, includes:

[0033] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0034] Detection process: The face video or image to be detected is selected by the user and input into the artificial intelligence face-changing detection model for detection;

[0035] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has been processed by artificial intelligence face-changing. The detection system displays this result on the screen.

[0036] Another embodiment of the present invention provides a face-changing detection device based on multi-granularity feature fusion, including:

[0037] A preprocessing module, used to preprocess the data set containing real face data and face-swapped data to obtain a preprocessed data set;

[0038] Model building module, which uses the Xception model based on deep separable convolution and residual connection as the baseline network, improves the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, builds a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introduces the convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby building an artificial intelligence face-changing detection model that integrates global and local multi-granularity features;

[0039] A model training module, used to train the artificial intelligence face-changing detection model based on the preprocessed data set;

[0040] The model deployment module is used to deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

[0041] As an improvement of the above solution, the model building module is specifically used for:

[0042] A global and local multi-granularity feature fusion framework is built between the intermediate stream and the outlet stream of the Xception model. By performing segmentation operations in the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multi-level granularity, forming a multi-granularity structure of coarse-grained global feature branches and fine-grained local feature branches.

[0043] The fine-grained local feature branch is constructed and processed. The construction and processing process includes: dividing the face feature map output from the Xception backbone network into 4 fine-grained local feature sub-blocks of the same size, and the 4 feature sub-blocks are respectively input into the 4 local feature branches; each fine-grained local feature branch is sequentially convolved through a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling, and the feature map size is output as 1x1x2048, so as to be input into the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0044] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the face feature map is processed by the three-stage convolution of the Xception network's inlet flow, intermediate flow, and outlet flow, and the global features are gradually extracted and abstracted and optimized into higher-level feature expressions; the whole image or feature map is averaged by performing an average pooling operation, the feature values ​​on each feature channel are averaged, the overall feature representation is obtained, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0045] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Where S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0046] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0047] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the intermediate flow stage of the Xception network using residual connections;

[0048] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and the spatial context information of two different dimensions is obtained. Then it is passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is Mc (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multilayer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0049] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, calculated as M s (F) = σ(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0050] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0051] Determine the loss function: Use a loss function consisting of global feature classification loss and fine-grained local feature classification loss. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global =-[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 represents the real face category, y=1 represents the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the prediction probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0052] As an improvement of the above solution, the model training module is specifically used for:

[0053] Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0054]

[0055] θ t =θ t-1 -ηυ t

[0056] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-changing detection model;

[0057] Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement, including:

[0058] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are taken as a group. In the experimental data set, the number of frames extracted from each real video is greater than the number of frames of the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0059] Sampling operation: Set the probability of random erasing operation to 50%, extract n frames from the 20 frames of face images in each real video, and extract 1 frame from the 3 frames of images in each face-swapped video;

[0060] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-changing images and frames of real face images accordingly, so that the frequency of data enhancement processing for the true and false categories is balanced;

[0061] Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The calculation formula of accuracy ACC is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate,

[0062] As an improvement of the above solution, the model deployment module is specifically used for:

[0063] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0064] Detection process: The face video or image to be detected is selected by the user and input into the artificial intelligence face-changing detection model for detection;

[0065] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has been processed by artificial intelligence face-changing. The detection system displays this result on the screen.

[0066] Another embodiment of the present invention provides a face-changing detection device based on multi-granularity feature fusion, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the face-changing detection method based on multi-granularity feature fusion described in the above-mentioned embodiment of the invention is implemented.

[0067] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0068] The dataset containing facial data is preprocessed to reduce computational complexity and improve training efficiency. Subsequently, the Xception model is constructed and optimized. By introducing the global-local multi-granularity feature fusion mechanism and the convolutional attention mechanism, the ability to focus on key areas in the image is enhanced, and the feature expression is refined and adaptively optimized in the spatial and channel dimensions, thereby constructing an artificial intelligence face-changing detection model. The preprocessed dataset is then used to train the improved artificial intelligence face-changing detection model. Finally, the trained artificial intelligence face-changing detection model is deployed into the detection system to achieve efficient face-changing recognition of input face videos or images. In the embodiment of the present invention, the multi-granularity feature fusion framework overcomes the insufficient feature expression caused by the single-scale feature extraction by extracting and utilizing coarse-grained global features and fine-grained local features at the same time, and the convolutional attention mechanism improves the model's attention to subtle and important features in the image, thereby significantly improving the robustness and accuracy of the deep classifier technology based on the visual artifacts in the video frame of the artificial intelligence face-changing video. Through the joint action of the above technical processes, it is ensured that the artificial intelligence face-changing detection model can not only maintain a high accuracy rate under complex scenes such as different lighting conditions, expression changes, and posture changes, but also effectively cope with different types of artificial intelligence face-changing technologies, providing a more reliable means of identity authentication. Therefore, the embodiment of the present invention can effectively improve the robustness and accuracy of face-changing detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1It is a flowchart of a face-changing detection method based on multi-granularity feature fusion provided by an embodiment of the present invention;

[0070] Figure 2 This is a structural diagram of an artificial intelligence face-changing detection network that integrates global and local multi-granularity features provided by an embodiment of the present invention;

[0071] Figure 3 is a schematic diagram of a feature segmentation method provided by an embodiment of the present invention;

[0072] Figure 4 is a schematic diagram of introducing a convolutional attention module provided by an embodiment of the present invention;

[0073] Figure 5 It is a structural schematic diagram of a face-changing detection device based on multi-granularity feature fusion provided by an embodiment of the present invention;

[0074] Figure 6 It is a structural schematic diagram of a face-changing detection device based on multi-granularity feature fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] See also Figure 1 , is a flow chart of a face-changing detection method based on multi-granularity feature fusion provided by an embodiment of the present invention. The face-changing detection method based on multi-granularity feature fusion comprises the following steps:

[0077] Step S1: preprocessing the data set containing real face data and face-swapped data to obtain a preprocessed data set;

[0078] Step S2: Using the Xception model based on deep separable convolution and residual connection as the baseline network, the feature extraction layer is improved based on the global-local multi-granularity feature fusion mechanism, a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches is constructed, and the convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine the features and adaptively optimize them, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features;

[0079] Step S3: training the artificial intelligence face-changing detection model based on the preprocessed data set;

[0080] Step S4: deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

[0081] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0082] The dataset containing facial data is preprocessed to reduce computational complexity and improve training efficiency. Subsequently, the Xception model is constructed and optimized. By introducing the global-local multi-granularity feature fusion mechanism and the convolutional attention mechanism, the ability to focus on key areas in the image is enhanced, and the feature expression is refined and adaptively optimized in the spatial and channel dimensions, thereby constructing an artificial intelligence face-changing detection model. The preprocessed dataset is then used to train the improved artificial intelligence face-changing detection model. Finally, the trained artificial intelligence face-changing detection model is deployed into the detection system to achieve efficient face-changing recognition of input face videos or images. In the embodiment of the present invention, the multi-granularity feature fusion framework overcomes the insufficient feature expression caused by the single-scale feature extraction by extracting and utilizing coarse-grained global features and fine-grained local features at the same time, and the convolutional attention mechanism improves the model's attention to subtle and important features in the image, thereby significantly improving the robustness and accuracy of the deep classifier technology based on the visual artifacts in the video frame of the artificial intelligence face-changing video. Through the joint action of the above technical processes, it is ensured that the artificial intelligence face-changing detection model can not only maintain a high accuracy rate under complex scenes such as different lighting conditions, expression changes, and posture changes, but also effectively cope with different types of artificial intelligence face-changing technologies, providing a more reliable means of identity authentication. Therefore, the embodiment of the present invention can effectively improve the robustness and accuracy of face-changing detection.

[0083] As one of the embodiments, step S1, data set preprocessing: a series of data preprocessing operations are performed on the data set images to better focus on the key features and visual artifact details of the artificial intelligence face-changing data set, reduce the model calculation complexity, and improve the training efficiency. Specifically, the data set preprocessing described in step S1 can be performed in the following manner:

[0084] Given that the dataset used in AI face-swap detection tasks is generally in the format of video frame images, directly using the image data of the entire video dataset will result in a huge amount of data, and the time cost and model complexity required for training the model will also be high. Therefore, the dataset can be uniformly sampled so that the number of AI face-swap images and real face images in the training set and test set is the same, and divided into training set and test set in proportion.

[0085] In order to better highlight the key features and visual artifact details of the AI ​​face-changing dataset, reduce the computational complexity of the model, and improve training efficiency, the preprocessing operation of face positioning and cropping should be performed first to eliminate irrelevant parts such as the background and body, and only retain the image containing the face. The specific operation is: for each frame of the image, the face detector and alignment algorithm in the Dlib toolkit are used to detect 68 facial key points respectively, and then a rectangular frame is obtained based on the 51 key points of the eyebrows, eyes, nose, lips and other parts, and the obtained rectangular frame is enlarged 1.2 times for cropping.

[0086] However, in the AI ​​face-changing image dataset, there are still large variations in lighting conditions, angle positions, facial expressions, and modal differences. This causes the image to inevitably be disturbed by a certain amount of abnormal noise, which affects the classification and recognition effect of AI face-changing detection. Therefore, a series of data preprocessing operations are required after cropping the face. This includes normalizing the image size to convert these images into tensor form and input them into the network model for iterative training; performing data normalization and standardization to reduce computational complexity and speed up convergence; performing data enhancement through random horizontal flipping, random erasing, and edge filling to increase data diversity, improve the generalization ability of the model, and reduce dependence on specific positions and specific pixel values, making the model more robust.

[0087] Specifically, considering that the face images in the dataset may have problems such as background interference, different sizes, and small number of samples, a series of image preprocessing operations are required to be performed on the images, including face positioning and cropping, de-impurity, uniform size scaling, and richness expansion. This will facilitate better training results when the subsequent dataset is input into the network model, thereby improving model performance from the perspective of data processing.

[0088] Face localization and cropping is a key step in the preprocessing of deepfake face detection. This step uses a face detector to accurately locate the face area in the input image and crop it out to obtain an image containing only the face.

[0089] In the deep fake face detection task, the main goal of this study is to determine whether there is a deep fake face in the input image, and the face area is our focus, and other parts do not contribute to the task. By locating and cropping the face in the image, the background, body and other irrelevant areas in the image can be eliminated, thereby reducing the processing complexity of the model and improving the computational efficiency. At the same time, the cropped face area is more concentrated and prominent, which can not only highlight the key features and tiny visual artifacts of the face, but also guide the model's neural network to focus on the face. This helps the model better learn and distinguish the subtle differences between real faces and fake faces, thereby improving the accuracy of detection.

[0090] The specific steps of face positioning and cropping are as follows:

[0091] (1) Load the Dlib library face detector for detecting faces in images, and use the OpenCV library to read the dataset image to be processed.

[0092] (2) If no face is detected, skip the image; if multiple faces are detected, select the face with the largest area as the target face.

[0093] (3) Obtain the bounding box coordinates (upper left and lower right corner coordinates) of the target face and make fine adjustments as needed, such as increasing a certain proportion of the bounds to include more key information.

[0094] (4) According to the calculated face region coordinates, the face image is cropped from the original image. In this step, the cropped images with too small size can be screened out to improve the data quality.

[0095] (5) According to the label of the face data (real face or fake face), the cropped face images are classified and stored.

[0096] In addition, image size normalization is the operation of unifying the size of image data input into the deep learning model. Since the sizes of face images cropped from the dataset are usually different. The deep learning model has specific requirements for the input image size, and the image size needs to be uniformly converted before training. For the deep fake face detection task, the image data needs to be converted into a tensor data form so that it can be input into the network model for training or testing. Since the tensor input dimension of the network model is usually fixed, it is necessary to unify the image size.

[0097] In order to improve the training effect and generalization ability of the model, it is also necessary to perform data enhancement and data standardization operations. Data enhancement operations are aimed at increasing the diversity of the data set, including random horizontal flipping, random cropping after padding, and random erasing. Data standardization operations are aimed at transforming and normalizing image data to improve the training effect and generalization ability of the model.

[0098] Data normalization is the process of mapping image pixel values ​​to a uniform range [0,1], ensuring that the minimum pixel value is 0, the maximum pixel value is 1, and other pixel values ​​are proportionally converted to decimals between 0 and 1. The normalization formula is as follows:

[0099]

[0100] Among them, X normalizedis the normalized pixel value, X is the original pixel value, Max and Min represent the maximum and minimum pixel values ​​of the original image respectively. Through normalization, the pixel values ​​of different images show corresponding gradients, eliminating the image differences caused by external factors such as light and modality, so that the model can better learn the characteristics of face images.

[0101] Data standardization is based on normalization, which transforms the pixel value range of each channel of the image from [0, 1] to [-1, 1] and makes it conform to the standard normal distribution with a specific mean and standard deviation. The formula for this data processing is expressed as:

[0102]

[0103] Among them, X standardized Represents the pixel value after standardization, Mean represents the mean value of the data set, and Std represents the standard deviation of the data set. The purpose of standardization is to speed up the convergence of the model and reduce oscillations. Through standardization, the image data is converted into a standard normal distribution with a mean of 0 and a variance of 1, which helps the model learn the statistical characteristics of the data faster and enables the model to focus more directly on the characteristics of the features during the learning process, thereby achieving faster convergence. In addition, standardization also helps to reduce the impact of areas with large local changes on the model, allowing the model to pay more attention to changes in the entire image. By removing the standard deviation of the image data, the model can reduce oscillations and increase sensitivity to artifacts or inconsistencies that may exist in deep fake faces. This standardization process enables the model to more accurately capture the details and features in deep fake faces, thereby improving overall detection performance and robustness.

[0104] As one of the embodiments, step S2: using the Xception model based on deep separable convolution and residual connection as the baseline network, improving the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introducing a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features, including:

[0105] A global and local multi-granularity feature fusion framework is built between the intermediate stream and the outlet stream of the Xception model. By performing segmentation operations in the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multi-level granularity, forming a multi-granularity structure of coarse-grained global feature branches and fine-grained local feature branches.

[0106] The fine-grained local feature branch is constructed and processed. The construction and processing process includes: dividing the face feature map output from the Xception backbone network into 4 fine-grained local feature sub-blocks of the same size, and the 4 feature sub-blocks are respectively input into the 4 local feature branches; each fine-grained local feature branch is sequentially convolved through a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling, and the feature map size is output as 1x1x2048, so as to be input into the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0107] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the face feature map is processed by the three-stage convolution of the Xception network's inlet flow, intermediate flow, and outlet flow, and the global features are gradually extracted and abstracted and optimized into higher-level feature expressions; the whole image or feature map is averaged by performing an average pooling operation, the feature values ​​on each feature channel are averaged, the overall feature representation is obtained, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0108] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Where S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0109] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0110] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the intermediate flow stage of the Xception network using residual connections;

[0111] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and the spatial context information of two different dimensions is obtained. Then it is passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is Mc (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multilayer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0112] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, calculated as M s (F) = o(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0113] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0114] Determine the loss function: Use a loss function consisting of global feature classification loss and fine-grained local feature classification loss. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global =-[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 represents the real face category, y=1 represents the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the prediction probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0115] In this embodiment, specifically, the network structure of the present application is improved based on the Xception network, and a multi-granularity feature fusion framework of coarse-grained global feature branches and fine-grained local feature branches is constructed. After introducing the convolutional attention mechanism module, the final deep artificial intelligence face-changing detection network is obtained.

[0116] Among them, in order to solve the problem of lack of multi-scale and feature view information in the features of traditional artificial intelligence face-changing detection methods, and to make full use of the complementarity between global and local features, this application constructs a framework for the fusion of global and local multi-granularity features between the intermediate stream and the outlet stream of the Xception network. The framework divides the feature data into sub-regions or sub-blocks of different scales with multi-level granularity by segmenting the image or feature map data in the width and height directions.

[0117] The fine-grained features obtained by this segmentation method are not only the representation of local features in spatial position, but also the representation of various features in the local area. These features can be extracted from the global semantic layer information through different calculation methods or operators. They are not only local spatial position information, but may also involve diversified information in the local area such as color distribution, texture pattern, gradient change, etc. Therefore, each cut sub-block represents the layer information under a specific semantics in the entire image set, and each local layer emphasizes features at different levels, thus forming a logically defined local feature. Each sub-block can contain rich information about the semantic feature, and these features can be further analyzed and utilized in subsequent processing.

[0118] At the same time, the coarse global feature branch is used to extract overall attributes and contextual information. Finally, by fusing local features with global features, the content of the face image can be described from a more comprehensive feature perspective. Multiple fine-grained local feature branches and coarse-grained global feature branches can have receptive fields of different scales, thereby better processing face features of different scales, effectively solving the problem of feature missing in traditional methods, and improving the accuracy and robustness of AI face-changing detection tasks.

[0119] This framework can obtain a richer and more detailed image representation, which includes local features and features at other semantic levels. This segmentation method can improve the expressiveness and flexibility of the model and help better understand the semantic information in the image.

[0120] The framework adopts a loss function consisting of a global feature classification loss and a fine-grained local feature classification loss, and both losses are calculated using the cross entropy (CE) loss.

[0121] The global feature classification loss calculation formula can be expressed as:

[0122] L global = -(y log(p) + (1-y)log(1-p))

[0123] Among them, y is the true label (y=0 represents the real face category; y=1 represents the artificial intelligence face-changing category) and p is the predicted probability.

[0124] The calculation formula of fine-grained local feature classification loss can be expressed as:

[0125]

[0126] Where y is the true label, pi is the predicted probability of the i-th fine-grained local feature branch, and N = 4 is the number of fine-grained local feature branch classifiers. The loss weight of each fine-grained local feature classifier should be the same.

[0127] The final loss function is a combination of global feature classification loss and fine-grained local feature classification loss, and the formula is as follows:

[0128] Loss = αL global +(1-α)L local

[0129] Among them, α is the weight of the global feature classification loss.

[0130] In the fine-grained local feature branch, after the face feature map is output from the backbone network of Xcepition, it is divided into 4 fine-grained local feature sub-blocks of the same size, and the 4 feature sub-blocks are input into the 4 local feature branches respectively. Then, each fine-grained local feature branch is processed by a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and the convolution operation and global average pooling are processed to output the feature map size as 1x1x2048, so as to input the fully connected layer classifier for dimensionality reduction classification and obtain the predicted category (true / false) output value of each input face image.

[0131] In the global feature branch, the face feature map is processed by the three-stage convolution of the Xception network: the inlet flow, the middle flow, and the outlet flow. The global features are gradually extracted and abstracted and optimized into higher-level feature expressions. The global feature branch performs an average pooling operation on the entire image or feature map, and takes the average value of the feature values ​​on each feature channel to obtain the overall feature representation. This operation reduces the dimension of the image or feature map to a fixed size, allowing the network to focus on the global feature information of the entire image without being affected by the specific location. Finally, the face authenticity classification prediction value is output through the fully connected layer.

[0132] In the stage of outputting the final prediction value, this study assigns different weight values ​​to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is as follows:

[0133]

[0134] Among them, S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively. The weight value is the same as the weight value of the classification loss.

[0135] In addition, the attention mechanism is a method that can focus on important areas in an image and suppress responses in irrelevant areas. The attention mechanism is essentially a resource allocation mechanism that can adjust the resource allocation method according to the importance of the target of attention so that more resources are concentrated on the target of attention. In convolutional neural networks, these resources are weight parameters. By allocating more weight parameters to the target of attention during model training, the model's feature extraction capability for the target of attention can be enhanced. This application introduces the attention mechanism into the artificial intelligence face-changing detection and classification task, aiming to enhance the representation ability of the model. By introducing the attention mechanism, the model can enhance the perception of key features in the target of attention, such as detailed texture and facial structure, thereby improving the classification effect.

[0136] Specifically, the present application uses a residual connection to embed the Convolutional Block Attention Module (CBAM) between the last four depth-separable convolutional blocks in the middle flow stage of the Xception network. Since the above four depth-separable convolutional blocks belong to the second half of the main feature extraction layer, they are responsible for capturing more abstract and semantically rich high-order features in the image. Therefore, adding a convolutional attention module here can significantly enhance the model's ability to model high-level features of the image. The calculation method of CBAM can be divided into two stages. In the channel attention stage, the main focus is on what information in the input face image is divided into two categories: true and false. In order to effectively calculate the channel attention, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers, and based on two-dimensional compression, the feature map is changed from C×H×W to C×1×1 in size to obtain two different dimensions of spatial context information (two descriptors). Then they are passed to a shared multi-layer perceptron (MLP) network, first reduced in dimension by 1*1 convolution and then increased in dimension by 1*1 convolution, the output feature vectors are added element by element, the weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is:

[0137] M c =σ(MLP(AυgPool(F))+MLP(MaxPool(F)))

[0138] Among them, F is the input feature map, AvgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multi-layer perceptron, σ is the sigmoid function, and Mc is the channel attention map. In the spatial attention stage, the main focus is on the position information of the input face target. In order to calculate the spatial attention, the average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor, and finally the convolution operation is used to form the final spatial attention feature map. The calculation formula is:

[0139] M s =σ(f 7 (AυgPool(F) T +MaxPool(F) T ))

[0140] Among them, F is the input feature map, AvgPool and MaxPool are average pooling and maximum pooling in the channel dimension, f7 is a 7x7 convolution layer, σ is the sigmoid function, and Ms is the spatial attention map.

[0141] Therefore, the convolutional attention mechanism module sequentially derives a 1D channel attention map Mc of size Cx1x1 and a 2D spatial attention map Ms of size 1xHxW, the process is:

[0142]

[0143] in, Formulate element-wise multiplication.

[0144] More specifically, construct Figure 2 The artificial intelligence face-changing detection network with global and local multi-granularity feature fusion shown in the figure: Based on the Xception benchmark network, it makes full use of the complementarity between global and local features to construct a multi-granularity feature fusion framework of coarse-grained global feature branches and fine-grained local feature branches. The framework consists of fine-grained local feature branches and global feature branches.

[0145] The fine-grained local feature branch performs multi-level granularity segmentation of feature map data at the boundary between the feature extraction network backbone (intermediate flow) and the feature aggregation processing and classification part (exit flow) of the Xception network, that is, between the 11th depth-separable convolution block and the 12th depth-separable convolution block. Traditional local feature segmentation methods usually use convolution kernels or pooling layers of different sizes and steps to extract and divide local features. The local feature segmentation methods that directly cut feature data from the spatial dimension generally use two methods: a sliding window of a fixed size and a slice segmentation operation of a fixed ratio. This paper adopts the latter method, which performs bisection of feature data slice segmentation in the height and width directions, thereby obtaining four fine-grained local feature sub-blocks and corresponding four local feature branches. These branches will independently learn and extract feature representations related to the corresponding local area sub-blocks.

[0146] Feature segmentation methods such as Figure 3As shown in the figure, the fine-grained features obtained by this segmentation method are not only the representation of local features in spatial position, but also the representation of various features in the local area. These features can be extracted from the global semantic layer information through different calculation methods or operators. They are not only local spatial position information, but may also involve diversified information in the local area such as color distribution, texture pattern, gradient change, etc. Therefore, each cut sub-block represents the layer information under a specific semantics in the entire image set, and each local layer emphasizes features at different levels, thus forming a logically defined local feature. Each sub-block can contain rich information about the semantic feature, and these features can be further analyzed and utilized in subsequent processing.

[0147] After the face feature map is output from the backbone network of Xcepition, the global features are divided into 4 fine-grained local feature sub-blocks of the same size and input into 4 local feature branches. Then, each fine-grained local feature branch is processed by a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and global average pooling, and the feature map size is output as 1x1x2048, so as to facilitate the input of the fully connected layer classifier for dimensionality reduction classification, and obtain the predicted category (true / false) output value of each input face image.

[0148] The structure of the global feature branch is as follows Figure 6 As shown in the figure, specifically, the face feature map is processed by three stages of convolution from the Xception inlet flow, intermediate flow, and outlet flow, gradually extracting global features and abstracting and optimizing them into higher-level feature expressions. The global feature branch performs an average pooling operation on the entire image or feature map, taking the average of the feature values ​​on each feature channel to obtain the overall feature representation. This operation reduces the dimension of the image or feature map to a fixed size, allowing the network to focus on the global feature information of the entire image without being affected by the specific location. Finally, the face authenticity classification prediction value is output through the fully connected layer.

[0149] In addition, a convolutional attention module is introduced into the feature extraction network. The convolutional attention mechanism module combines the channel attention module and the spatial attention module, which can focus on important areas in the image and suppress responses in irrelevant areas. This application uses a residual connection to embed the convolutional attention module between the last four depth-separable convolutional blocks in the middle flow stage of the Xception network. The embedding method is as follows: Figure 4 shown.

[0150] As one of the embodiments, the step S3: training the artificial intelligence face-changing detection model based on the preprocessed data set includes:

[0151] 3.1 Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0152]

[0153] θ t =θ t-1 -ηυ t

[0154] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-changing detection model;

[0155] 3.2 Data enhancement processing: A lightweight version of the algorithm based on the Face-Cutout method is used for data enhancement. The specific steps include:

[0156] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are taken as a group. In the experimental data set, the number of frames extracted from each real video is greater than the number of frames of the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0157] Sampling operation: Set the probability of random erasing operation to 50%, extract n frames from the 20 frames of face images in each real video, and extract 1 frame from the 3 frames of images in each face-swapped video;

[0158] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-changing images and frames of real face images accordingly, so that the frequency of data enhancement processing for the true and false categories is balanced;

[0159] 3.3 Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The calculation formula of accuracy ACC is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate,

[0160] In this embodiment, specifically, the SGDM optimizer is selected in step S3. SGDM is an optimization algorithm based on the gradient descent method, and is an improved version of the stochastic gradient descent algorithm SGD. It introduces the concept of momentum mechanism, which can accelerate the convergence of parameter updates and suppress oscillations, considering not only the current gradient but also the previously accumulated gradient. Momentum can be regarded as the inertia of parameter updates, which takes into account the direction and amplitude of the previous update and plays a certain role in the current update. This can help the optimizer converge faster, and better jump out of the local optimal point or the area with smaller gradient in the parameter space, and better search for the global optimal solution.

[0161] The parameter update formula of SGDM is as follows:

[0162]

[0163] θ t =θ t-1 -υ t

[0164] Among them, v t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θt is the parameter value at the tth iteration; η is the learning rate; β is the momentum decay factor, which is generally 0.9. Represents the gradient of the loss function with respect to the parameters.

[0165] In addition, in order to achieve better training results, a lightweight version of the algorithm based on the Face-Cutout method is proposed in step S3. First, the real video and the corresponding multiple face-swapped videos are taken as a group for data enhancement processing. In the experimental data set, the number of frames extracted from each real video is greater than the number of frames of the face-swapped video. Assuming that a real video can generate n corresponding AI face-swapped videos, we select the first frame from the real video and the n face-swapped videos to calculate the difference mask. In this way, a group of videos can calculate n difference masks.

[0166] Next, the probability of random erasing is set to 50%. This setting is to allow the network model to learn the overall properties of the image, such as the contours of the face, rather than just the incomplete face that has been erased. Therefore, it is not necessary to perform Face-Cutout processing on all frames of each group of videos. Only a certain number of frames need to be selected for processing. Then, n frames are extracted from the 20 frames of face images in each real video, and 1 frame is extracted from the 3 frames of images in each face-swapped video. Using the n masks obtained in the previous step, n frames of artificial intelligence face-swapped images and n frames of real face images are processed accordingly to balance the frequency of data enhancement processing for true and false categories.

[0167] Specifically, during the training process, the SGDM optimizer is selected, and the momentum decay parameter is set to 0.9 and the weight decay parameter is set to 1e-4. SGDM is an optimization algorithm based on the gradient descent method. It is an improved version of the stochastic gradient descent algorithm SGD. It introduces the concept of momentum mechanism, which can accelerate the convergence of parameter updates and suppress oscillations, considering not only the current gradient but also the previously accumulated gradient. Momentum can be regarded as the inertia of parameter updates. It takes into account the direction and amplitude of the previous update and plays a certain role in the current update. This can help the optimizer converge faster, and better jump out of the local optimal point or the area with small gradient in the parameter space, and better search for the global optimal solution. After the training is completed, the obtained model is evaluated on the test set to determine its detection effect. The evaluation indicators are AUC and ACC.

[0168] Accuracy (ACC) is an indicator for evaluating the correctness of a classifier on the entire data set. It refers to the proportion of all correctly predicted samples to all samples, that is:

[0169]

[0170] Among them, TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively. In deep fake face detection, accuracy measures the model's ability to correctly classify real faces and fake faces. A higher accuracy means that the classifier model can more accurately distinguish between real and fake faces during the prediction process. The closer ACC is to 1, the more accurate the model is. However, it should be noted that it does not reflect the detector's ability to distinguish between different categories (real or fake), nor does it reflect the detector's sensitivity to different confidence thresholds.

[0171] AUC measures the classification accuracy of the binary classification model and can effectively measure the accuracy and performance of the model in distinguishing real faces from fake faces. The general definition of the AUC value is the area under the ROC curve. Under this definition, the AUC value can be obtained by integrating the curve. The calculation formula is:

[0172] AUC = ∫TPR(FPR)dFPR

[0173] Among them, TPR and FPR are true positive rate and false positive rate respectively.

[0174] As one of the embodiments, the step S4: deploying the trained artificial intelligence face-changing detection model, performing face-changing detection on the input face video or image, and obtaining the face-changing recognition result, includes:

[0175] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0176] Detection process: The face video or image to be detected is selected by the user and input into the artificial intelligence face-changing detection model for detection;

[0177] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has been processed by artificial intelligence face-changing. The detection system displays this result on the screen.

[0178] See also Figure 5 , is a schematic diagram of a face-changing detection device based on multi-granularity feature fusion provided by an embodiment of the present invention. The face-changing detection device based on multi-granularity feature fusion comprises:

[0179] Preprocessing module 1, used to preprocess the data set containing real face data and face-swapped data to obtain a preprocessed data set;

[0180] Model building module 2 is used to use the Xception model based on deep separable convolution and residual connection as the baseline network, improve the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, build a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introduce a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby building an artificial intelligence face-changing detection model that integrates global and local multi-granularity features;

[0181] Model training module 3, used for training the artificial intelligence face-changing detection model based on the preprocessed data set;

[0182] The model deployment module 4 is used to deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

[0183] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0184] The dataset containing facial data is preprocessed to reduce computational complexity and improve training efficiency. Subsequently, the Xception model is constructed and optimized. By introducing the global-local multi-granularity feature fusion mechanism and the convolutional attention mechanism, the ability to focus on key areas in the image is enhanced, and the feature expression is refined and adaptively optimized in the spatial and channel dimensions, thereby constructing an artificial intelligence face-changing detection model. The preprocessed dataset is then used to train the improved artificial intelligence face-changing detection model. Finally, the trained artificial intelligence face-changing detection model is deployed into the detection system to achieve efficient face-changing recognition of input face videos or images. In the embodiment of the present invention, the multi-granularity feature fusion framework overcomes the insufficient feature expression caused by the single-scale feature extraction by extracting and utilizing coarse-grained global features and fine-grained local features at the same time, and the convolutional attention mechanism improves the model's attention to subtle and important features in the image, thereby significantly improving the robustness and accuracy of the deep classifier technology based on the visual artifacts in the video frame of the artificial intelligence face-changing video. Through the joint action of the above technical processes, it is ensured that the artificial intelligence face-changing detection model can not only maintain a high accuracy rate under complex scenes such as different lighting conditions, expression changes, and posture changes, but also effectively cope with different types of artificial intelligence face-changing technologies, providing a more reliable means of identity authentication. Therefore, the embodiment of the present invention can effectively improve the robustness and accuracy of face-changing detection.

[0185] As an improvement of the above solution, the model building module is specifically used for:

[0186] A global and local multi-granularity feature fusion framework is built between the intermediate stream and the outlet stream of the Xception model. By performing segmentation operations in the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multi-level granularity, forming a multi-granularity structure of coarse-grained global feature branches and fine-grained local feature branches.

[0187] The fine-grained local feature branch is constructed and processed. The construction and processing process includes: dividing the face feature map output from the Xception backbone network into 4 fine-grained local feature sub-blocks of the same size, and the 4 feature sub-blocks are respectively input into the 4 local feature branches; each fine-grained local feature branch is sequentially convolved through a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling, and the feature map size is output as 1x1x2048, so as to be input into the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained;

[0188] The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the face feature map is processed by the three-stage convolution of the Xception network's inlet flow, intermediate flow, and outlet flow, and the global features are gradually extracted and abstracted and optimized into higher-level feature expressions; the whole image or feature map is averaged by performing an average pooling operation, the feature values ​​on each feature channel are averaged, the overall feature representation is obtained, and finally the face authenticity classification prediction value is output through the fully connected layer;

[0189] Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Where S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively;

[0190] The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes:

[0191] The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the intermediate flow stage of the Xception network using residual connections;

[0192] In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and the spatial context information of two different dimensions is obtained. Then it is passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is M c (F) = σ(MLP(AυgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AυgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multilayer perceptron, σ is the sigmoid function, and M c is the channel attention map;

[0193] In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, calculated as M s (F) = o(f 7×7 ([AυgPool c (F); MaxPool c (F)])), where F is the input feature map, AυgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7x7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map;

[0194] The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication;

[0195] Determine the loss function: Use a loss function consisting of global feature classification loss and fine-grained local feature classification loss. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global =-[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 represents the real face category, y=1 represents the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the prediction probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

[0196] As an improvement of the above solution, the model training module is specifically used for:

[0197] Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows:

[0198]

[0199] θ t =θ t-1 -ηυ t

[0200] Among them, t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-changing detection model;

[0201] Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement, including:

[0202] Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are taken as a group. In the experimental data set, the number of frames extracted from each real video is greater than the number of frames of the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos.

[0203] Sampling operation: Set the probability of random erasing operation to 50%, extract n frames from the 20 frames of face images in each real video, and extract 1 frame from the 3 frames of images in each face-swapped video;

[0204] Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-changing images and frames of real face images accordingly, so that the frequency of data enhancement processing for the true and false categories is balanced;

[0205] Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The calculation formula of accuracy ACC is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is AUC = ∫ 0 1 TPR(FPR)dFPR, where TPR is the true positive rate, TPR FPR is the false positive rate,

[0206] As an improvement of the above solution, the model deployment module is specifically used for:

[0207] Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system.

[0208] Detection process: The face video or image to be detected is selected by the user and input into the artificial intelligence face-changing detection model for detection;

[0209] Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has been processed by artificial intelligence face-changing. The detection system displays this result on the screen.

[0210] See also Figure 6 , is a schematic diagram of a face-changing detection device based on multi-granularity feature fusion provided by an embodiment of the present invention. The face-changing detection device based on multi-granularity feature fusion of this embodiment includes: a processor 100, a memory 101, and a computer program stored in the memory 101 and executable on the processor 100, such as a device program. When the processor 100 executes the computer program, the steps in the above-mentioned various method embodiments are implemented. Alternatively, when the processor 100 executes the computer program, the functions of each module / unit in the above-mentioned various device embodiments are implemented.

[0211] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the face-changing detection device based on multi-granularity feature fusion.

[0212] The face-changing detection device based on multi-granularity feature fusion can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The face-changing detection device based on multi-granularity feature fusion may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the schematic diagram is only an example of a face-changing detection device based on multi-granularity feature fusion, and does not constitute a limitation on the face-changing detection device based on multi-granularity feature fusion. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the face-changing detection device based on multi-granularity feature fusion may also include input and output devices, network access devices, buses, etc.

[0213] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the face-changing detection device based on multi-granularity feature fusion, and uses various interfaces and lines to connect the various parts of the face-changing detection device based on multi-granularity feature fusion.

[0214] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the face-changing detection device based on multi-granularity feature fusion by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0215] Wherein, if the module / unit integrated in the face-changing detection device based on multi-granularity feature fusion is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0216] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0217] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A face-changing detection method based on multi-granularity feature fusion, characterized in that: include: Step S1: preprocessing the data set containing real face data and face-swapped data to obtain a preprocessed data set; Step S2: Using the Xception model based on deep separable convolution and residual connection as the baseline network, the feature extraction layer is improved based on the global-local multi-granularity feature fusion mechanism, a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches is constructed, and the convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine the features and adaptively optimize them, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features; Step S3: training the artificial intelligence face-changing detection model based on the preprocessed data set; Step S4: deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

2. The face-changing detection method based on multi-granularity feature fusion as claimed in claim 1, characterized in that: The step S2: using the Xception model based on deep separable convolution and residual connection as the baseline network, improving the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, constructing a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introducing a convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby constructing an artificial intelligence face-changing detection model that integrates global and local multi-granularity features, including: A global and local multi-granularity feature fusion framework is built between the intermediate stream and the outlet stream of the Xception model. By performing segmentation operations in the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multi-level granularity, forming a multi-granularity structure of coarse-grained global feature branches and fine-grained local feature branches. The fine-grained local feature branch is constructed and processed. The construction and processing process includes: dividing the face feature map output from the Xception backbone network into 4 fine-grained local feature sub-blocks of the same size, and the 4 feature sub-blocks are respectively input into the 4 local feature branches; each fine-grained local feature branch is sequentially convolved through a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling, and the feature map size is output as 1x1x2048, so as to be input into the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained; The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the face feature map is processed by the three-stage convolution of the Xception network's inlet flow, intermediate flow, and outlet flow, and the global features are gradually extracted and abstracted and optimized into higher-level feature expressions; the whole image or feature map is averaged by performing an average pooling operation, the feature values ​​on each feature channel are averaged, the overall feature representation is obtained, and finally the face authenticity classification prediction value is output through the fully connected layer; Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Where S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively; The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes: The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the intermediate flow stage of the Xception network using residual connections; In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and the spatial context information of two different dimensions is obtained. Then it is passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is M c (F) = σ(MLP(AvgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AvgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multilayer perceptron, σ is the sigmoid function, and M c is the channel attention map; In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, which is calculated as M s (F) = σ(f 7×7 ([AvgPool c (F); MaxPool c (F)])), where F is the input feature map, AvgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7×7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map; The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication; Determine the loss function: Use a loss function consisting of global feature classification loss and fine-grained local feature classification loss. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global =-[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 represents the real face category, y=1 represents the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the prediction probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

3. The face-changing detection method based on multi-granularity feature fusion as claimed in claim 1, characterized in that: The step S3: training the artificial intelligence face-changing detection model based on the preprocessed data set, comprises: 3.1 Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows: i t =θ t-1 -nv t Among them, v t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-changing detection model; 3.2 Data enhancement processing: A lightweight version of the algorithm based on the Face-Cutout method is used for data enhancement. The specific steps include: Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are taken as a group. In the experimental data set, the number of frames extracted from each real video is greater than the number of frames of the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos. Sampling operation: Set the probability of random erasing operation to 50%, extract n frames from the 20 frames of face images in each real video, and extract 1 frame from the 3 frames of images in each face-swapped video; Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-changing images and frames of real face images accordingly, so that the frequency of data enhancement processing for the true and false categories is balanced; 3.3 Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The calculation formula of accuracy ACC is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate, 4. The face-changing detection method based on multi-granularity feature fusion as claimed in claim 1, characterized in that: The step S4: deploying the trained artificial intelligence face-changing detection model, performing face-changing detection on the input face video or image, and obtaining a face-changing recognition result, includes: Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system. Detection process: The face video or image to be detected is selected by the user and input into the artificial intelligence face-changing detection model for detection; Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has been processed by artificial intelligence face-changing. The detection system displays this result on the screen.

5. A face-changing detection device based on multi-granularity feature fusion, characterized in that: include: A preprocessing module, used to preprocess the data set containing real face data and face-swapped data to obtain a preprocessed data set; Model building module, which uses the Xception model based on deep separable convolution and residual connection as the baseline network, improves the feature extraction layer based on the global-local multi-granularity feature fusion mechanism, builds a multi-granularity feature fusion framework including coarse-grained global feature branches and fine-grained local feature branches, and introduces the convolutional attention mechanism to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize, thereby building an artificial intelligence face-changing detection model that integrates global and local multi-granularity features; A model training module, used to train the artificial intelligence face-changing detection model based on the preprocessed data set; The model deployment module is used to deploy the trained artificial intelligence face-changing detection model, perform face-changing detection on the input face video or image, and obtain the face-changing recognition result.

6. The face-changing detection device based on multi-granularity feature fusion as claimed in claim 5, characterized in that: The model building module is specifically used for: A global and local multi-granularity feature fusion framework is built between the intermediate stream and the outlet stream of the Xception model. By performing segmentation operations in the width and height directions of the image or feature map data, the feature data is divided into sub-regions or sub-blocks of different scales with multi-level granularity, forming a multi-granularity structure of coarse-grained global feature branches and fine-grained local feature branches. The fine-grained local feature branch is constructed and processed. The construction and processing process includes: dividing the face feature map output from the Xception backbone network into 4 fine-grained local feature sub-blocks of the same size, and the 4 feature sub-blocks are respectively input into the 4 local feature branches; each fine-grained local feature branch is sequentially convolved through a residual connected depthwise separable convolution block and two depthwise separable convolution layers, and then processed by global average pooling, and the feature map size is output as 1x1x2048, so as to be input into the fully connected layer classifier for dimensionality reduction classification, and the true / false prediction category output value of each input face image under the local feature branch is obtained; The coarse-grained global feature branch is constructed and processed. The construction and processing process includes: the face feature map is processed by the three-stage convolution of the Xception network's inlet flow, intermediate flow, and outlet flow, and the global features are gradually extracted and abstracted and optimized into higher-level feature expressions; the whole image or feature map is averaged by performing an average pooling operation, the feature values ​​on each feature channel are averaged, the overall feature representation is obtained, and finally the face authenticity classification prediction value is output through the fully connected layer; Perform decision fusion: At the stage of outputting the final prediction value, different weight values ​​are assigned to the classification prediction output results of the global feature branch and the fine-grained local feature branch for decision fusion. The calculation formula is: Where S global represents the global feature branch decision score, S i represents the decision score of the i-th fine-grained feature branch, α and (1-α) represent the weights of the decision scores of the global feature branch and the fine-grained local feature branch, respectively; The convolutional attention mechanism is introduced to allocate attention in the spatial and channel dimensions to refine features and adaptively optimize. The introduction process includes: The convolutional attention module is embedded between the last four depth-wise separable convolutional blocks in the intermediate flow stage of the Xception network using residual connections; In the channel attention stage, the spatial dimension of the input feature map is compressed, and the channel dimension remains unchanged. The specific operation is to first pass the input feature map through two parallel maximum pooling layers and average pooling layers. Based on the two-dimensional compression, the feature map is changed from C×H×W to C×1×1, and the spatial context information of two different dimensions is obtained. Then it is passed to a shared multi-layer perceptron network, first reduced in dimension by 11 convolutions and then increased in dimension by 11 convolutions, and the output feature vectors are added element by element. The weight of each channel of the feature map is normalized by the sigmoid activation function, and finally the weight is multiplied by the input feature map to generate a channel attention feature map of size C×H×W. The calculation formula is M c (F) = σ(MLP(AvgPool(F))+MLP(MaxPool(F))), where F is the input feature map, AvgPool and MaxPool are average pooling and maximum pooling in the spatial dimension, MLP is a multilayer perceptron, σ is the sigmoid function, and M c is the channel attention map; In the spatial attention stage, average pooling and maximum pooling operations are applied along the channel axis to obtain two 1×H×W feature maps, which are then concatenated to generate a valid feature descriptor. Finally, a convolution operation is used to form the final spatial attention feature map, which is calculated as M s (F) = σ(f 7×7 ([AvgPool c (F); MaxPool c (F)])), where F is the input feature map, AvgPool c and MaxPool c It is average pooling and maximum pooling in the channel dimension, f 7×7 is a 7×7 convolutional layer, σ is the sigmoid function, M s is the spatial attention map; The convolutional attention mechanism module sequentially derives a 1D channel attention map of size M and a 2D spatial attention map M of size 1×H×W. s , the process is in Express element-wise multiplication; Determine the loss function: Use a loss function consisting of global feature classification loss and fine-grained local feature classification loss. Both losses are calculated using cross entropy loss. The global feature classification loss calculation formula is L global =-[ylog(p)+(1-y)log(1-p)], where y is the true label, y=0 represents the real face category, y=1 represents the artificial intelligence face-changing category; p is the predicted probability; the calculation formula for fine-grained local feature classification loss is Where y is the true label, p i is the prediction probability of the i-th fine-grained local feature branch, N is the number of fine-grained local feature branch classifiers, and each fine-grained local feature classifier has the same loss weight; the final loss function is L = αL global +(1-α)L local , where α is the weight of the global feature classification loss.

7. The face-changing detection device based on multi-granularity feature fusion as claimed in claim 5, characterized in that: The model training module is specifically used for: Optimizer selection: Select the SGDM optimizer, and its parameter update formula is as follows: i t =θ t-1 -nv t Among them, v t is the first-order momentum at the tth iteration, that is, the exponential moving average of the current gradient; θ t is the parameter value of the artificial intelligence face-changing detection model at the tth iteration; η is the learning rate; β is the momentum decay factor; Represents the gradient of the loss function with respect to the parameters of the AI ​​face-changing detection model; Data enhancement processing: A lightweight version of the Face-Cutout method is used for data enhancement, including: Grouping and mask calculation: The real video and the corresponding multiple face-swapped videos are taken as a group. In the experimental data set, the number of frames extracted from each real video is greater than the number of frames of the face-swapped video. The first frame is selected from the real video and its corresponding n face-swapped videos to calculate the difference mask. In this way, n difference masks can be calculated for a group of videos. Sampling operation: Set the probability of random erasing operation to 50%, extract n frames from the 20 frames of face images in each real video, and extract 1 frame from the 3 frames of images in each face-swapped video; Balanced processing: Using the n masks obtained in the previous step, process n frames of AI face-changing images and frames of real face images accordingly, so that the frequency of data enhancement processing for the true and false categories is balanced; Model evaluation: After training, the AI ​​face-changing detection model is evaluated on the test set. The evaluation indicators are classification accuracy AUC and accuracy ACC. The calculation formula of accuracy ACC is: TP, TN, FP, and FN are the number of true positive, true negative, false positive, and false negative samples, respectively; AUC measures the classification accuracy of the binary classification model, and its value is the area under the ROC curve. The calculation formula is: Where TPR is the true positive rate, FPR is the false positive rate, 8. The face-changing detection device based on multi-granularity feature fusion as claimed in claim 5, characterized in that: The model deployment module is specifically used for: Deployment system: Deploy the trained AI face-swap detection model to a visual detection system consisting of computers, screens, and an early warning system. Detection process: The face video or image to be detected is selected by the user and input into the artificial intelligence face-changing detection model for detection; Result output and display: After the detection is completed, the artificial intelligence face-changing detection model outputs the face-changing recognition result, which indicates whether the input face video or image has been processed by artificial intelligence face-changing. The detection system displays this result on the screen.

9. A face-changing detection device based on multi-granularity feature fusion, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the face-changing detection method based on multi-granularity feature fusion as described in any one of claims 1 to 4.

Images