Distributed aigc forgery detection method based on deep fusion network

Abstract

The application discloses a kind of distributed AIGC forgery detection methods based on depth fusion network, the method includes the following steps: by terminal equipment preprocessing acquisition face image data training set and face image data test set;Based on cloud server light weight face target detection network and AIGC face authentication forgery model, and the light weight face target detection network that is constructed is transmitted to edge device setting;According to the face image data training set, AIGC forgery detection model is trained to AIGC forgery detection model to obtain AIGC forgery detection optimization model;According to AIGC forgery detection optimization model realizes to face image data test set the authenticity identification of the present application solves the problem that existing detection scheme has not considered the size of model, calculation consumption and inference efficiency when designing, leading to they far cannot satisfy the urgent needs of growing, fast detection face forgery content in real world.

Description

Technical Field

[0001] This invention relates to the field of AIGC forgery detection technology, and in particular to a distributed AIGC forgery detection method based on deep fusion networks. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence synthesis technology in application areas such as text-to-speech conversion, object generation, and object replacement, many security issues have inevitably arisen. Taking face synthesis as an example, artificial intelligence synthesis technology can easily generate highly realistic facial images and videos. However, if this technology is maliciously used to generate fake facial audio and video for false advertising, facial identity verification, etc., it will cause significant resource losses.

[0003] Due to the high real-time requirements and intensive use of computing resources, face recognition tasks have great application potential in cloud computing environments. However, when this task is deployed on a cloud platform, it may encounter a series of challenges, such as network bandwidth limitations and the instability of wireless connections. For traditional deepfake detection tasks, this results in low classification accuracy, making it impossible to effectively evaluate model performance. The accuracy of forgery detection is usually positively correlated with model complexity. However, traditional forgery detection methods do not take model size, computational cost, and inference efficiency into account in their design, making them far from meeting the growing real-world demand for rapid and accurate detection of face forgeries. Summary of the Invention

[0004] This invention provides a distributed AIGC forgery detection method based on deep fusion networks to overcome the above-mentioned technical problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A distributed AIGC forgery detection method based on deep fusion networks includes the following steps:

[0007] S1: Store the collected face image data in the terminal device for preprocessing to obtain a face image dataset, and randomly divide the face image dataset to obtain a face image data training set and a face image data test set;

[0008] S2: An AIGC forgery detection model is built based on a cloud server. The AIGC forgery detection model includes a lightweight face target detection network for detecting and acquiring face feature maps and an AIGC face authentication model for realizing forgery recognition based on face feature maps.

[0009] The completed lightweight face detection network is then transmitted to the edge device for configuration.

[0010] S3: Train the AIGC forgery detection model based on the face image data training set to obtain the AIGC forgery detection optimized model;

[0011] The model training includes: using the edge device to perform feature extraction training on the face image data training set to obtain face target feature maps based on a lightweight face target detection network;

[0012] The facial feature map is then transmitted to the cloud server.

[0013] The cloud server uses the AIGC facial authentication model to train the recognition of the authenticity of facial images based on the facial target feature map.

[0014] S4: The AIGC forgery detection optimization model is used to verify the authenticity of the face image data test set.

[0015] Furthermore, the lightweight face detection network constructed in S2 includes an input layer, an SSH detection module, and a feature pyramid network module;

[0016] The input layer is used to transmit sample images with real face bounding boxes to the SSH detection module;

[0017] The SSH detection module is connected to each convolutional neural network layer to obtain face enhancement sample images, and the feature pyramid network module includes several pre-set stacked convolutional neural network layers.

[0018] Furthermore, each convolutional neural network layer is used to extract facial features at multiple scales from the enhanced face sample image; the facial features include shape features and texture features;

[0019] Based on the extracted multi-scale facial features, face detection boxes of different scales are obtained based on regression boxes, and face targets of different sizes are obtained based on preset face feature anchor points of different sizes, so as to obtain face target feature maps.

[0020] Furthermore, the AIGC face authentication model constructed in S2 includes a CNN branch network, a Transformer branch network, and several Conv-AttributionMap convolutional attention map modules;

[0021] The CNN branch network is used to extract local features from the output of the lightweight face object detection network, and the CNN branch network includes a sequentially connected Conv-input input layer, three identical network convolutional branches, a Concat concatenation layer, a first convolutional layer, and a second convolutional layer.

[0022] The Conv-input layer is used to input the output of the lightweight face detection network into each network convolution branch for convolution operation, and the Concat concatenation layer concatenates the outputs of each network convolution branch to obtain local initial feature maps.

[0023] The first convolutional layer is used to perform a convolution operation on the local initial feature map to obtain a local intermediate feature map, and then transmit the local intermediate feature map to the second convolutional layer;

[0024] The second convolutional layer is used to perform convolution operations on the local intermediate feature maps to obtain the local final feature maps;

[0025] The Transformer branch network is used to fuse features based on the output of the lightweight face object detection network and the local features extracted by the CNN branch network to obtain a face feature fusion map.

[0026] The Transformer branch network includes a patch partitioning layer, a linear embedding layer, a first Swing Transformer block module, a second Swing Transformer block module, a third Swing Transformer block module, and a fourth Swing Transformer block module connected in sequence. A patch merging layer is connected between two adjacent Swing Transformer block modules. The patch merging layer includes a first patch merging layer, a second patch merging layer, and a third patch merging layer.

[0027] The patch partitioning layer is used to downsample the output of the lightweight face object detection network and segment it to obtain non-overlapping patch blocks to obtain patch feature maps.

[0028] The Linear Embedding layer is used to transform the patch feature map into a first-scale feature map of a set dimension, and transmit the first-scale feature map to the first Swing Transformer block module;

[0029] The first Swin Transformer block module is used to extract features from the first-scale feature map to obtain the first-scale feature vector;

[0030] The first Patch Merging layer and the Concat layer are connected sequentially by the first Conv-AttributionMap convolutional attention map module and the first patch Embedding layer.

[0031] Furthermore, the first Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local initial feature map and obtain an optimized local initial feature map;

[0032] The first patch Embedding layer is connected to the first Conv-Attribution Map convolutional attention map module, which is used to transform the optimized local initial feature map into a first local feature vector and transmit it to the first PatchMerging fusion layer;

[0033] The first Patch Merging fusion layer is used to fuse the first local feature vector with the first scale feature vector to obtain a second scale feature map; and to transmit the second scale feature map to the second SwinTransformer block module.

[0034] The second Swin Transformer block module is used to extract features from the second-scale feature map to obtain the second-scale feature vector;

[0035] The second Patch Merging fusion layer and the first convolutional layer are connected sequentially by the second Conv-AttributionMap convolutional attention map module and the second patch Embedding layer;

[0036] Furthermore, the second Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local intermediate feature map and obtain an optimized local intermediate feature map;

[0037] The second patch embedding layer is connected to the second conv-attribution map module, which is used to transform the optimized local intermediate feature map into a second local feature vector and transmit it to the second patch merging layer.

[0038] The second Patch Merging fusion layer is used to fuse the second local feature vector with the second scale feature vector to obtain a third scale feature map; and to transmit the third scale feature map to the third SwinTransformer block module;

[0039] The third Swin Transformer block module is used to extract features from the third-scale feature map and obtain the third-scale feature vector.

[0040] The third Patch Merging fusion layer and the second convolutional layer are connected sequentially by the third Conv-AttributionMap convolutional attention graph module and the third patch Embedding layer;

[0041] Furthermore, the third Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local final feature map and obtain an optimized local final feature map;

[0042] The third patch embedding layer is connected to the third Conv-Attribution Map convolutional attention map module, which is used to transform the optimized local final feature map into a third local feature vector and transmit it to the third patchMerging fusion layer.

[0043] The third Patch Merging fusion layer is used to fuse the third local feature vector with the third scale feature vector to obtain a face feature fusion map; and the face feature fusion map is transmitted to the fourth SwinTransformer block module for feature extraction so as to achieve AIGC face authentication results through the detector.

[0044] Furthermore, S3 specifically includes the following steps:

[0045] S31: Based on the face image data training set, sample images with real face target boxes are used as input data, and the true / false category labels of the sample images are used as output data, and are transmitted sequentially to the edge device and the cloud server.

[0046] S32: Based on the loss function of the constructed face target detection network and the AIGC face authentication loss function, the edge device performs feature extraction training on the face image data to obtain the face target feature map based on the lightweight face target detection network;

[0047] The facial feature map is then transmitted to the cloud server.

[0048] Furthermore, the cloud server uses the AIGC facial authentication model to train the authenticity recognition of facial images based on the facial target feature map.

[0049] To enable model training for the AIGC forgery detection model;

[0050] S33: If the output of the trained AIGC forgery detection model converges, then the trained AIGC forgery detection model at this time is the AIGC forgery detection optimization model;

[0051] Otherwise, adaptively adjust the network parameter weights of the lightweight face object detection network and the AIGC face authentication model, and repeat steps 31 to 32.

[0052] Furthermore, the loss function Loss of the lightweight face detection network constructed in S3 includes the additive angle and margin loss function L. arc α-IoU loss function and reconstruction loss function L of feature point regression model rec The weighted sum is as follows:

[0053] Loss=λ1L arc +λ2L IoU +λ3L rec

[0054] In the formula: L arc L represents the additive corner margin loss used to improve inter-class features in the feature space; IoU L represents the α-IoU loss used to optimize the bounding box prediction; rec λ1, λ2, and λ3 represent the reconstruction loss of the feature point regression model used to ensure accurate reconstruction of shape and texture; λ1, λ2, and λ3 represent the weight parameters of the loss function.

[0055] The additive corner margin loss function L arc The expression is

[0056]

[0057] In the formula: N represents the number of training samples; y i θ represents the true class label of sample i; j x represents i and W j The angle between them, and W j The weight vector j and x of the neural network are represented by these vectors. i The vector representing the facial target features of sample i; S represents the scaling parameter set; m represents the custom-added additive corner margin;

[0058] The expression for the α-IoU loss function is as follows:

[0059]

[0060] In the formula: The value of the loss function is represented by IoU; IoU represents the intersection-over-union ratio between the predicted bounding box and the actual bounding box; α represents the power parameter.

[0061] The reconstruction loss function L of the feature point regression model rec The expression is

[0062]

[0063] In the formula: θ D Indicate decoder The parameters, This represents a feature point regression decoder used to reconstruct the encoded features into a visual representation of face shape and texture; E M This represents a feature point regression encoder used to extract features of face shape and texture. Indicates encoder E M The parameters; S i Represents the shape of the original feature point regression; A i The texture represents the original feature point regression; ||·||2 represents the shape reconstruction error term calculated using Euclidean distance; ||·||1 represents the texture reconstruction error term calculated using absolute distance.

[0064] Furthermore, the constructed AIGC face authentication loss function adopts the cross-entropy loss function.

[0065] Furthermore, this also includes interactive updates of the model;

[0066] New facial image data is periodically obtained from the collected facial video data through terminal devices to generate and store new facial image datasets;

[0067] When the capacity of new face image data in the face image dataset recorded and stored by the terminal device reaches a preset capacity threshold, the terminal device sends a trigger update signal to the cloud server.

[0068] When the cloud server receives the trigger update signal, it starts the training process of the AIGC forgery detection model to update the AIGC forgery detection model, and distributes the trained and updated lightweight face target detection network to the edge device for online update, and stores the trained and updated AIGC face forgery detection model in the cloud service for online update.

[0069] Furthermore, the preprocessing described in S1 specifically includes:

[0070] We assign true / false category labels to each sample image of the face image data and perform compression processing. Then, we use regression boxes to locate and label the sample images of the face image dataset to obtain sample images with real face target boxes.

[0071] Beneficial Effects: This invention provides a distributed AIGC forgery detection method based on deep fusion networks. By constructing an AIGC forgery detection model based on a cloud computing platform, the use of cloud computing reduces network bandwidth consumption, achieving efficient resource utilization and system performance optimization. The AIGC forgery detection model includes a lightweight face detection network set up in an edge device and an AIGC face authentication model set up in a cloud server. The AIGC forgery detection model is optimized by constructing loss functions for the face detection network and the AIGC face authentication loss function, and by training the model using a face image data training set, thereby obtaining an optimized AIGC forgery detection model to achieve authenticity verification of a face image data test set. By dividing the AIGC forgery detection model into a lightweight face detection network and an AIGC face authentication model, and through collaboration and interaction between terminal devices, edge servers, and cloud servers, the local edge face detection model and the cloud-trained model library are dynamically updated. This allows for efficient resource utilization while improving the rapid detection and recognition of face forgeries through optimization of the AIGC forgery detection model. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 This is a flowchart of the distributed AIGC forgery detection method based on deep fusion networks according to the present invention;

[0074] Figure 2 This is a schematic diagram of the structural connections of the cloud computing platform in this embodiment;

[0075] Figure 3 This is a schematic diagram of the AIGC forgery detection model structure in this embodiment;

[0076] Figure 4 This is a schematic diagram of the lightweight face detection network structure in this embodiment;

[0077] Figure 5 This is the robust dataset used in the robustness experiment of the AIGC forgery detection method in this embodiment;

[0078] Figure 6 This is a simulation diagram of the robustness test results of the AIGC forgery detection method in this embodiment.

[0079] Figure 7This is a schematic diagram of the abnormal attention region of the AIGC face in this embodiment. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0081] This embodiment provides a distributed AIGC forgery detection method based on deep fusion networks, such as... Figure 1 As shown, it includes the following steps:

[0082] S1: The collected face image data is stored in the terminal device for preprocessing to obtain a face image dataset. The face image dataset is then randomly divided into a face image data training set and a face image data test set. The preprocessing specifically involves setting true / false category labels for each sample image of the face image data and performing compression processing. Regression boxes are used to locate and mark the sample images of the face image dataset to obtain sample images with real face target boxes.

[0083] Specifically, the terminal devices on the cloud computing platform apply JPEG2000 compression encoding technology to intelligently compress video data pre-captured by cameras, and upload the compressed data to the edge server. This prioritizes high-quality representation of key information while effectively reducing the volume of non-critical data, improving transmission efficiency. Furthermore, multiple face image datasets are extracted from the compressed video data stored on the edge server. These terminal devices are widely used media data tools, including real-time video and image data uploading and browsing. However, their limited computing power makes them unsuitable for computationally intensive operations, such as direct object detection. This is especially true considering their frequent reliance on bandwidth-constrained and often unstable wireless connections to interface with the edge server. The terminal devices primarily undertake two tasks. Firstly, they act as frontline data acquisition agents, capturing real-time visual data from their deployment environment. Secondly, given their frequent reliance on bandwidth-constrained and often unstable wireless connections, these devices prioritize data efficiency. Before transmitting the captured images to the edge server, the terminal devices employ compression technology to minimize transmission costs and ensure timely data delivery. This approach not only saves bandwidth but also reduces potential latency caused by data transmission between the terminal device and its associated edge or cloud server.

[0084] S2: The cloud computing platform includes terminal devices, edge devices, and a cloud server that are sequentially connected in communication; such as Figure 3 As shown, an AIGC forgery detection model is built on a cloud server based on a cloud computing platform. The AIGC forgery detection model includes a lightweight face target detection network for detecting and acquiring face feature maps and an AIGC face authentication model for realizing forgery recognition based on face feature maps. The completed lightweight face target detection network is then transmitted to an edge device for configuration.

[0085] Specifically, edge servers are located at the edge of the internet and typically act as local servers or base stations. Their main function is to perform object detection on compressed images transmitted from terminal devices, providing real-time detection services. Most captured images are processed directly on these edge servers, eliminating the need to transmit data to the cloud via the core network. Compared to central cloud-based servers, edge servers typically have relatively limited computing power. Therefore, the algorithms deployed on them must balance maintaining application-specific accuracy with ensuring computational lightweightness. Where computing power allows, this application utilizes the capabilities of the Faster R-CNN architecture, supplemented by the RetinaFace face detection algorithm, for accurate facial object localization. Notably, in scenarios where edge server computing and storage resources are particularly limited, a lightweight face object detection network based on RetinaFace is constructed, exhibiting minimal storage and computational overhead, and ensuring efficient face object detection even on resource-constrained edge servers.

[0086] In this embodiment, the cloud server plays a crucial role in aggregating data from edge servers, utilizing global knowledge to update models on both edge servers and terminal devices. Considering the local limitations of edge server models on initial training samples, their accuracy may decrease over time. To mitigate this issue, the cloud server can integrate labeled data from various edge servers, coordinating an ensemble model specifically tailored for object detection. When an edge server identifies a potentially unknown object, the cloud server can intervene, providing its expertise in detecting such anomalies. Furthermore, under specific conditions, such as when the detection system repeatedly encounters critical facial data, the cloud server can initiate a model update operation, training a lightweight face detection network on the cloud server. Once training is complete, the lightweight face detection network is transferred to an adapted edge server and periodically updated as needed. Figure 2As shown, this embodiment utilizes the communication connection between terminal devices, edge devices, and cloud servers, and takes into account the complexity of wireless communication. To enhance cloud-edge-device interaction, conserve bandwidth, and optimize the widespread deployment of multi-task models in wireless networks and cloud computing, the edge server plays a crucial role. It not only deploys preliminary models for specific application scenarios but also performs precise data processing on terminal devices within its service range. Compared to the global data processing capabilities of the cloud, the edge server, while lacking in computing resources, has a relatively limited range of object types to identify within its designated area. A specially optimized local model with high detection accuracy can meet the requirements. The model on the edge server needs to be updated regularly, primarily for two reasons: first, to address inevitable attribute changes during system operation, i.e., the potential performance degradation of the model over time; and second, to address new task objects introduced by changing scenarios, requiring timely updates based on global information provided by the cloud to maintain service stability and adaptability. To address these challenges, this application designs an automatic model update mechanism, allowing timely updates of the model on the edge server via the cloud. This mechanism mainly consists of three stages: identification of newly emerging unknown face objects, activation of update triggers, and execution of the update process. Throughout the system operation, the interaction between the terminal device and the edge server is crucial. To maximize the utilization of the edge server's computing resources, this application also performs intelligent compression processing on the media data uploaded by the terminal device to the edge server, aiming to reduce network bandwidth consumption without sacrificing data quality, thereby achieving efficient resource utilization and system performance optimization.

[0087] Specifically, such as Figure 4 As shown, the lightweight face object detection network includes an input layer, an SSH detection module, and a feature pyramid network module; the input layer is used to transmit sample images with real face object bounding boxes to the lightweight face object detection network;

[0088] The feature pyramid network module includes several pre-set stacked convolutional neural network layers. The SSH detection module is connected to each convolutional neural network layer and is used to enhance the capture of face and surrounding environment information in the sample image to obtain face enhancement sample image. Each convolutional neural network layer is used to extract face features at multiple scales from the face enhancement sample image. The face features include shape features and texture features.

[0089] Based on the extracted multi-scale facial features, face detection boxes of different scales are obtained based on regression boxes, and face targets of different sizes are obtained based on preset face feature anchor points of different sizes, so as to obtain face target feature maps.

[0090] Specifically, the Feature Pyramid Network (FPN) is constructed using stacked layers of convolutional neural networks, enabling the network to capture multi-scale image features ranging from fine to coarse. The FPN first performs dense scanning at low levels using a series of small convolutional kernels, preserving local details of the image. As the layers increase, the FPN integrates semantic information from the upper layers using larger receptive fields, achieving a deeper understanding of image features. This multi-layered feature fusion strategy ensures that the FPN can comprehensively process image features at different scales, providing reliable support for recognizing diverse faces. Based on the FPN structure, this application introduces an SSH detection module in each layer, designed to enhance the capture of information about the face and its surrounding environment. This module employs a multi-branch structure, where each branch is equipped with convolutional kernels of different sizes to capture spatial features ranging from subtle to broad. These branches work in parallel, comprehensively capturing the contextual information of the face. After passing through the SSH module, the comprehensive feature map obtained by the FPN enhances the representation of environmental information at multiple scales, improving the overall understanding of facial features. The SSH module also includes additional context-enhanced convolutional layers to further refine and enhance the merged features, effectively improving the model's accuracy and robustness in handling various complex scenarios. The face detection network constructed in this invention is based on the RetinaNet architecture and utilizes feature pyramid technology to fuse multi-scale information. Specifically, the stacked convolutional neural network layers in the feature pyramid network consist of three feature layers, generating three face detection boxes at different scales for each detection box layer, and introducing face feature anchors of different sizes to ensure the detection of face targets of different sizes. Furthermore, this application uses the detection module of the SSH detection network in FPN, which includes an SSH context module. Figure 4 As shown, this network module superimposes a context module into the convolution to generate classification and regression heads, thereby obtaining the network output. Furthermore, target point regression is added on top of the classification and regression heads to further improve network performance.

[0091] The AIGC face authentication model includes a CNN branch network, a Transformer branch network, and several Conv-Attribution Map modules.

[0092] Specifically, the hybrid network architecture of the AIGC face authentication model is deployed in the cloud (cloud server), fully leveraging the advantages of convolutional operations and self-attention mechanisms to improve the detection capability of AIGC forged regions in facial images. The AIGC face authentication model is a deep neural network that integrates CNN and Transformer. It takes an image cropped according to the face bounding box as input and outputs the discrimination result of genuine and fake face images end-to-end on the cloud server. The fusion network of the AIGC face authentication model includes two branches: a CNN network based on the Xception architecture and a Transformer network based on the Swim Transformer architecture. The CNN branch uses convolutional attention maps to evaluate the sensitivity of different facial regions, thereby identifying regions where face forgery may occur. By utilizing convolutional attention maps, potential forgery regions in facial images can be accurately located, fully leveraging the powerful ability of CNN to capture local features. Simultaneously, the Transformer branch employs a self-attention mechanism to detect long-range dependencies in the image and combines this with path merging from the CNN's attention maps. By integrating and fusing attention results, the network in this application effectively captures the complex relationships between different regions in an image. By inputting a cropped face image, it outputs the discrimination results of real and fake face images end-to-end in the cloud server, thereby achieving reliable detection of manipulated abnormal regions.

[0093] The CNN branch network is used to extract local features from the output of the lightweight face object detection network, and the CNN branch network includes a sequentially connected Conv-input input layer, three identical network convolutional branches, a Concat concatenation layer, a first convolutional layer, and a second convolutional layer.

[0094] The Conv-input layer is used to input the output of the lightweight face detection network into each network convolution branch for convolution operation, and the Concat concatenation layer concatenates the outputs of each network convolution branch to obtain a local initial feature map.

[0095] The first convolutional layer is used to perform a convolution operation on the local initial feature map to obtain a local intermediate feature map, and then transmit the local intermediate feature map to the second convolutional layer;

[0096] The second convolutional layer is used to perform convolution operations on the local intermediate feature maps to obtain the local final feature maps;

[0097] The Transformer branch network is used to fuse features based on the output of the lightweight face object detection network and the local features extracted by the CNN branch network to obtain a face feature fusion map.

[0098] The Transformer branch network includes a patch partitioning layer, a linear embedding layer, a first Swing Transformer block module, a second Swing Transformer block module, a third Swing Transformer block module, and a fourth Swing Transformer block module connected in sequence. A patch merging layer is connected between two adjacent Swing Transformer block modules. The patch merging layer includes a first patch merging layer, a second patch merging layer, and a third patch merging layer.

[0099] The patch partitioning layer is used to downsample the output of the lightweight face object detection network and segment it to obtain non-overlapping patch blocks to obtain patch feature maps.

[0100] The Linear Embedding layer is used to transform the patch feature map into a first-scale feature map of a set dimension, and transmit the first-scale feature map to the first Swing Transformer block module;

[0101] The first Swin Transformer block module is used to extract features from the first-scale feature map to obtain the first-scale feature vector;

[0102] The first Patch Merging layer and the Concat layer are connected sequentially by the first Conv-AttributionMap convolutional attention map module and the first patch Embedding layer.

[0103] Furthermore, the first Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local initial feature map and obtain an optimized local initial feature map;

[0104] The first patch Embedding layer is connected to the first Conv-Attribution Map convolutional attention map module, which is used to transform the optimized local initial feature map into a first local feature vector and transmit it to the first PatchMerging fusion layer;

[0105] The first Patch Merging fusion layer is used to fuse the first local feature vector with the first scale feature vector to obtain a second scale feature map; and to transmit the second scale feature map to the second SwinTransformer block module.

[0106] The second Swin Transformer block module is used to extract features from the second-scale feature map to obtain the second-scale feature vector;

[0107] The second Patch Merging fusion layer and the first convolutional layer are connected sequentially by the second Conv-AttributionMap convolutional attention map module and the second patch Embedding layer;

[0108] Furthermore, the second Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local intermediate feature map and obtain an optimized local intermediate feature map;

[0109] The second patch Embedding layer is connected to the second Conv-AttributionMap convolutional attention map module, which is used to transform the optimized local intermediate feature map into a second local feature vector and transmit it to the second PatchMerging fusion layer;

[0110] The second Patch Merging fusion layer is used to fuse the second local feature vector with the second scale feature vector to obtain a third scale feature map; and to transmit the third scale feature map to the third SwinTransformer block module;

[0111] The third Swin Transformer block module is used to extract features from the third-scale feature map and obtain the third-scale feature vector.

[0112] The third Patch Merging fusion layer and the second convolutional layer are connected sequentially by the third Conv-AttributionMap convolutional attention graph module and the third patch Embedding layer;

[0113] Furthermore, the third Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local final feature map and obtain an optimized local final feature map;

[0114] The third patch Embedding layer is connected to the third Conv-AttributionMap convolutional attention map module, which is used to transform the optimized local final feature map into a third local feature vector and transmit it to the third PatchMerging fusion layer.

[0115] The third Patch Merging fusion layer is used to fuse the third local feature vector with the third scale feature vector to obtain a face feature fusion map; and the face feature fusion map is transmitted to the fourth SwinTransformer block module for feature extraction so as to achieve AIGC face authentication results through the detector.

[0116] Specifically, the Transformer branch network is built on the powerful Swim Transformer architecture and integrates convolutional attention maps (Conv-Attribution Maps) generated from CNN branches. The Swim Transformer branch network is a Transformer-based image processing module designed specifically for AIGC forgery detection. It utilizes a self-attention mechanism to comprehensively model long-range dependencies between different regions in an image, achieving powerful global information modeling. By adaptively weighting features from different regions, the Swim Transformer branch network enhances the network's representational power, ensuring better representation of key features. Furthermore, its ability to handle long-range dependencies allows it to capture a wider range of features, effectively resolving complex relationships between different regions. In the context of AIGC forgery detection, the advantages of the Swim Transformer branch network are significantly demonstrated. Since facial forgery may involve forging the entire facial image, modeling global information and handling long-range dependencies are crucial for accurately detecting forged regions. The self-attention mechanism of the Swim Transformer branch network enables it to capture important relationships between various regions in an image, thereby achieving more accurate detection. Compared to traditional convolutional neural networks, the Swim Transformer branch network can handle a wider range of features and better capture the complex relationships between forged regions, thereby further improving the model's detection performance.

[0117] Furthermore, to address the inconsistency in feature mapping between CNN and Transformer branches, the following operations are used to integrate convolutional attention maps and optimize the network of the AIGC forgery detection model, thereby better performing the patch embedding operation:

[0118] First, the feature dimensions of the CNN and Transformer network branches differ. The CNN branch has a feature map of size C×H×W (where C represents the number of channels, and H and W represent the height and width, respectively), while the Transformer branch contains a patch embedding of shape (K+1)×E (where K, 1, and E represent the number of image patches, class tokens, and embedding dimension, respectively). Subsequently, the adjusted feature map is combined with the patch embedding to integrate information. Furthermore, considering the significant semantic gap between the feature map and the patch embedding—where the feature map is collected through local convolutional operators, and the patch embedding is aggregated using a global self-attention mechanism—the semantic gap is gradually bridged by applying Patch Embedding in each block (except the first block). Patch embedding achieves interactive coupling between local features and global representations, leading to a more comprehensive integration of semantic information from the features. This effectively addresses the inconsistency in feature dimensions between the CNN and Transformer network branches and utilizes convolutional attention maps to optimize the CNN forgery detection branch of this application. The convolutional attention maps accurately demonstrate the detector's sensitivity to different facial regions. As a result, the network can more accurately locate potential forgery areas, thereby improving the performance and reliability of facial forgery detection.

[0119] Specifically, the detection performance of the CNN fake detection branch is optimized by setting a convolutional attention map (Conv-Attribution Map). The convolutional attention map is a feature map based on the network that precisely locates the detector's sensitivity to different facial regions. Taking a human face image as input, forward propagation is performed to obtain the detector's output, i.e., real and fake, used to evaluate the image's authenticity. The real / fake gradient is then utilized. and The impact of perturbations on the logit output of the image is evaluated. Then, using the maximum absolute difference function, the sensitivity of the detector at each image pixel is visually displayed through a convolutional attention map. Specifically, the convolutional attention map defines the most sensitive region that has the greatest impact on the detection results. During forward propagation, the detector takes a face image I as input and generates two results, real and fake, to measure the realism of the image. Since any perturbation affects both results simultaneously, the detection result depends on the relative magnitude of the two logits. The real and fake gradients are utilized. and This represents the effect of the perturbation on the logit output of image I. Then, the convolutional attention map is defined as... The maximum absolute difference concisely captures the perturbation effect of each convolutional attention map value on the detection result, accurately representing the detector's sensitivity at each pixel, expressed as:

[0120]

[0121] Here, I represents a face image, the function max(·) calculates the maximum value along the channel axis, and the function abs(·) calculates the absolute value of each pixel. By using convolutional attention maps, key regions of interest to the detector in face images are effectively identified, resulting in significant improvements in guided erasure and forgery visualization. In summary, the forgery detection branch of the CNN network is optimized through the configured convolutional attention map module, achieving performance improvements and providing a more reliable and accurate solution for AIGC forgery detection.

[0122] Furthermore, the convolutional attention map accurately determines the detector's attention to forged regions, and this attention is used to optimize the CNN forgery detection branch of this application. During the optimization process, this application combines the convolutional attention map with the forgery detection cross-entropy loss function, improving the robustness and generalization of the AIGC face forgery detection model network across different forgery techniques. This enables the network to accurately identify potential abnormal regions in forged images, enhancing the performance and reliability of AIGC forgery detection.

[0123] S3: Based on the face image data training set, train the AIGC forgery detection model to obtain an optimized AIGC forgery detection model;

[0124] Furthermore, the model training includes: using the edge device based on a lightweight face target detection network to perform feature extraction training on face image data to obtain face target feature maps;

[0125] The facial target feature map is then transmitted to a cloud server; the cloud server, based on the AIGC facial authentication model, trains the facial image authentication based on the facial target feature map.

[0126] Specifically, the following steps are included:

[0127] S31 uses the face image data training set as input data, takes the sample image with the real face target box as input data, and takes the true and false category label of the sample image as output data, and transmits it to the edge device and the cloud server in sequence.

[0128] S32: Based on the loss function of the constructed face target detection network and the AIGC face authentication loss function, the edge device performs feature extraction training on the face image data to obtain the face target feature map based on the lightweight face target detection network;

[0129] The facial feature map is then transmitted to the cloud server.

[0130] Furthermore, the cloud server uses the AIGC facial authentication model to train the authenticity recognition of facial images based on the facial target feature map.

[0131] To enable model training for the AIGC forgery detection model;

[0132] Specifically, in order to achieve a multi-faceted optimization objective in the loss function of a lightweight face detection network for edge servers, L... arc , and L rec Three loss functions are used as the overall loss function for the edge server face detection model. This ensemble approach not only improves the discriminative power of face features but also accurately regresses bounding boxes and optimizes the model's ability to handle face shape and texture. The overall loss function considers classification accuracy, localization precision, and quality, thereby improving the overall performance of the edge server face detection network. The loss function Loss of the lightweight face object detection network in the edge server includes the additive corner margin loss function L. arc α-IoU loss function and reconstruction loss function L of feature point regression model rec The weighted sum is as follows:

[0133] Loss=λ1L arc +λ2L IoU +λ3L rec

[0134] In the formula: L arc This indicates that it is used to improve the additive angle margin loss between classes in the feature space, thereby increasing the classification accuracy; L IoU This represents the α-IoU loss used to optimize the bounding box prediction, improving localization accuracy; L rec λ1, λ2, and λ3 represent the reconstruction loss of the feature point regression model used to ensure accurate reconstruction of face shape and texture, thereby enhancing the network's understanding of complex face data; λ1, λ2, and λ3 represent the weight parameters of the loss function. These weight parameters are used to balance the contributions of each loss function to the total loss, and their optimal values ​​are typically selected through cross-validation.

[0135] Using the additive angle margin loss function L arc The feature data of the final obtained face target feature map are embedded on a hypersphere of radius S for class boundary constraints to achieve generalization performance on large datasets during the face target detection network training process. The additive angular margin loss function L... arc The expression is

[0136]

[0137] In the formula: N represents the number of training samples; yi θ represents the true class label of sample i; j x represents i and W j The angle between them, and W j The weight vector j and x of the neural network are represented by these vectors. i represents the vector of facial target features of sample i; S represents the scaling parameter set to control the magnitude of the feature vector; m represents the custom additive angular margin added to add a fixed margin at the angle of the target category to enhance the distinction between categories;

[0138] Specifically, in this embodiment, to enhance the feature learning and discrimination capabilities of the face detection network, the Additive Corner Margin Loss (ArcFace) method is introduced. This method aims to optimize the loss function so that the features learned by the network can more accurately reflect the similarity and differences between faces. Unlike the traditional Softmax Loss, the ArcFace loss function is specifically optimized for feature embedding to enhance intra-class consistency and inter-class discriminability, especially when facing large-scale datasets and significant intra-class variations. The loss function... We emphasize the target class for each sample and rescale all logits using S, ensuring that the feature embeddings are distributed on a hypersphere of radius S. This guarantees that the prediction depends only on the angle θ. j Furthermore, by using additive angular margin m, the loss function forces samples of the same class in the feature space to be more compact and samples of different classes to have larger margins, thereby improving the discriminative power of the features, helping the model to generalize on large-scale datasets, and effectively handling intra-class appearance variations.

[0139] The α-IoU loss function is used to adaptively adjust the network parameter weights of the lightweight face detection network based on the face detection bounding boxes and face target bounding boxes, thereby improving the localization of face targets. The expression of the α-IoU loss function is as follows:

[0140]

[0141] In the formula: The value of the loss function is represented by IoU; IoU represents the intersection-over-union ratio between the predicted bounding box and the actual bounding box; α represents the power parameter.

[0142] To achieve more accurate bounding box regression and object detection, the calculation of the α-IoU loss function is incorporated into the face object detection network. The α-IoU loss function is a unified power-based IoU loss function; by adjusting the α parameter, the detector can have greater flexibility in achieving different levels of bounding box regression accuracy. Furthermore, networks trained using the α-IoU loss function exhibit stronger robustness and are better able to handle the effects of small datasets and noise.

[0143] First, the intersection and union of the face detection bounding boxes and the face target bounding boxes need to be calculated, which forms the basis of IoU calculation. In lightweight face object detection networks, calculating the intersection and union of the face detection bounding boxes and the face target bounding boxes is a key step in evaluating prediction accuracy. This calculation process can be performed using Intersection over Union (IoU), a commonly used performance metric for object detection tasks. The formula for calculating IoU is:

[0144]

[0145] The specific calculation steps are as follows: 1. Calculate the intersection: For face detection boxes and face target boxes, find their overlapping regions. The area of ​​the intersection is the area of ​​the overlapping regions. 2. Calculate the union: The area of ​​the union is equal to the sum of the areas of the two bounding boxes minus the area of ​​the intersection. To further improve the accuracy and robustness of prediction, this application introduces the α-IoU loss function. The α-IoU loss enhances the sensitivity to the accuracy of predicted bounding boxes by applying a power transformation to the basic calculation of IoU; α is a power parameter that can be adjusted to control the shape and slope of the loss function. When α equals 1, the loss function becomes the traditional IoU loss. When α is greater than 1, the loss function pays more attention to high IoU values, which is very useful in situations where accuracy is critical. When α is less than 1, the loss function is more tolerant of low IoU values, which helps to improve recall. By adjusting the value of α, this application can trade off between precision and recall, thereby customizing the performance of the loss function according to specific application needs.

[0146] Feature point regression is performed on the sample images and face target feature maps of the face image dataset to obtain the face features of the corresponding sample images and face target feature maps, and the face features are encoded.

[0147] The facial features include shape features and texture features;

[0148] And through the reconstruction loss function L of the feature point regression model rec Based on the shape and texture feature errors between the sample image and the facial target feature map, the network parameter weights of the lightweight facial target detection network are adaptively adjusted to improve the extraction of facial target features. The reconstruction loss function L of the feature point regression model... rec The expression is

[0149]

[0150] In the formula: θ D Indicate decoder The parameters determine how shape and texture are extracted from the encoded features. This represents a feature point regression decoder used to reconstruct the encoded features into a visual representation of face shape and texture; 'i' represents the index used to iterate over all face shape and texture data points; E M This represents a feature point regression encoder used to extract features of shape and texture. Indicates encoder E M The parameters; S i Represents the shape of the original feature point regression; A i The texture represents the original feature point regression; ||·||2 represents the shape reconstruction error term calculated using Euclidean distance; ||·||1 represents the texture reconstruction error term calculated using absolute distance.

[0151] This embodiment uses a loss function L of a feature point regression model. rec This loss function takes into account the shape (S) i ) and texture (A i There are two aspects to the error. Shape error is calculated using the feature point regression encoder E. M Feature representations of shape Si are extracted, and the encoded shape is decoded using a feature point regression decoder D to obtain the final shape. Euclidean distance (L2 norm) is used to measure the difference between the current shape and the original shape S. i The difference between them. Texture error is then addressed through feature point regression encoder E. M Get texture A i Feature representation, using absolute distance (L1 norm) to calculate the difference between the texture and the original texture A i The difference between them. By minimizing the reconstruction loss function L of the feature point regression model. rec It can adjust the parameters of the encoder and decoder to improve the accuracy of shape and texture, enabling the model to more accurately reproduce the details of the input data.

[0152] Furthermore, the constructed AIGC face authentication loss function adopts the cross-entropy loss function;

[0153] Based on the cross-entropy loss function, the loss value between the predicted value for fake detection obtained from the face feature map output by the lightweight face object detection network and the corresponding true / false category label is minimized; the parameter weights of the AIGC face fake detection model are adaptively adjusted to optimize the AIGC face fake detection model.

[0154] S33: If the output of the trained AIGC forgery detection model converges, then the trained AIGC forgery detection model is the AIGC forgery detection optimization model; otherwise, adaptively adjust the network parameter weights of the lightweight face target detection network and the AIGC face forgery detection model, and repeat steps 31 to 32.

[0155] S4: Based on the AIGC forgery detection optimization model, the authenticity of the face image data test set is determined.

[0156] In specific embodiments, interactive updates of the model are also included;

[0157] New facial image data is periodically obtained from the collected facial video data through terminal devices to generate and store new facial image datasets;

[0158] When the capacity of new face image data in the face image dataset recorded and stored by the terminal device reaches a preset capacity threshold, the terminal device sends a trigger update signal to the cloud server.

[0159] When the cloud server receives the trigger update signal, it starts the training process of the AIGC forgery detection model to update the AIGC forgery detection model, and distributes the trained and updated lightweight face target detection network to the edge device for online update, and stores the trained and updated AIGC face forgery detection model in the cloud service for online update.

[0160] In this embodiment, an AIGC forgery detection model based on a cloud computing platform is constructed. The use of the cloud computing platform reduces network bandwidth consumption, achieving efficient resource utilization and system performance optimization. The AIGC forgery detection model includes a lightweight face detection network deployed in an edge device and an AIGC face authentication model deployed in a cloud server. The AIGC forgery detection model is optimized by constructing loss functions for the face detection network and the AIGC face authentication loss function, and then trained using a face image data training set to obtain an optimized AIGC forgery detection model, enabling the authentication of face image data test sets. By dividing the AIGC forgery detection model into a lightweight face detection network and an AIGC face authentication model, and through collaboration and interaction between the terminal device, edge server, and cloud server, the local edge face detection model and the cloud-trained model library are dynamically updated. This allows for efficient resource utilization while improving the rapid detection and recognition of face forgeries through optimization of the AIGC forgery detection model. By leveraging the complementary strengths of CNN and Transformer backbone networks, and through cascading operations, the backpropagation gradient of CNN is used to extract weight-based salient regions, and an attention map is generated and fused with the Transformer merging module to enhance attention to facial regions. Extensive evaluation through experiments has demonstrated its effectiveness.

[0161] The experimental results are as follows:

[0162] 1. Experiment and Dataset Setup

[0163] In this embodiment, a comprehensive evaluation of the AIGC forgery detection model based on a cloud computing platform was conducted, primarily analyzing its model accuracy, computational complexity, and inference latency. A series of experiments were performed in a cloud-edge collaborative environment, involving two main categories of tasks: machine learning model training and logical inference. For model training, a cloud server was selected, configured with a 64-core Intel Xeon Gold 6226R CPU and an Nvidia V100 GPU. During training, a periodic task offloading and model update strategy was implemented to migrate the optimized lightweight face detection network to the edge server.

[0164] In terms of logical inference, this application follows the experimental configuration shown in Table 1. To comprehensively evaluate the lightweight face object detection network, the AIGC face authentication model, and the interaction strategy, three representative face datasets were selected as experimental platforms.

[0165] Table 1. Experimental Equipment Configuration

[0166]

[0167]

[0168] Experiments were conducted using the FaceForensics++, Celeb-DF, Deeper Forensics, and DeeperForensics-1.0 datasets, which are described below:

[0169] FaceForensics++ is a large-scale video dataset containing 1000 original videos and manipulated videos obtained through four facial manipulation methods: Deepfakes, Face2Face, FaceSwap, and NeuralTextures. This dataset has been widely used to train and test the AIGC forgery detection network and is considered a benchmark for evaluating the performance of different detection methods.

[0170] Celeb-DF is another video dataset widely used in AIGC forgery detection research. It contains 408 real videos and 795 synthetic videos, with the synthetic videos generated using a combination of face swapping and reconstruction techniques. The synthetic videos in this dataset have been processed to reduce visual artifacts, making them more difficult to detect. Celeb-DF has played a crucial role in advancing state-of-the-art techniques in the field of AIGC forgery detection.

[0171] Deeper Forensics is a recent addition to the field of AIGC forgery detection datasets. It proposes a novel method for generating AIGC forgeries based on adversarial training and a new architecture called DeepFake Encoder-Decoder (DFED). This method aims to generate higher-quality, more difficult-to-detect AIGC forgery videos, outperforming videos generated using traditional methods. To evaluate the effectiveness of their method, the authors generated 1000 forged videos using FaceForensics++. The resulting dataset, known as Deeper Forensics-1.0, has been used to evaluate the performance of different AIGC forgery detection methods and has proven to be a valuable resource for advancing the state-of-the-art in this field.

[0172] 2. Comparison of AIGC forgery detection performance

[0173] Table 2 lists the performance of the detection method proposed in this application on three different datasets. The FaceForensics++ and Celeb-DF datasets are closed-set experiments, while the Deeper Forensics dataset is an open-set experiment (i.e., the model trained on FaceForensics++ is used for testing on the Deeper Forensics dataset). As can be seen from the table, the method proposed in this application achieved the highest average accuracy on all three datasets, at 99.36%, 98.43%, and 75.17%, respectively. These results demonstrate that the proposed method is more reliable and accurate than existing techniques and can better detect forged face images. In the closed-set case, the algorithm proposed in this application achieved near-perfect accuracy on all three datasets. While the accuracy decreased slightly in the open-set case, it is still significantly higher than other algorithms, such as the Xception-c23 algorithm, which only achieves 73.76% accuracy in the open set. Furthermore, this study also found that a single deep neural network structure does not guarantee better detection results. For example, on the Deeper Forensics dataset, although the Xception-c23 and Face X-ray methods employ a single deep network structure, their average accuracy is still lower than that of the fusion network algorithm constructed in this invention. Therefore, experimental results show that a greater emphasis on model structure design and training strategies can lead to higher detection accuracy and robustness. In summary, on public datasets, the novel method proposed in this application demonstrates higher accuracy and generalization.

[0174] Table 2. Comparison of main experimental results

[0175]

[0176] Further testing experiments were conducted on AIGC forgery detection videos in real-world scenarios. These real test videos were meticulously forged using celebrity faces and downloaded from YouTube channels. The results obtained from testing this real dataset are impressive. According to the results listed in Table 3, most existing AIGC forgery detection methods have relatively low accuracy. In fact, on this real dataset, many existing detection methods achieve an accuracy of less than 50%. However, the network fusion algorithm proposed in this application achieves high performance, with an average accuracy approaching 95%. This result demonstrates that the detection method proposed in this application can effectively identify AIGC forgery images and videos in the real world.

[0177] Table 3. Depth-based fake detection AUC (%) of meticulously crafted videos collected from the Internet

[0178] Multi-task 62.50 26.45 78.50 55.81 Xception-c23 44.56 83.85 99.31 75.90 FFD 56.04 71.16 79.07 68.75 FaceX-ray 66.49 94.03 87.10 82.54 Ours 82.27 97.78 98.05 94.36

[0179] In a real-world scenario simulation, this application further demonstrates a typical example of real-world AIGC spoofing detection. The authentic AIGC spoofing videos were downloaded from the YouTube channel "CtrlShiftFace2," which is meticulously crafted, with all spoofed identities being celebrities. Based on the results listed in Table 2, this application found that most existing methods have relatively low accuracy. However, for the information and communication technology in this application, high performance is achieved, averaging close to 95%. The reference auxiliary version can even improve performance to 100%.

[0180] 3. Robustness test

[0181] Deploying forgery detectors in real-world scenarios requires them to withstand common perturbations while maintaining detection performance. To evaluate the robustness of the detector presented in this application, an uncompressed AIGC forgery detection model was trained and tested on various forms of invisible damage using perturbated images. These perturbations included saturation changes, contrast changes, blocky distortion, Gaussian white noise addition, blurring, pixelation, and video compression using the H.264 codec. The perturbations were evaluated at five severity levels to analyze their impact on the detection accuracy of the model. A representation of the dataset is shown below. Figure 5 As shown.

[0182] Figure 6 The effect of increasing the severity of each corruption level is shown, and Table 3 shows the average AUC score for all severity levels of each corruption level. Notably, the algorithm in this application exhibits significantly higher robustness to most perturbations than other methods. Figure 6 The average results in (g) demonstrate this. Even at the most severe levels of compression and pixelation degradation, the algorithm of this application still achieves an accuracy of over 80%. While the algorithm does exhibit a slight downward trend in terms of Gaussian noise and filtering operations, it still outperforms other methods and holds promise for addressing the greater challenges of image corruption and degradation.

[0183] The experimental results of this application confirm the robustness of the proposed detector to these perturbations, as shown in Table 3. Specifically, the average detection accuracy of the detector remains above 90% across all five severity levels of perturbations, highlighting its effectiveness in detecting depth-spoofed images and videos even under the most challenging conditions. These findings demonstrate that the detector of this application can be confidently deployed in real-world scenarios where images and videos are frequently subjected to various forms of corruption and compression.

[0184] 4. Attention Experiment

[0185] In this experiment, the goal of this application is to investigate the attention regions for face forgery detection and evaluate the effectiveness of the proposed method. To achieve this, experiments were conducted on datasets of both forged and real facial images, and the attention maps generated by the proposed method were analyzed. Figure 7 As shown, the results of this experiment provide convincing evidence of the effectiveness of the proposed method in capturing attention regions for facial forgery detection. Analysis of the attention maps shows that they effectively capture the image regions most relevant to detecting image forgery. Specifically, the attention maps highlight the areas around the eyes, nose, and mouth, which are the most critical areas for distinguishing real from forged facial images. Furthermore, the attention maps also highlight areas with obvious forgery traces, such as areas with uneven textures and abnormal pixel distributions, which are important indicators of image forgery. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed AIGC forgery detection method based on deep fusion networks, characterized in that, Includes the following steps: S1: Store the collected face image data in the terminal device for preprocessing to obtain a face image dataset, and randomly divide the face image dataset to obtain a face image data training set and a face image data test set; S2: An AIGC forgery detection model is built based on a cloud server. The AIGC forgery detection model includes a lightweight face target detection network for detecting and acquiring face feature maps and an AIGC face authentication model for realizing forgery recognition based on face feature maps. The completed lightweight face detection network is then transmitted to the edge device for configuration. The constructed lightweight face object detection network includes an input layer, an SSH detection module, and a feature pyramid network module; The input layer is used to transmit sample images with real face bounding boxes to the SSH detection module; The SSH detection module is connected to each convolutional neural network layer to obtain face enhancement sample images; and the feature pyramid network module includes several pre-set stacked convolutional neural network layers. Each convolutional neural network layer is used to extract facial features at multiple scales from the face enhancement sample image; the facial features include shape features and texture features; Based on the extracted multi-scale facial features, face detection boxes of different scales are obtained based on regression boxes, and face targets of different sizes are obtained based on preset face feature anchor points of different sizes, so as to obtain face target feature maps; The constructed AIGC face authentication model includes a CNN branch network, a Transformer branch network, and several Conv-Attribution Map convolutional attention map modules; The CNN branch network is used to extract local features from the output of the lightweight face object detection network, and the CNN branch network includes a sequentially connected Conv-input input layer, three identical network convolutional branches, a Concat concatenation layer, a first convolutional layer, and a second convolutional layer. The Conv-input layer is used to input the output of the lightweight face detection network into each network convolution branch for convolution operation, and the Concat concatenation layer concatenates the outputs of each network convolution branch to obtain a local initial feature map. The first convolutional layer is used to perform a convolution operation on the local initial feature map to obtain a local intermediate feature map, and then transmit the local intermediate feature map to the second convolutional layer; The second convolutional layer is used to perform convolution operations on the local intermediate feature maps to obtain the local final feature maps; The Transformer branch network is used to fuse features based on the output of the lightweight face object detection network and the local features extracted by the CNN branch network to obtain a face feature fusion map. The Transformer branch network includes a patch partitioning layer, a linear embedding layer, a first Swin Transformer block module, a second Swin Transformer block module, a third Swin Transformer block module, and a fourth Swin Transformer block module connected in sequence. Adjacent Swin Transformer block modules are connected by a patch merging layer, which includes a first patch merging layer, a second patch merging layer, and a third patch merging layer. The patch partitioning layer is used to downsample the output of the lightweight face object detection network and segment it to obtain non-overlapping patch blocks to obtain patch feature maps. The Linear Embedding layer is used to transform the patch feature map into a first-scale feature map of a set dimension, and transmit the first-scale feature map to the first Swing Transformer block module; The first Swin Transformer block module is used to extract features from the first-scale feature map to obtain the first-scale feature vector; The first Patch Merging layer and the Concat layer are connected sequentially by the first Conv-AttributionMap convolutional attention map module and the first patch Embedding layer. Furthermore, the first Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local initial feature map and obtain an optimized local initial feature map; The first patch embedding layer is connected to the first convolutional attention map module, which is used to transform the optimized local initial feature map into a first local feature vector and transmit it to the first patch merging fusion layer; The first Patch Merging fusion layer is used to fuse the first local feature vector with the first scale feature vector to obtain a second scale feature map; and to transmit the second scale feature map to the second Swing Transformerblock module. The second Swin Transformer block module is used to extract features from the second-scale feature map to obtain the second-scale feature vector; The second Patch Merging fusion layer and the first convolutional layer are connected sequentially by the second Conv-AttributionMap convolutional attention map module and the second patch Embedding layer; Furthermore, the second Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local intermediate feature map and obtain an optimized local intermediate feature map; The second patch embedding layer is connected to the second conv-attribution map convolutional attention map module, which is used to transform the optimized local intermediate feature map into a second local feature vector and transmit it to the second patch merging fusion layer; The second Patch Merging fusion layer is used to fuse the second local feature vector with the second scale feature vector to obtain a third scale feature map; and to transmit the third scale feature map to the third Swing Transformerblock module; The third Swin Transformer block module is used to extract features from the third-scale feature map and obtain the third-scale feature vector. The third Patch Merging fusion layer and the second convolutional layer are connected sequentially by the third Conv-AttributionMap convolutional attention graph module and the third patch Embedding layer; Furthermore, the third Conv-Attribution Map convolutional attention map module is used to evaluate the sensitivity of the facial region in the local final feature map and obtain an optimized local final feature map; The third patch embedding layer is connected to the third Conv-Attribution Map convolutional attention map module, which is used to transform the optimized local final feature map into a third local feature vector and transmit it to the third patch merging fusion layer; The third Patch Merging fusion layer is used to fuse the third local feature vector with the third scale feature vector to obtain a face feature fusion map; and the face feature fusion map is transmitted to the fourth Swing Transformerblock module for feature extraction so as to achieve AIGC face authentication results through the detector. S3: Train the AIGC forgery detection model based on the face image data training set to obtain the AIGC forgery detection optimized model; The model training includes: using the edge device, based on a lightweight face target detection network, performing feature extraction training on a face image data training set to obtain face target feature maps; The facial feature map is then transmitted to the cloud server. Using the cloud server, the AIGC facial authentication model is used to train the facial image authenticity recognition system based on the facial target feature map. S4: Based on the AIGC forgery detection optimization model, the authenticity of the face image data test set is determined.

2. The distributed AIGC forgery detection method based on deep fusion networks according to claim 1, characterized in that, S3 specifically includes the following steps: S31: Based on the face image data training set, sample images with real face target boxes are used as input data, and the true / false category labels of the sample images are used as output data, and are transmitted sequentially to the edge device and the cloud server. S32: Based on the loss function of the constructed face target detection network and the AIGC face authentication loss function, the edge device performs feature extraction training on the face image data to obtain the face target feature map based on the lightweight face target detection network; The facial feature map is then transmitted to the cloud server. Furthermore, the cloud server uses the AIGC facial authentication model to train the authenticity recognition of facial images based on the facial target feature map. To enable model training for the AIGC forgery detection model; S33: If the output of the trained AIGC forgery detection model converges, then the trained AIGC forgery detection model at this time is the AIGC forgery detection optimization model. Otherwise, adaptively adjust the network parameter weights of the lightweight face object detection network and the AIGC face authentication model, and repeat steps 31 to 32.

3. The distributed AIGC forgery detection method based on deep fusion networks according to claim 2, characterized in that, The loss function of the lightweight face detection network constructed in S3 Including additive angle margin loss function α-IoU loss function and reconstruction loss function of feature point regression model The weighted sum is as follows: In the formula: This represents the additive corner margin loss used to improve inter-class features in the feature space; This represents the α-IoU loss used to optimize the bounding box prediction; This represents the reconstruction loss of the feature point regression model used to ensure accurate reconstruction of shape and texture; , , The weight parameters represent the loss function; The additive corner margin loss function The expression is In the formula: Indicates the number of training samples; Indicates sample The true category label; express and The angle between them, and Represents the weight vector of a neural network ; Indicates sample A vector of facial target features; This indicates the scaling parameters that are set; This indicates a custom-added additive corner margin; The expression for the α-IoU loss function is as follows: In the formula: This represents the value of the loss function; IoU This represents the intersection-union ratio between the predicted bounding box and the actual bounding box; Indicates an exponential parameter; The reconstruction loss function of the feature point regression model The expression is In the formula: Indicate decoder The parameters, This represents a feature point regression decoder used to reconstruct the encoded features into a visual representation of face shape and texture. This represents a feature point regression encoder used to extract features of face shape and texture. Indicates encoder Parameters; This represents the shape of the regressed original feature points; Texture representing the regression of the original feature points; This represents the reconstruction error term calculated using Euclidean distance; This represents the reconstruction error term calculated using absolute distance; Furthermore, the constructed AIGC face authentication loss function adopts the cross-entropy loss function.

4. The distributed AIGC forgery detection method based on deep fusion networks according to claim 1, characterized in that, It also includes interactive updates of the model; New facial image data is periodically obtained from the collected facial video data through terminal devices to generate and store new facial image datasets; When the capacity of new face image data in the face image dataset recorded and stored by the terminal device reaches a preset capacity threshold, the terminal device sends a trigger update signal to the cloud server. When the cloud server receives the trigger update signal, it starts the training process of the AIGC forgery detection model to update the AIGC forgery detection model, and distributes the trained and updated lightweight face target detection network to the edge device for online update, and stores the trained and updated AIGC face forgery detection model to the cloud server for online update.

5. The distributed AIGC forgery detection method based on deep fusion networks according to claim 1, characterized in that, The preprocessing described in S1 specifically includes... We assign true / false category labels to each sample image of the face image data and perform compression processing. Then, we use regression boxes to locate and label the sample images of the face image dataset to obtain sample images with real face target boxes.

Images