Deep forgery detection method based on self-mixing

Abstract

The invention provides a deep counterfeit detection method based on self-mixing, and compared with the prior art, the method generates a mixed image containing counterfeit artifacts by using a self-mixing technology, and carries out the precise detection and recognition of an artifact region of an input image through combining a backbone network OfficientNet-b3 and a counterfeit artifact detection module. The method comprises the following steps of: firstly, performing framing processing on a real image, generating a source image and a target image by utilizing a self-mixing technology, and generating a mixed image with counterfeit artifacts by combining a mask generator with an alpha blowing model; and then, the backbone network extracts global features from the mixed image, and the fake artifact detection module reinforces local feature expression of a mixed region through an artifact space cooperation attention module, so that an artifact region is accurately positioned. And finally, fusing global and local features by using residual connection, and transmitting the fused features to a full connection layer to complete classification. According to the method, various forged image types can be effectively detected, and excellent generalization ability is shown in cross-dataset and unknown scenes.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a deep fake detection method based on self-mixing. Background Art

[0002] In recent years, the rapid development of generative technology has made the application of forged images on the Internet more and more extensive, and face forgery detection has become a hot research field. Current detection methods can be divided into two categories: those based on single-frame images and those based on multi-frame images. Traditional methods based on digital image processing mainly detect differences at the pixel level of images, and effectively identify GAN-generated face images by analyzing the pixel deviation between synthetic images and real images. However, with the advancement of generative technology, modern GAN-generated images are more difficult to distinguish from real images in terms of pixels. Existing image forgery detection methods have limited effectiveness when dealing with forgery methods that only involve facial manipulation of the face. They also perform poorly on low-quality or highly compressed images, making it difficult to capture artifacts.

[0003] This experimental team has been browsing and researching a large number of relevant records and materials on the related technologies of forged image recognition for a long time. At the same time, it relies on relevant resources and conducts a large number of related experiments. After a large number of searches, it is found that there are existing technologies such as CN112686331B, CN112149608B, CN116580409B, and CN111860314B disclosed in the prior art. For example, an image recognition method disclosed in the prior art includes: obtaining an image to be recognized; using a trained neural network to recognize the image to determine whether the image is a forged image; wherein the neural network includes a generative adversarial network The invention discloses a generator and a classifier and trains them by the following steps: obtaining an original image group including a plurality of original images and a label of each original image, wherein the original images include real images and forged images corresponding to the real images; for each original image, obtaining a first type of noise for the original image using a classifier to construct an associated first noisy image, and obtaining a second type of noise for the original image using a generative adversarial network to construct an associated second noisy image; establishing a training set based on the original image and its associated first noisy image and second noisy image; and training a neural network based on the training set to obtain parameters of the neural network.

[0004] The present invention is made to solve the common problems in the field, such as the detection methods are highly targeted but lack generalization ability; the detection effect of low-quality or highly compressed images is poor; the complexity is high and relies on large-scale data training, etc. Summary of the invention

[0005] The purpose of the present invention is to propose a deep fake detection method based on self-mixing to address the deficiencies in the current field.

[0006] In order to overcome the shortcomings of the prior art, the present invention adopts the following technical solutions:

[0007] A deep fake detection method based on self-mixing, the deep fake detection method based on self-mixing comprising:

[0008] S101: First, the data set FF is divided into frames, and only the real images in the data set FF are input. The real images are used to generate mixed images with common forgery artifacts using the self-mixing technology, and the real images and their corresponding mixed images are input into the backbone network. The backbone network is EfficientNet-b3 pre-trained on ImageNet.

[0009] S102: using the forged artifact detection module to locate and detect the mixed area in the mixed image, and output a mixed image that more prominently highlights the artifact area features and is easy to distinguish later.

[0010] S103: The enhanced mixed image obtained after processing by the forged artifact detection module is residually connected to the global features obtained by global average pooling at the end of the backbone network EfficientNet-b3 to obtain fused features, and the fused features are passed to the final fully connected layer of the backbone network for decision-making, thereby generating an accurate judgment on the authenticity of the input image.

[0011] Furthermore, the specific steps of generating the mixed image in step S101 are as follows:

[0012] S1011: generating a source image and a target image pair by a source-target image generator, wherein the source-target image generator is used to perform random image enhancement operations on the real image to obtain a source image and a target image with differences, further, the source-target image generator resizes and rotates the source image to make the resolution and size of the generated source image and the target image consistent, and the image enhancement operation includes adjusting the hue, saturation and brightness of the real image,

[0013] S1012: extracting a face region from a real image, detecting facial feature points of the face region using a Dlib or RetinaFace key point detection algorithm, wherein the facial feature points at least include eyes, nose, mouth, eyebrows, and facial contours, and calculating a convex hull of the face region based on the facial feature points.

[0014] S1013: Use the mask generator to initialize a basic mask through the convex hull. The basic mask uses the convex hull of the facial area as the shape outline. The basic mask is a grayscale image with a grayscale value range between [0.0, 1.0]. The basic mask defines the basic distribution of the mixed area with the convex hull shape outline.

[0015] S1014: using the Face X-ray method to randomly deform the shape of the basic mask, wherein the deformation includes random expansion, boundary adjustment, and translation, to generate mask images of different shapes.

[0016] S1015: Use two Gaussian filters with different parameters to smooth the boundary of the mask image, eliminate the abrupt transition of the boundary of the mask image, and make the gray value distribution of the mask image more natural.

[0017] S1016: Use alpha blending model to obtain a mixed image:

[0018]

[0019] ☉ means element-wise multiplication, I s is the generated source image, I t is the target image, R is the dynamic adjustment coefficient, M is the smoothed mask image generated by the mask generator, the grayscale value range of the mask image is [0.0, 1.0], M is used to control the proportion of the source image and the target image in the mixed area, when all pixels are limited to 0 and 1, I is adjusted before mixing s Use color correction technology to make its color consistent with I t Color matching, changing the mixing ratio of the source image and the target image by R, generates a mask with diverse grayscale value distributions, wherein the color correction technology is color transfer, gamma correction or color mapping, R∈(0,1].

[0020] Furthermore, the forged artifact detection module also includes a forged artifact detection additional layer, a forged artifact detection end layer and an artifact space collaborative attention module.

[0021] Furthermore, the operation steps of the forgery artifact detection module are as follows:

[0022] S1021: Take the mixed image extracted by the backbone network EfficientNet-b3 as input,

[0023] S1022: Process the mixed image through an additional layer of artifact detection: Use 1×1 convolution to perform channel transformation on the mixed image to highlight potential artifact information in a local range, then perform batch normalization and ReLU activation on the output of the 1×1 convolution. Normalization reduces internal covariate shift, making it easier for the model to converge and maintain training stability when learning complex forged features. Subsequently, 3×3 convolution is applied to the processed mixed image to extract more fine-grained local artifact features. After batch normalization again, the resolution of the mixed image is reduced by downsampling, so that subsequent layers can better focus on the local mixed area and alleviate the gradient vanishing problem. The processed mixed image is output.

[0024] S1023: After the artifact detection extra layer processing, an artifact space collaborative attention module is introduced to weight the spatial and channel information of the mixed image through the artifact space collaborative attention module to highlight the inconsistency of the mixed boundary of the mixed image, and further enhance the sensitivity of the model to artifact features.

[0025] S1024: Repeat steps S1022-S1023 twice, and output the processed mixed image.

[0026] S1025: First, use 3×3 convolution to adjust the number of channels without changing the spatial dimension of the mixed image, so that the features are more suitable for subsequent calculations. At the same time, ReLU activation is used to enhance the nonlinear expression ability of the network. Then, use 1×1 convolution to finely integrate the channels, and integrate the local artifact features extracted by the three artifact detection extra layers. Finally, downsampling is performed again to make the output mixed image size consistent with the input mixed image size in step S1021.

[0027] Furthermore, the artifact space collaborative attention module implements the following steps:

[0028] S201: In order to allow the network to distinguish the difference between local mixed artifact areas and local real areas at a finer level, the channel attention is decomposed into two parallel one-dimensional feature encoding processes by reducing the position information loss caused by two-dimensional global pooling, and then the two one-dimensional feature encodings are averaged pooled. Along the vertical and horizontal input channels, the two spatial ranges of the pooling kernel (H, 1) or (1, W) are used to encode each channel along the horizontal coordinate and vertical coordinate respectively, where H and W represent the height and width of the mixed image, and the output of the c-th channel with a height of h is expressed as follows:

[0029]

[0030] h represents the position index of the mixed image in the vertical direction, i represents the position index of the mixed image in the horizontal direction, is the aggregated value of the relevant features of the cth channel with height h, i∈{0,1,2...W-1}, h∈{0,1,2...H-1}, x c (h,i) is the response value of the input mixed image to the intensity of the artifact area at the cth channel with height h and width i,

[0031] S202: is the aggregated value of the relevant features of the cth channel width w, The calculation formula is as follows:

[0032] x c (j,w) is the response value of the input mixed image at the cth channel with height h and width i,

[0033] S203: and The cascade operation in the spatial dimension obtains Using a 1×1 convolution F1 Perform linear transformation and dimension adaptation to obtain

[0034] S204:

[0035] Yes Perform sigmoid nonlinear activation function processing, all elements in f are within the real number domain, c is the number of channels,

[0036] S205: Split f into f in the spatial dimension h and f w , f h Input 1×1 convolution F h And after passing the Sigmoid activation function, the output is g h , f w Input 1×1 convolution F w And after passing the Sigmoid activation function, the output is g w :

[0037] g h =σ(F h (f h )),

[0038] g w =σ(F w (f w )),

[0039] Among them, f wf is obtained by one-dimensional average pooling in the horizontal direction, the kernel size of the one-dimensional average pooling in the vertical direction is H×1, and the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the vertical direction from the mixed image. The kernel size of the one-dimensional average pooling in the horizontal direction is 1×W, and the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the horizontal direction from the mixed image, and f h All elements in are within the real number domain, and f w All elements in are in the real number domain, σ is the Sigmoid activation function, σ is used to normalize the output to the range of (0,1), f h It is obtained by performing one-dimensional average pooling in the vertical direction of f.

[0040] S206: g h The original dimension of is C×H. By broadcasting the width dimension W, the dimension becomes C×H×W, and the same height direction weight value can be applied to all width positions i in the mixed image.

[0041] g w The original dimension is C×W. By broadcasting the height dimension H, the dimension becomes C×H×W, and the same width direction weight value can be applied to all height positions h in the mixed image.

[0042] S207: Perform weight processing on each pixel in the mixed image, and output the weight processed mixed image:

[0043]

[0044] x c (i, j) is the response value of the intensity of the artifact area at the position of the input mixed image with a width of i and a height of j, y c (i, j) is the response value of the artifact area intensity at the position of width i and height j of the weighted mixed image, which integrates the attention information in the height and width directions, making the mixed image more focused on the artifact area.

[0045] The beneficial effects achieved by the present invention are:

[0046] 1. Generate a variety of mixed artifact images through self-mixing technology, truly simulate forged features, reduce dependence on real forged data, effectively improve the quality and diversity of training data, and provide more comprehensive artifact samples for model training.

[0047] 2. The forgery artifact detection module uses multiple iterative artifact detection additional layers and end layers, combined with convolution and attention mechanisms, to accurately capture the fine-grained features of the mixed artifact area, enhance the model's perception of the artifact area, and make the detection more accurate and efficient.

[0048] 3. The global features extracted by the backbone network EfficientNet-b3 and the local artifact features extracted by the artifact detection module are fused through residual connections to form a comprehensive feature representation, which effectively improves the detection performance of complex artifact scenes.

[0049] 4. The artifact space collaborative attention module is used to capture the inconsistency of the inner boundaries of mixed images, which improves the model's adaptability to unknown forged images and complex mixed scenes, making artifact detection more robust and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0051] Figure 1 This is a schematic diagram of the process of generating a mixed image in step S101 of the present invention.

[0052] Figure 2 It is a schematic diagram of the process of the forgery artifact detection module of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is further described in detail in combination with its embodiments below; it should be pointed out that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present case. For those skilled in the art, after reviewing the following detailed description, other systems, methods and / or features of this embodiment will become apparent. And the terms describing the positional relationship in the accompanying drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to the specific circumstances.

[0054] Embodiment 1: Combining with Figure 1 and attached Figure 2 This embodiment constructs a deep fake detection method based on self-mixing, and the deep fake detection method based on self-mixing includes:

[0055] S101: First, the data set FF is divided into frames, and only the real images in the data set FF are input. The real images are used to generate mixed images with common forgery artifacts using the self-mixing technology, and the real images and their corresponding mixed images are input into the backbone network. The backbone network is EfficientNet-b3 pre-trained on ImageNet.

[0056] S102: using the forged artifact detection module to locate and detect the mixed area in the mixed image, and output a mixed image that more prominently highlights the artifact area features and is easy to distinguish later.

[0057] S103: The enhanced mixed image obtained after processing by the forged artifact detection module is residually connected to the global features obtained by global average pooling at the end of the backbone network EfficientNet-b3 to obtain fused features, and the fused features are passed to the final fully connected layer of the backbone network for decision-making, thereby generating an accurate judgment on the authenticity of the input image.

[0058] The specific steps of generating a mixed image in step S101 are as follows:

[0059] S1011: generating a source image and a target image pair by a source-target image generator, wherein the source-target image generator is used to perform random image enhancement operations on the real image to obtain a source image and a target image with differences, further, the source-target image generator resizes and rotates the source image to make the resolution and size of the generated source image and the target image consistent, and the image enhancement operation includes adjusting the hue, saturation and brightness of the real image,

[0060] S1012: extracting a face region from a real image, detecting facial feature points of the face region using a Dlib or RetinaFace key point detection algorithm, wherein the facial feature points at least include eyes, nose, mouth, eyebrows, and facial contours, and calculating a convex hull of the face region based on the facial feature points.

[0061] S1013: Use the mask generator to initialize a basic mask through the convex hull. The basic mask uses the convex hull of the facial area as the shape outline. The basic mask is a grayscale image with a grayscale value range between [0.0, 1.0]. The basic mask defines the basic distribution of the mixed area with the convex hull shape outline.

[0062] S1014: using the Face X-ray method to randomly deform the shape of the basic mask, wherein the deformation includes random expansion, boundary adjustment, and translation, to generate mask images of different shapes.

[0063] S1015: Use two Gaussian filters with different parameters to smooth the boundary of the mask image, eliminate the abrupt transition of the boundary of the mask image, and make the gray value distribution of the mask image more natural.

[0064] S1016: Use alpha blending model to obtain a mixed image:

[0065]

[0066] ☉ means element-wise multiplication, I s is the generated source image, I t is the target image, R is the dynamic adjustment coefficient, M is the smoothed mask image generated by the mask generator, the grayscale value range of the mask image is [0.0, 1.0], M is used to control the proportion of the source image and the target image in the mixed area, when all pixels are limited to 0 and 1, I is adjusted before mixing s Use color correction technology to make its color consistent with I t Color matching, changing the mixing ratio of the source image and the target image by R, generates a mask with diverse grayscale value distributions, wherein the color correction technology is color transfer, gamma correction or color mapping, R∈(0,1].

[0067] The forged artifact detection module also includes an additional forged artifact detection layer and a forged artifact detection end layer. The resolution of the mixed image input to the forged artifact detection module gradually decreases. The forged artifact detection module performs a series of convolution, activation and downsampling operations on the mixed image to deeply refine and enhance the features of the mixed image, effectively improving the model's ability to express the mixed artifact area.

[0068] The present invention generates diversified artifact mixed images through self-mixing technology, which can not only truly simulate forged features, but also reduce the dependence on real forged data, and improve the quality and diversity of training data. Secondly, the artifact detection module accurately captures the fine-grained features of the mixed artifact area through multiple iterations of feature extraction and artifact space collaborative attention mechanism, and enhances the model's perception of the artifact area. Thirdly, the local artifact features are fused with the global features extracted by the backbone network through residual connection, which effectively combines the global and local feature information and achieves higher detection accuracy.

[0069] In addition, the use of the pre-trained EfficientNet-b3 backbone network improves training efficiency, while the modular design facilitates expansion and optimization. The present invention demonstrates strong generalization capabilities in cross-dataset evaluation and is suitable for complex and diverse counterfeit detection scenarios, with broad application prospects and practical value.

[0070] Embodiment 2: Combining with Figure 1, Attachment Figure 2 In addition to the contents of the above embodiments, the forged artifact detection module also includes a forged artifact detection additional layer, a forged artifact detection end layer and an artifact space collaborative attention module.

[0071] The operation steps of the forgery artifact detection module are as follows:

[0072] S1021: Take the mixed image extracted by the backbone network EfficientNet-b3 as input,

[0073] S1022: Process the mixed image through an additional layer of artifact detection: Use 1×1 convolution to perform channel transformation on the mixed image to highlight potential artifact information in a local range, then perform batch normalization and ReLU activation on the output of the 1×1 convolution. Normalization reduces internal covariate shift, making it easier for the model to converge and maintain training stability when learning complex forged features. Subsequently, 3×3 convolution is applied to the processed mixed image to extract more fine-grained local artifact features. After batch normalization again, the resolution of the mixed image is reduced by downsampling, so that subsequent layers can better focus on the local mixed area and alleviate the gradient vanishing problem. The processed mixed image is output.

[0074] S1023: After the artifact detection extra layer processing, an artifact space collaborative attention module is introduced to weight the spatial and channel information of the mixed image through the artifact space collaborative attention module to highlight the inconsistency of the mixed boundary of the mixed image, and further enhance the sensitivity of the model to artifact features.

[0075] S1024: Repeat steps S1022-S1023 twice, and output the processed mixed image.

[0076] S1025: First, use 3×3 convolution to adjust the number of channels without changing the spatial dimension of the mixed image, so that the features are more suitable for subsequent calculations. At the same time, ReLU activation is used to enhance the nonlinear expression ability of the network. Then, use 1×1 convolution to finely integrate the channels, and integrate the local artifact features extracted by the three artifact detection extra layers. Finally, downsampling is performed again to make the output mixed image size consistent with the input mixed image size in step S1021.

[0077] The artifact space collaborative attention module implements the following steps:

[0078] S201: In order to allow the network to distinguish the difference between local mixed artifact areas and local real areas at a finer level, the channel attention is decomposed into two parallel one-dimensional feature encoding processes by reducing the position information loss caused by two-dimensional global pooling, and then the two one-dimensional feature encodings are averaged pooled. Along the vertical and horizontal input channels, the two spatial ranges of the pooling kernel (H, 1) or (1, W) are used to encode each channel along the horizontal coordinate and vertical coordinate respectively, where H and W represent the height and width of the mixed image, and the output of the c-th channel with a height of h is expressed as follows:

[0079]

[0080] h represents the position index of the mixed image in the vertical direction, i represents the position index of the mixed image in the horizontal direction, is the aggregated value of the relevant features of the cth channel with height h, i∈{0,1,2...W-1}, h∈{0,1,2...H-1}, x c (h,i) is the response value of the input mixed image to the intensity of the artifact area at the cth channel with height h and width i,

[0081] S202: is the aggregated value of the relevant features of the cth channel width w, The calculation formula is as follows:

[0082]

[0083] x c (j,w) is the response value of the input mixed image at the cth channel with height h and width i,

[0084] S203: and The cascade operation in the spatial dimension obtains Using a 1×1 convolution F1 Perform linear transformation and dimension adaptation to obtain

[0085] S204:

[0086] Yes Perform sigmoid nonlinear activation function processing, all elements in f are within the real number domain, c is the number of channels,

[0087] S205: Split f into f in the spatial dimension h and f w , f h Input 1×1 convolution F hAnd after passing the Sigmoid activation function, the output is g h , f w Input 1×1 convolution F w And after passing the Sigmoid activation function, the output is g w :

[0088] g h =σ(F h (f h )),

[0089] g w =σ(F w (f w )),

[0090] Among them, f w f is obtained by one-dimensional average pooling in the horizontal direction, the kernel size of the one-dimensional average pooling in the vertical direction is H×1, and the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the vertical direction from the mixed image. The kernel size of the one-dimensional average pooling in the horizontal direction is 1×W, and the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the horizontal direction from the mixed image, and f h All elements in are within the real number domain, and f w All elements in are in the real number domain, σ is the Sigmoid activation function, σ is used to normalize the output to the range of (0,1), f h It is obtained by performing one-dimensional average pooling in the vertical direction of f.

[0091] S206: g h The original dimension of is C×H. By broadcasting the width dimension W, the dimension becomes C×H×W, and the same height direction weight value can be applied to all width positions i in the mixed image.

[0092] g w The original dimension is C×W. By broadcasting the height dimension H, the dimension becomes C×H×W, and the same width direction weight value can be applied to all height positions h in the mixed image.

[0093] S207: Perform weight processing on each pixel in the mixed image, and output the weight processed mixed image:

[0094]

[0095] x c (i, j) is the response value of the intensity of the artifact area at the position of the input mixed image with a width of i and a height of j, y c(i, j) is the response value of the artifact area intensity at the position of width i and height j of the weighted mixed image, which integrates the attention information in the height and width directions, making the mixed image more focused on the artifact area.

[0096] The forged artifact detection module of the present invention greatly enhances the sensitivity of the model to mixed artifact areas through efficient feature extraction, precise artifact area detection and the introduction of attention mechanism, making forged image detection more accurate and robust. Combined with the global feature extraction of the backbone network EfficientNet-b3 and the local focus of collaborative attention in the artifact space, the performance of the model in the artifact detection task is further improved, especially when processing diverse and complex forged images.

[0097] Although the present invention has been described above with reference to various embodiments, it should be understood that many changes and modifications may be made without departing from the scope of the present invention. That is, the methods, systems and devices discussed above are examples. Various configurations may appropriately omit, replace or add various processes or components. For example, in alternative configurations, the method may be performed in an order different from that described, and / or various components may be added, omitted and / or combined. Moreover, the features described with respect to certain configurations may be combined in various other configurations, such as different aspects and elements of the configuration may be combined in a similar manner. In addition, as technology develops, the elements therein may be updated, i.e., many elements are examples and do not limit the scope of the present disclosure or claims. And it should be understood that after reading the contents of the record of the present invention, the technician may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A deep fake detection method based on self-mixing, characterized in that: Self-mixing based deepfake detection methods include: S101: First, the data set FF is divided into frames, and only the real images in the data set FF are input. The real images are used to generate mixed images with common forgery artifacts using the self-mixing technology, and the real images and their corresponding mixed images are input into the backbone network. The backbone network is EfficientNet-b3 pre-trained on ImageNet. S102: using the forged artifact detection module to locate and detect the mixed area in the mixed image, and output a mixed image that more prominently highlights the artifact area features and is easy to distinguish later. S103: The enhanced mixed image obtained after processing by the forged artifact detection module is residually connected to the global features obtained by global average pooling at the end of the backbone network EfficientNet-b3 to obtain fused features, and the fused features are passed to the final fully connected layer of the backbone network for decision-making, thereby generating an accurate judgment on the authenticity of the input image.

2. The self-mixing based deep fake detection method according to claim 1, characterized in that: The specific steps of generating a mixed image in step S101 are as follows: S1011: generating a source image and a target image pair by a source-target image generator, wherein the source-target image generator is used to perform random image enhancement operations on the real image to obtain a source image and a target image with differences, further, the source-target image generator resizes and rotates the source image to make the resolution and size of the generated source image and the target image consistent, and the image enhancement operation includes adjusting the hue, saturation and brightness of the real image, S1012: extracting a face region from a real image, detecting facial feature points of the face region using a Dlib or RetinaFace key point detection algorithm, wherein the facial feature points at least include eyes, nose, mouth, eyebrows, and facial contours, and calculating a convex hull of the face region based on the facial feature points. S1013: Use the mask generator to initialize a basic mask through the convex hull. The basic mask uses the convex hull of the facial area as the shape outline. The basic mask is a grayscale image with a grayscale value range between [0.0, 1.0]. The basic mask defines the basic distribution of the mixed area with the convex hull shape outline. S1014: using the Face X-ray method to randomly deform the shape of the basic mask, wherein the deformation includes random expansion, boundary adjustment, and translation, to generate mask images of different shapes. S1015: Use two Gaussian filters with different parameters to smooth the boundary of the mask image, eliminate the abrupt transition of the boundary of the mask image, and make the gray value distribution of the mask image more natural. S1016: Use alpha blending model to obtain a mixed image: ☉ means element-wise multiplication, I s is the generated source image, I t is the target image, R is the dynamic adjustment coefficient, M is the smoothed mask image generated by the mask generator, the grayscale value range of the mask image is [0.0, 1.0], M is used to control the proportion of the source image and the target image in the mixed area, when all pixels are limited to 0 and 1, I is adjusted before mixing s Use color correction technology to make its color consistent with I t Color matching, changing the mixing ratio of the source image and the target image by R, generates a mask with diverse grayscale value distributions, wherein the color correction technology is color transfer, gamma correction or color mapping, R∈(0,1].

3. The self-mixing based deep fake detection method according to claim 2, characterized in that: The forgery artifact detection module also includes a forgery artifact detection extra layer, a forgery artifact detection end layer and an artifact space co-attention module.

4. The self-mixing based deep fake detection method according to claim 3, characterized in that: The operation steps of the forgery artifact detection module are as follows: S1021: Take the mixed image extracted by the backbone network EfficientNet-b3 as input, S1022: Process the mixed image through an additional layer of artifact detection: Use 1×1 convolution to perform channel transformation on the mixed image to highlight potential artifact information in a local range, then perform batch normalization and ReLU activation on the output of the 1×1 convolution. Normalization reduces internal covariate shift, making it easier for the model to converge and maintain training stability when learning complex forged features. Subsequently, 3×3 convolution is applied to the processed mixed image to extract more fine-grained local artifact features. After batch normalization again, the resolution of the mixed image is reduced by downsampling, so that subsequent layers can better focus on the local mixed area and alleviate the gradient vanishing problem. The processed mixed image is output. S1023: After the additional layer of artifact detection processing, an artifact space collaborative attention module is introduced to weight the spatial and channel information of the mixed image through the artifact space collaborative attention module to highlight the inconsistency of the mixed boundary of the mixed image, and further enhance the sensitivity of the model to artifact features. S1024: Repeat steps S1022-S1023 twice, and output the processed mixed image. S1025: First, use 3×3 convolution to adjust the number of channels without changing the spatial dimension of the mixed image, so that the features are more suitable for subsequent calculations. At the same time, ReLU activation is used to enhance the nonlinear expression ability of the network. Then, use 1×1 convolution to finely integrate the channels, and integrate the local artifact features extracted from the three artifact detection extra layers. Finally, downsampling is performed again to make the output mixed image size consistent with the input mixed image size in step S1021.

5. The self-mixing based deep fake detection method according to claim 4, characterized in that: The artifact space collaborative attention module implements the following steps: S201: In order to allow the network to distinguish the difference between local mixed artifact areas and local real areas at a finer level, the channel attention is decomposed into two parallel one-dimensional feature encoding processes by reducing the position information loss caused by two-dimensional global pooling, and then the two one-dimensional feature encodings are averaged pooled. Along the vertical and horizontal input channels, the two spatial ranges of the pooling kernel (H, 1) or (1, W) are used to encode each channel along the horizontal coordinate and vertical coordinate respectively, where H and W represent the height and width of the mixed image, and the output of the c-th channel with a height of h is expressed as follows: h represents the position index of the mixed image in the vertical direction, i represents the position index of the mixed image in the horizontal direction, is the aggregated value of the relevant features of the c-th channel with height h, i∈{0,1,2...W-1}, h∈{0,1,2...H-1}, x c (h,i) is the response value of the input mixed image to the intensity of the artifact area at the cth channel with height h and width i, S202: is the aggregated value of the relevant features of the cth channel width w, The calculation formula is as follows: x c (j,w) is the response value of the input mixed image with height h and width i at the cth channel, S203: and The cascade operation in the spatial dimension obtains Using a 1×1 convolution F1 Perform linear transformation and dimension adaptation to obtain S204： Yes Perform sigmoid nonlinear activation function processing, all elements in f are within the real number domain, c is the number of channels, S205: Split f into f in the spatial dimension h and f w , f h Input 1×1 convolution F h And after passing the Sigmoid activation function, the output is g h , f w Input 1×1 convolution F w And after passing the Sigmoid activation function, the output is g w : g h =σ(F h (f h )), g w =σ(F w (f w )), Among them, f w f is obtained by one-dimensional average pooling in the horizontal direction, the kernel size of the one-dimensional average pooling in the vertical direction is H×1, and the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the vertical direction from the mixed image. The kernel size of the one-dimensional average pooling in the horizontal direction is 1×W, and the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the horizontal direction from the mixed image, and f h All elements in are within the real number domain, and f w All elements in are in the real number domain, σ is the Sigmoid activation function, σ is used to normalize the output to the range of (0,1), f h It is obtained by performing one-dimensional average pooling in the vertical direction of f. S206: g h The original dimension of is C×H. By broadcasting the width dimension W, the dimension becomes C×H×W, and the same height direction weight value can be applied to all width positions i in the mixed image. g w The original dimension is C×W. By broadcasting the height dimension H, the dimension becomes C×H×W, and the same width direction weight value can be applied to all height positions h in the mixed image. S207: Perform weight processing on each pixel in the mixed image, and output the weight processed mixed image: x c (i, j) is the response value of the intensity of the artifact area at the position of the input mixed image with a width of i and a height of j, y c (i, j) is the response value of the artifact area intensity at the position of width i and height j of the weighted mixed image, which integrates the attention information in the height and width directions, making the mixed image more focused on the artifact area.

Images