A self-mixing based deepfake detection method

Abstract

The application provides a kind of based on self-mixing depth forgery detection method, compared with prior art, the application generates mixed image containing forgery artifact by using self-mixing technology, and combines EfficientNet-b3 and forgery artifact detection module, the artifact area of input image is accurately detected and identified.First, the real image is frame processed, the source image and target image are generated using self-mixing technology, and the mixed image with forgery artifact is generated by mask generator combined with alpha blending model.Subsequently, the backbone network extracts global features from the mixed image, and the forgery artifact detection module strengthens the local feature representation of the mixed area through the artifact space collaborative attention module, so as to accurately locate the artifact area.Finally, the global and local features are fused by using residual connection, and are input into the fully connected layer to complete classification.This method can effectively detect various types of forged images and exhibit excellent generalization ability in cross-dataset and unknown scenarios.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a deep fake detection method based on self-mixing. Background Art

[0002] In recent years, the rapid development of generative technology has led to the widespread use of forged images on the internet, making face forgery detection a hot research area. Current detection methods can be categorized as those based on single-frame images and those based on multiple frames. Traditional digital image processing-based methods primarily detect differences at the pixel level, effectively identifying GAN-generated facial images by analyzing the pixel deviations between synthetic and real images. However, with advances in generative technology, modern GAN-generated images are increasingly difficult to distinguish from real images at the pixel level. Existing image forgery detection methods are limited in their effectiveness when dealing with forgeries that only involve facial manipulation. They also perform poorly on low-quality or highly compressed images, making it difficult to detect artifacts.

[0003] Our experimental team has been browsing and researching a large amount of relevant records and materials for the related technologies of forged image recognition for a long time. At the same time, we rely on relevant resources and conduct a large number of related experiments. After a lot of searches, we found that there are existing technologies such as CN112686331B, CN112149608B, and CN116580409B. , and CN111860314B, as disclosed in the prior art, an image recognition method, comprising: obtaining an image to be recognized; using a trained neural network to recognize the image and determine whether the image is a forged image; wherein the neural network comprises a generator and a classifier constituting a generative adversarial network and is trained using the following steps: obtaining an original image group comprising a plurality of original images and a label of each original image, the original images comprising a real image and a forged image corresponding to the real image; for each original image, obtaining a first type of noise for the original image using the classifier to construct an associated first noisy image, and obtaining a second type of noise for the original image using the generative adversarial network to construct an associated second noisy image; establishing a training set based on the original image and its associated first noisy image and second noisy image; and training the neural network based on the training set to obtain parameters of the neural network.

[0004] The present invention is made to solve the common problems in this field, such as the detection methods are highly targeted but lack generalization ability; the detection effect of low-quality or highly compressed images is poor; the complexity is high and relies on large-scale data training, etc. Summary of the Invention

[0005] The purpose of the present invention is to address the shortcomings of the current field and propose a deep fake detection method based on self-mixing.

[0006] In order to overcome the deficiencies of the prior art, the present invention adopts the following technical solutions:

[0007] A deep fake detection method based on self-mixing, the deep fake detection method based on self-mixing comprising:

[0008] S101: First, the dataset FF is divided into frames. Only the real images in the dataset FF are input. The real images are used to generate mixed images with common forgery artifacts using the self-mixing technology. The real images and their corresponding mixed images are input into the backbone network. The backbone network is EfficientNet-b3 pre-trained on ImageNet.

[0009] S102: Using the forgery artifact detection module to locate and detect the mixed area in the mixed image, output a mixed image that more prominently features the artifact area and is easier to identify later.

[0010] S103: The enhanced mixed image obtained after processing by the forgery artifact detection module is residually connected to the global features obtained by global average pooling at the end of the backbone network EfficientNet-b3 to obtain fused features, and the fused features are passed to the final fully connected layer of the backbone network for decision-making, thereby generating an accurate judgment on the authenticity of the input image.

[0011] Furthermore, the specific steps of generating the mixed image in step S101 are as follows:

[0012] S1011: generating a source image and a target image pair by a source-target image generator, wherein the source-target image generator is used to perform random image enhancement operations on the real image to obtain a source image and a target image with differences. Further, the source-target image generator resizes and rotates the source image to make the resolution and size of the generated source image and the target image consistent. The image enhancement operations include adjusting the hue, saturation, and brightness of the real image.

[0013] S1012: Extracting a face region from a real image, detecting facial feature points of the face region using a Dlib or RetinaFace key point detection algorithm, wherein the facial feature points include at least eyes, nose, mouth, eyebrows, and facial contours, and calculating the convex hull of the face region based on the facial feature points.

[0014] S1013: Use the mask generator to initialize a basic mask using the convex hull. The basic mask uses the convex hull of the facial area as the shape outline. The basic mask is a grayscale image with a grayscale value range of [0.0, 1.0]. The basic mask defines the basic distribution of the mixed area with the convex hull shape outline.

[0015] S1014: Using the Face X-ray method, randomly deforming the shape of the basic mask, the deformation includes random expansion, boundary adjustment and translation, to generate mask images of different shapes.

[0016] S1015: Use two Gaussian filters with different parameters to smooth the boundaries of the mask image, eliminate the abrupt transition of the mask image boundary, and make the grayscale value distribution of the mask image more natural.

[0017] S1016: Use alpha blending to obtain a mixed image:

[0018] ,

[0019] ☉ means element-wise multiplication, is the generated source image, is the target image, R is the dynamic adjustment coefficient, M is the smoothed mask image generated by the mask generator, the grayscale value range of the mask image is [0.0, 1.0], M is used to control the proportion of the source image and the target image in the mixed area, when all pixels are limited to 0 and 1, before mixing Use color correction technology to make its color consistent with Color matching, changing the mixing ratio of the source image and the target image by R, generates a mask with diverse grayscale value distribution, wherein the color correction technology is color transfer, gamma correction or color mapping, R∈(0,1].

[0020] Furthermore, the forgery artifact detection module also includes a forgery artifact detection extra layer, a forgery artifact detection end layer and an artifact space collaborative attention module.

[0021] Furthermore, the operation steps of the forgery artifact detection module are as follows:

[0022] S1021: Take the mixed image extracted by the backbone network EfficientNet-b3 as input,

[0023] S1022: The mixed image is processed by the artifact detection additional layer: a 1x1 convolution is used for channel transformation on the mixed image to highlight potential artifact information in a local range, followed by batch normalization and ReLU activation on the output of the 1x1 convolution, which reduces the internal covariate shift to make the model more easily convergent and maintain training stability when learning complex fake features, then a 3x3 convolution is applied to the processed mixed image to extract more fine-grained local artifact features, and after batch normalization again, the resolution of the mixed image is reduced by downsampling to enable the subsequent layer to better focus on the local mixed area and alleviate the gradient vanishing problem, and the processed mixed image is output,

[0024] S1023: After the artifact detection additional layer processing, an artifact spatial collaborative attention module is introduced, which weights the spatial and channel information of the mixed image to highlight the inconsistency of the mixed boundary of the mixed image and further enhance the sensitivity of the model to artifact features,

[0025] S1024: Steps S1022-S1023 are repeated twice to output the processed mixed image,

[0026] S1025: First, a 3x3 convolution is used to adjust the channel number without changing the spatial dimension of the mixed image to make the features more suitable for subsequent calculations, and the non-linear expression ability of the network is enhanced through ReLU activation, then a 1x1 convolution is used to fine integrate the channels, and the three local artifact features extracted by the artifact detection additional layer are fused into one, and finally the output mixed image size is consistent with the input mixed image size in step S1021 through the downsampling operation again.

[0027] Further, the artifact spatial collaborative attention module implements the following steps:

[0028] S201: In order to let the network distinguish the difference between the local mixed artifact area and the local real area at a more fine-grained level, the channel attention is decomposed into two parallel one-dimensional feature encoding processes by reducing the position information loss caused by two-dimensional global pooling, and then the two one-dimensional feature encodings are averaged pooled along the vertical and horizontal directions of the input channels, using the two spatial ranges (H, 1) or (1, W) of the pooling kernel to encode each channel along the horizontal and vertical coordinates, respectively, where H and W represent the height and width of the mixed image, and the output of the c-th channel with a height of h is represented as follows:

[0029]

[0030] h represents the position index of the mixed image in the vertical direction, and i represents the position index of the mixed image in the horizontal direction, ​is the aggregated value of the relevant features of the c-th channel with height h, is the response value of the input mixed image to the intensity of the artifact area at the cth channel with a height of h and a width of i,

[0031] S202: is the aggregated value of the relevant features of the c-th channel width w, The calculation formula is as follows: ,

[0032] is the response value of the input mixed image with a height of h and a width of i in the cth channel,

[0033] S203: and The cascade operation in the spatial dimension obtains [ , ], using a 1×1 convolution F1 to [ , ] linear transformation and dimension adaptation to obtain ,

[0034] S204: ,

[0035] Yes Perform sigmoid nonlinear activation function processing, all elements in f are within the real number domain, c is the number of channels,

[0036] S205: Split f into and ,Will Input 1×1 convolution F h And output after Sigmoid activation function ,Will Input 1×1 convolution F w And output after Sigmoid activation function :

[0037] ,

[0038] ,

[0039] Among them, f wf is obtained by one-dimensional average pooling in the horizontal direction, the kernel size of the one-dimensional average pooling in the vertical direction is H×1, the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the vertical direction from the mixed image, the kernel size of the one-dimensional average pooling in the horizontal direction is 1×W, the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the horizontal direction from the mixed image, and f h All elements in are within the real number domain, and f w All elements in are within the real number domain, σ is the Sigmoid activation function, σ is used to normalize the output to the (0,1) range, f h Obtained by performing one-dimensional average pooling in the vertical direction of f,

[0040] S206: The original dimension of is C×H, and by broadcasting the width dimension W, the dimension becomes C×H×W, and the same height direction weight value is obtained for all width positions i in the mixed image. ,

[0041] The original dimension is C×W. By broadcasting the height dimension H, the dimension becomes C×H×W, and the same width direction weight value can be applied to all height positions h in the mixed image. ,

[0042] S207: Perform weight processing on each pixel in the mixed image and output the weighted mixed image:

[0043] ,

[0044] The response value of the artifact area intensity at the position of the input mixed image with a width of i and a height of j is: It is the response value of the intensity of the artifact area at the position of width i and height j of the weighted mixed image, which integrates the attention information in the height and width directions, making the mixed image more focused on the artifact area.

[0045] The beneficial effects achieved by the present invention are:

[0046] 1. Generate diverse mixed artifact images through self-mixing technology, realistically simulate forged features, reduce dependence on real forged data, effectively improve the quality and diversity of training data, and provide more comprehensive artifact samples for model training.

[0047] 2. The artifact detection module uses multiple iterations of artifact detection layers and final layers, combined with convolution and attention mechanisms, to accurately capture the fine-grained features of mixed artifact areas. This enhances the model's perception of artifact areas, making detection more accurate and efficient.

[0048] 3. The global features extracted by the backbone network EfficientNet-b3 and the local artifact features extracted by the artifact detection module are fused through residual connections to form a comprehensive feature representation, which effectively improves the detection performance of complex artifact scenes.

[0049] 4. The artifact space collaborative attention module captures the inconsistency of the boundaries within the mixed image, improving the model's adaptability to unknown forged images and complex mixed scenes, making artifact detection more robust and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but rather the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0051] Figure 1 This is a schematic diagram of the process of generating a mixed image in step S101 of the present invention.

[0052] Figure 2 Schematic diagram of the process of the forgery artifact detection module of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with its embodiments; it should be pointed out that the specific embodiments described herein are only used to explain the present invention and are not used to limit this case. For those skilled in the art, after reviewing the following detailed description, other systems, methods and / or features of this embodiment will become apparent. In addition, the terms describing the positional relationship in the accompanying drawings are only for illustrative purposes and cannot be understood as limiting this patent. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to the specific circumstances.

[0054] Example 1: Combined with the attached Figure 1 and attached Figure 2 This embodiment constructs a deep fake detection method based on self-mixing, which includes:

[0055] S101: First, the dataset FF is divided into frames. Only the real images in the dataset FF are input. The real images are used to generate mixed images with common forgery artifacts using the self-mixing technology. The real images and their corresponding mixed images are input into the backbone network. The backbone network is EfficientNet-b3 pre-trained on ImageNet.

[0056] S102: Using the forgery artifact detection module to locate and detect the mixed area in the mixed image, output a mixed image that more prominently features the artifact area and is easier to identify later.

[0057] S103: The enhanced mixed image obtained after processing by the forgery artifact detection module is residually connected to the global features obtained by global average pooling at the end of the backbone network EfficientNet-b3 to obtain fused features, and the fused features are passed to the final fully connected layer of the backbone network for decision-making, thereby generating an accurate judgment on the authenticity of the input image.

[0058] The specific steps of generating the mixed image in step S101 are as follows:

[0059] S1011: generating a source image and a target image pair by a source-target image generator, wherein the source-target image generator is used to perform random image enhancement operations on the real image to obtain a source image and a target image with differences. Further, the source-target image generator resizes and rotates the source image to make the resolution and size of the generated source image and the target image consistent. The image enhancement operations include adjusting the hue, saturation, and brightness of the real image.

[0060] S1012: Extracting a face region from a real image, detecting facial feature points of the face region using a Dlib or RetinaFace key point detection algorithm, wherein the facial feature points include at least eyes, nose, mouth, eyebrows, and facial contours, and calculating the convex hull of the face region based on the facial feature points.

[0061] S1013: Use the mask generator to initialize a basic mask using the convex hull. The basic mask uses the convex hull of the facial area as the shape outline. The basic mask is a grayscale image with a grayscale value range of [0.0, 1.0]. The basic mask defines the basic distribution of the mixed area with the convex hull shape outline.

[0062] S1014: Using the Face X-ray method, randomly deforming the shape of the basic mask, the deformation includes random expansion, boundary adjustment and translation, to generate mask images of different shapes.

[0063] S1015: Use two Gaussian filters with different parameters to smooth the boundaries of the mask image, eliminate the abrupt transition of the mask image boundary, and make the grayscale value distribution of the mask image more natural.

[0064] S1016: Use alpha blending to obtain a mixed image:

[0065] ,

[0066] ☉ means element-wise multiplication, is the generated source image, is the target image, R is the dynamic adjustment coefficient, M is the smoothed mask image generated by the mask generator, the grayscale value range of the mask image is [0.0, 1.0], M is used to control the proportion of the source image and the target image in the mixed area, when all pixels are limited to 0 and 1, before mixing Use color correction technology to make its color consistent with Color matching, changing the mixing ratio of the source image and the target image by R, generates a mask with diverse grayscale value distribution, wherein the color correction technology is color transfer, gamma correction or color mapping, R∈(0,1].

[0067] The artifact detection module also includes an additional artifact detection layer and an artifact detection end layer. The resolution of the mixed image fed into the module gradually decreases. The module performs a series of convolution, activation, and downsampling operations on the mixed image to deeply refine and enhance its features, effectively improving the model's ability to represent mixed artifact areas.

[0068] The present invention generates diversified artifact mixed images through self-mixing technology, which can not only realistically simulate forged features, but also reduce the dependence on real forged data, and improve the quality and diversity of training data. Secondly, the artifact detection module accurately captures the fine-grained features of the mixed artifact area through multiple iterations of feature extraction and artifact space collaborative attention mechanism, thereby enhancing the model's perception of the artifact area. Thirdly, the local artifact features are fused with the global features extracted by the backbone network through residual connection, effectively combining global and local feature information to achieve higher detection accuracy.

[0069] In addition, the use of the pre-trained EfficientNet-b3 backbone network improves training efficiency, while the modular design facilitates expansion and optimization. The present invention demonstrates strong generalization capabilities in cross-dataset evaluations and is suitable for complex and diverse forgery detection scenarios, with broad application prospects and practical value.

[0070] Example 2: Combined with the attached Figure 1 , Attachment Figure 2 In addition to the contents of the above embodiments, the forged artifact detection module further includes a forged artifact detection additional layer, a forged artifact detection end layer and an artifact space collaborative attention module.

[0071] The operation steps of the forgery artifact detection module are as follows:

[0072] S1021: Take the mixed image extracted by the backbone network EfficientNet-b3 as input,

[0073] S1022: Process the mixed image through an additional layer of artifact detection: Use 1×1 convolution to perform channel transformation on the mixed image to highlight potential artifact information in a local range, then perform batch normalization and ReLU activation on the output of the 1×1 convolution. Normalization reduces internal covariate shift, making it easier for the model to converge and maintain training stability when learning complex forged features. Subsequently, 3×3 convolution is applied to the processed mixed image to extract more fine-grained local artifact features. After batch normalization again, the resolution of the mixed image is reduced by downsampling, so that subsequent layers can better focus on the local mixed area and alleviate the gradient vanishing problem. The processed mixed image is output.

[0074] S1023: After the additional layer of artifact detection processing, the artifact space collaborative attention module is introduced to weight the spatial and channel information of the mixed image through the artifact space collaborative attention module to highlight the inconsistency of the mixed boundary of the mixed image, further enhancing the model's sensitivity to artifact features.

[0075] S1024: Repeat steps S1022-S1023 twice, and output the processed mixed image.

[0076] S1025: First, use 3×3 convolution to adjust the number of channels without changing the spatial dimension of the mixed image, so that the features are more suitable for subsequent calculations. At the same time, the nonlinear expression ability of the network is enhanced through ReLU activation. Then, use 1×1 convolution to finely integrate the channels, and integrate the local artifact features extracted by the three additional layers of artifact detection. Finally, through the downsampling operation again, the output mixed image size is consistent with the input mixed image size in step S1021.

[0077] The artifact space collaborative attention module implements the following steps:

[0078] S201: In order to allow the network to distinguish the difference between local mixed artifact areas and local real areas at a finer level, by reducing the position information loss caused by two-dimensional global pooling, the channel attention is decomposed into two parallel one-dimensional feature encoding processes, and then the two one-dimensional feature encodings are averaged and pooled. Along the vertical and horizontal input channels, the two spatial ranges of the pooling kernel (H, 1) or (1, W) are used to encode each channel along the horizontal coordinate and vertical coordinate respectively, where H and W represent the height and width of the mixed image. The output of the c-th channel with height h is expressed as follows:

[0079] ,

[0080] h represents the position index of the mixed image in the vertical direction, i represents the position index of the mixed image in the horizontal direction, is the aggregated value of the relevant features of the c-th channel with height h, is the response value of the input mixed image to the intensity of the artifact area at the cth channel with a height of h and a width of i,

[0081] S202: is the aggregated value of the relevant features of the c-th channel width w, The calculation formula is as follows: ,

[0082] is the response value of the input mixed image with a height of h and a width of i in the cth channel,

[0083] S203: and The cascade operation in the spatial dimension obtains [ , ], using a 1×1 convolution F1 to [ , ] linear transformation and dimension adaptation to obtain ,

[0084] S204: ,

[0085] Yes Perform sigmoid nonlinear activation function processing, all elements in f are within the real number domain, c is the number of channels,

[0086] S205: Split f into and ,Will Input 1×1 convolution F h And output after Sigmoid activation function ,Will Input 1×1 convolution F w And output after Sigmoid activation function :

[0087] ,

[0088] ,

[0089] Among them, f w f is obtained by one-dimensional average pooling in the horizontal direction, the kernel size of the one-dimensional average pooling in the vertical direction is H×1, the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the vertical direction from the mixed image, the kernel size of the one-dimensional average pooling in the horizontal direction is 1×W, the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the horizontal direction from the mixed image, and f h All elements in are within the real number domain, and f w All elements in are within the real number domain, σ is the Sigmoid activation function, σ is used to normalize the output to the (0,1) range, f h Obtained by performing one-dimensional average pooling in the vertical direction of f,

[0090] S206: The original dimension of is C×H, and by broadcasting the width dimension W, the dimension becomes C×H×W, and the same height direction weight value is obtained for all width positions i in the mixed image. ,

[0091] The original dimension is C×W. By broadcasting the height dimension H, the dimension becomes C×H×W, and the same width direction weight value can be applied to all height positions h in the mixed image. ,

[0092] S207: Perform weight processing on each pixel in the mixed image and output the weighted mixed image:

[0093] ,

[0094] The response value of the artifact area intensity at the position of the input mixed image with a width of i and a height of j is: It is the response value of the intensity of the artifact area at the position of width i and height j of the weighted mixed image, which integrates the attention information in the height and width directions, making the mixed image more focused on the artifact area.

[0095] The forgery artifact detection module of the present invention greatly enhances the model's sensitivity to mixed artifact areas through efficient feature extraction, precise artifact area detection, and the introduction of an attention mechanism, making forged image detection more accurate and robust. Combining the global feature extraction of the backbone network EfficientNet-b3 with the local focus of collaborative attention in the artifact space, the model's performance in artifact detection tasks is further improved, especially when processing diverse and complex forged images.

[0096] Although the present invention has been described above with reference to various embodiments, it will be appreciated that many changes and modifications may be made without departing from the scope of the present invention. That is, the methods, systems, and devices discussed above are examples. Various configurations may omit, replace, or add various processes or components as appropriate. For example, in alternative configurations, the methods may be performed in an order different from that described, and / or various components may be added, omitted, and / or combined. Moreover, the features described with respect to certain configurations may be combined in various other configurations, such as different aspects and elements of the configurations may be combined in a similar manner. In addition, as technology develops, the elements therein may be updated, i.e., many elements are examples and do not limit the scope of the present disclosure or claims. It will also be appreciated that, after reading the contents of the present invention, a technician may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A deep fake detection method based on self-mixing, characterized in that Self-mixing-based deepfake detection methods include: S101: First, the dataset FF is divided into frames. Only the real images in the dataset FF are input. The real images are used to generate mixed images with common forgery artifacts using the self-mixing technology. The real images and their corresponding mixed images are input into the backbone network. The backbone network is EfficientNet-b3 pre-trained on ImageNet. S102: Using the forgery artifact detection module to locate and detect the mixed area in the mixed image, output a mixed image that more prominently features the artifact area and is easier to identify later. S103: The enhanced mixed image obtained after processing by the forgery artifact detection module is residually connected to the global features obtained by global average pooling at the end of the backbone network EfficientNet-b3 to obtain a fused feature. The fused feature is then passed to the final fully connected layer of the backbone network for decision-making, thereby generating an accurate judgment on the authenticity of the input image; The forgery artifact detection module also includes a forgery artifact detection extra layer, a forgery artifact detection end layer and an artifact space co-attention module; The operation steps of the forgery artifact detection module are as follows: S1021: Take the mixed image extracted by the backbone network EfficientNet-b3 as input, S1022: Process the mixed image through an additional layer of artifact detection: Use 1×1 convolution to perform channel transformation on the mixed image to highlight potential artifact information in a local range, then perform batch normalization and ReLU activation on the output of the 1×1 convolution. Normalization reduces internal covariate shift, making it easier for the model to converge and maintain training stability when learning complex forged features. Subsequently, 3×3 convolution is applied to the processed mixed image to extract more fine-grained local artifact features. After batch normalization again, the resolution of the mixed image is reduced by downsampling, so that subsequent layers can better focus on the local mixed area and alleviate the gradient vanishing problem. The processed mixed image is output. S1023: After the additional layer of artifact detection processing, the artifact space collaborative attention module is introduced to weight the spatial and channel information of the mixed image through the artifact space collaborative attention module to highlight the inconsistency of the mixed boundary of the mixed image, further enhancing the model's sensitivity to artifact features. S1024: Repeat steps S1022-S1023 twice, and output the processed mixed image. S1025: First, a 3×3 convolution is used to adjust the number of channels without changing the spatial dimension of the mixed image, so that the features are more suitable for subsequent calculations. At the same time, ReLU activation is used to enhance the nonlinear expression ability of the network. Then, a 1×1 convolution is used to finely integrate the channels, and the local artifact features extracted by the three artifact detection extra layers are integrated. Finally, a downsampling operation is performed again to make the output mixed image size consistent with the input mixed image size in step S1021. Among them, the alpha blending model is used to obtain a mixed image.

2. The self-mixing based deep fake detection method according to claim 1, characterized in that: The specific steps of generating the mixed image in step S101 are as follows: S1011: generating a source image and a target image pair by a source-target image generator, wherein the source-target image generator is used to perform random image enhancement operations on the real image to obtain a source image and a target image with differences. Further, the source-target image generator resizes and rotates the source image to make the resolution and size of the generated source image and the target image consistent. The image enhancement operations include adjusting the hue, saturation, and brightness of the real image. S1012: Extracting a face region from a real image, detecting facial feature points of the face region using a Dlib or RetinaFace key point detection algorithm, wherein the facial feature points include at least eyes, nose, mouth, eyebrows, and facial contours, and calculating the convex hull of the face region based on the facial feature points. S1013: Use the mask generator to initialize a basic mask using the convex hull. The basic mask uses the convex hull of the facial area as the shape outline. The basic mask is a grayscale image with a grayscale value range of [0.0, 1.0]. The basic mask defines the basic distribution of the mixed area with the convex hull shape outline. S1014: Using the Face X-ray method, randomly deforming the shape of the basic mask, the deformation includes random expansion, boundary adjustment and translation, to generate mask images of different shapes. S1015: Use two Gaussian filters with different parameters to smooth the boundaries of the mask image, eliminate the abrupt transition of the mask image boundary, and make the grayscale value distribution of the mask image more natural. S1016: Use alpha blending to obtain a mixed image: ； ☉ means element-wise multiplication, is the generated source image, is the target image, R is the dynamic adjustment coefficient, M is the smoothed mask image generated by the mask generator, the grayscale value range of the mask image is [0.0, 1.0], M is used to control the proportion of the source image and the target image in the mixed area, when all pixels are limited to 0 and 1, before mixing Use color correction technology to make its color consistent with Color matching, changing the mixing ratio of the source image and the target image by R, generates a mask with diverse grayscale value distribution, wherein the color correction technology is color transfer, gamma correction or color mapping, R∈(0,1].

3. The self-mixing based deep fake detection method according to claim 2, characterized in that: The artifact space collaborative attention module implements the following steps: S201: In order to allow the network to distinguish the difference between local mixed artifact areas and local real areas at a finer level, by reducing the position information loss caused by two-dimensional global pooling, the channel attention is decomposed into two parallel one-dimensional feature encoding processes, and then the two one-dimensional feature encodings are averaged and pooled. Along the vertical and horizontal input channels, the two spatial ranges of the pooling kernel (H, 1) or (1, W) are used to encode each channel along the horizontal coordinate and vertical coordinate respectively, where H and W represent the height and width of the mixed image. The output of the c-th channel with height h is expressed as follows: ， h represents the position index of the mixed image in the vertical direction, i represents the position index of the mixed image in the horizontal direction, is the aggregated value of the relevant features of the c-th channel with height h, is the response value of the input mixed image to the intensity of the artifact area at the cth channel with a height of h and a width of i, S202: is the aggregated value of the relevant features of the c-th channel width w, The calculation formula is as follows: , is the response value of the input mixed image with a height of h and a width of i in the cth channel, S203: and Cascade operations in the spatial dimension obtain [ , ], using a 1×1 convolution F1 to [ , ] linear transformation and dimension adaptation to obtain , S204： ， Yes Perform sigmoid nonlinear activation function processing, all elements in f are within the real number domain, c is the number of channels, S205: Split f into and ,Will Input 1×1 convolution F h And output after Sigmoid activation function ,Will Input 1×1 convolution F w And output after Sigmoid activation function : ， ， Among them, f w f is obtained by one-dimensional average pooling in the horizontal direction, the kernel size of the one-dimensional average pooling in the vertical direction is H×1, the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the vertical direction from the mixed image, the kernel size of the one-dimensional average pooling in the horizontal direction is 1×W, the one-dimensional average pooling in the vertical direction is used to extract and aggregate features along the horizontal direction from the mixed image, and f h All elements in are within the real number domain, and f w All elements in are within the real number domain, σ is the Sigmoid activation function, σ is used to normalize the output to the (0,1) range, f h Obtained by performing one-dimensional average pooling in the vertical direction of f, S206: The original dimension of is C×H, and by broadcasting the width dimension W, the dimension becomes C×H×W, and the same height direction weight value is obtained for all width positions i in the mixed image. , The original dimension is C×W. By broadcasting the height dimension H, the dimension becomes C×H×W, and the same width direction weight value can be applied to all height positions h in the mixed image. , S207: Perform weight processing on each pixel in the mixed image and output the weighted mixed image: ， The response value of the artifact area intensity at the position of the input mixed image with a width of i and a height of j is: It is the response value of the intensity of the artifact area at the position of width i and height j of the weighted mixed image, which integrates the attention information in the height and width directions, making the mixed image more focused on the artifact area.

Images