Face image tampering passive detection method and device, terminal equipment and storage medium

Abstract

The application is suitable for the technical field of image processing, and provides a face image tampering passive detection method and device, terminal equipment and storage medium, wherein the method comprises the following steps: acquiring a face image to be detected, and performing frequency domain enhancement processing and compressed noise suppression processing on the face image to be detected to obtain a first face image subjected to frequency domain enhancement and a second face image subjected to suppression of compressed noise; then, the first face image and the second face image are input into a face image tampering detection model for tampering detection to obtain a detection result; the detection result is used to prompt that the face image to be detected is a tampered image or a real image; and a feature extraction part of the face image tampering detection model comprises a multi-frequency channel attention mechanism module. The application can improve the detection accuracy of face image tampering.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a face image tampering passive detection method and device, a terminal equipment and a storage medium. BACKGROUND

[0002] With the rapid development of face synthesis technology, fake face images are increasingly realistic, and it is difficult for human eyes to identify whether the images have been tampered with. At the same time, due to the commercialization of tampering technology, the threshold for face synthesis is lowered, which brings serious social credibility and security problems. In addition, during the transmission of pictures, information loss will occur after image compression algorithm processing, which reduces the image quality, and the decline of image quality will cause the detection accuracy of the image tampering detection model to be low, making it difficult to distinguish the artificial artifacts in the pictures. SUMMARY

[0003] The present application provides a face image tampering passive detection method, device, terminal equipment and storage medium, which aims to improve the detection accuracy of face image tampering.

[0004] In a first aspect, the present application provides a face image tampering passive detection method, comprising:

[0005] obtaining a face image to be detected;

[0006] performing frequency domain enhancement processing and compression noise suppression processing on the face image to be detected to obtain a first face image enhanced in frequency domain and a second face image with suppressed image compression noise;

[0007] inputting the first face image and the second face image into a face image tampering detection model for tampering detection to obtain a detection result;

[0008] the detection result is used to prompt that the face image to be detected is a tampered image or a real image, and the feature extraction part of the face image tampering detection model comprises a multi-frequency channel attention mechanism module.

[0009] Optionally, the frequency domain enhancement processing on the face image to be detected to obtain the first face image enhanced in frequency domain and the second face image with suppressed image compression noise comprises:

[0010] converting the face image to be detected from an RGB color space to a YCbCr color space to obtain a Y channel sub-image, a Cb channel sub-image and a Cr channel sub-image;

[0011] performing DCT transformation on the Y channel sub-image, the Cb channel sub-image and the Cr channel sub-image respectively to obtain a first original DCT image corresponding to the Y channel sub-image, a second original DCT image corresponding to the Cb channel sub-image and a third original DCT image corresponding to the Cr channel sub-image;

[0012] The first face image enhanced in the frequency domain and the second face image suppressing image compression noise are obtained according to the first original DCT image, the second original DCT image and the third original DCT image.

[0013] Optionally, the first face image enhanced in the frequency domain is obtained according to the first original DCT image, the second original DCT image and the third original DCT image, and the method comprises the following steps of:

[0014] The first original DCT image, the second original DCT image and the third original DCT image are normalized respectively to obtain a first DCT weight image corresponding to the first original DCT image, a second DCT weight image corresponding to the second original DCT image and a third DCT weight image corresponding to the third original DCT image;

[0015] The first original DCT image, the second original DCT image and the third original DCT image are divided into n*n blocks respectively; wherein n is a power of 2, and n is greater than or equal to 8;

[0016] The first original DCT image, the second original DCT image and the third original DCT image are divided into n*n blocks respectively; wherein n is a power of 2, and n is greater than or equal to 8;

[0017] The first DCT weight image, the second DCT weight image and the third DCT weight image are added to the first frequency domain weight image, the second frequency domain weight image and the third frequency domain weight image respectively according to the color channel to obtain a first enhanced weight image corresponding to the first original DCT image, a second enhanced weight image corresponding to the second original DCT image and a third enhanced weight image corresponding to the third original DCT image;

[0018] The first face image enhanced in the frequency domain is obtained according to the first enhanced weight image, the second enhanced weight image and the third enhanced weight image.

[0019] Optionally, the first face image enhanced in the frequency domain is obtained according to the first enhanced weight image, the second enhanced weight image and the third enhanced weight image, and the method comprises the following steps of:

[0020] The first enhanced weight image, the second enhanced weight image and the third enhanced weight image are added to the first frequency domain weight image, the second frequency domain weight image and the third frequency domain weight image respectively according to the color channel to obtain a first enhanced weight image corresponding to the first original DCT image, a second enhanced weight image corresponding to the second original DCT image and a third enhanced weight image corresponding to the third original DCT image;

[0021] The inverse DCT transformation and the RGB conversion are sequentially performed on each of the first product, the second product and the third product, to obtain a first face image enhanced in a frequency domain.

[0022] Optionally, the second face image with the image compression noise suppressed is obtained according to the first original DCT image, the second original DCT image and the third original DCT image, and includes the following steps.

[0023] Each of the first original DCT image, the second original DCT image and the third original DCT image is divided into a first image region including high-frequency components and a second image region excluding the first region, and the first image region is subjected to overall normalization processing and the second image region is subjected to sliding window normalization processing, to obtain a fourth DCT weight image corresponding to the first original DCT image, a fifth DCT weight image corresponding to the second original DCT image and a sixth DCT weight image corresponding to the third original DCT image.

[0024] Each of the fourth DCT weight image, the fifth DCT weight image and the sixth DCT weight image is subjected to the inverse DCT transformation and the RGB conversion, to obtain the second face image with the image compression noise suppressed.

[0025] Optionally, the face image tampering detection model is an Xception classification model, an activation function of a feature extraction part of the Xception classification model is a parameterized ReLU activation function, all pooling layers of the feature extraction part except for a last pooling layer are detail-preserving pooling layers, and each separable convolution layer of the feature extraction part is connected with the multi-frequency channel attention mechanism module.

[0026] Optionally, each feature channel map of the multi-frequency channel attention mechanism module is processed by N different frequency components, and N is an integer that can be evenly divided by a channel number C of the multi-frequency channel attention mechanism module.

[0027] In a second aspect, an embodiment of the present application provides a face image tampering passive detection device, including:

[0028] A first acquisition module is configured to acquire a face image to be detected.

[0029] A second acquisition module is configured to perform frequency domain enhancement processing and compression noise suppression processing on the face image to be detected, to obtain a first face image enhanced in a frequency domain and a second face image with image compression noise suppressed.

[0030] The detection module is configured to input the first face image and the second face image into a face image tampering detection model to perform tampering detection, and obtain a detection result. The detection result is used to prompt that the face image to be detected is a tampered image or a real image. The feature extraction part of the face image tampering detection model comprises a multi-frequency channel attention mechanism module.

[0031] In a third aspect, an embodiment of the present application provides a terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the face image tampering passive detection method described above is implemented.

[0032] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the face image tampering passive detection method described above is implemented.

[0033] The above-mentioned scheme of the present application has the following beneficial effects:

[0034] In the embodiments of the present application, the face image to be detected is obtained, and the frequency domain enhancement processing and the compressed noise suppression processing are performed on the face image to be detected, to obtain the first face image subjected to the frequency domain enhancement and the second face image subjected to the compressed noise suppression. Then, the first face image and the second face image are input into the face image tampering detection model to perform tampering detection, and the detection result used to prompt that the face image to be detected is a tampered image or a real image is obtained. Since the frequency domain enhancement can enhance the spatial domain features, the first face image can enhance the spatial domain features of the face image to be detected and improve the important features of the face image to be detected. Meanwhile, since the second face image can reduce the influence of the high-frequency noise caused by the image compression algorithm, the face image tampering detection model can improve the detection accuracy of the face image tampering when performing the tampering detection on the face image to be detected based on the first face image and the second face image. In addition, since the feature extraction part of the face image tampering detection model comprises the multi-frequency channel attention mechanism module, the global average pooling is used in the multi-frequency channel attention mechanism module, the mean value of each channel in the feature channel is taken as the weight, and the content of interest is enhanced in a weighted manner to suppress the background features, so as to further improve the detection accuracy of the face image tampering.

[0035] Other beneficial effects of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0037] Figure 1 The flow chart of the face image tampering passive detection method provided by an embodiment of the present application is shown in the following figure.

[0038] Figure 2 The flow chart of the face image tampering passive detection method provided by an embodiment of the present application is shown in the following figure.

[0039] Figure 3 The flow chart of the face image tampering passive detection method provided by an embodiment of the present application is shown in the following figure.

[0040] Figure 4 The region division schematic diagram of the divided image when obtaining the second face image provided by an embodiment of the present application is shown in the following figure.

[0041] Figure 5 The structure schematic diagram of the feature extraction part of the face image tampering model provided by an embodiment of the present application is shown in the following figure.

[0042] Figure 6 The flow chart of the specific steps of the qualitative research provided by an embodiment of the present application is shown in the following figure.

[0043] Figure 7 The structure schematic diagram of the face image tampering passive detection device provided by an embodiment of the present application is shown in the following figure.

[0044] Figure 8 The structure schematic diagram of the terminal device provided by an embodiment of the present application is shown in the following figure. DETAILED DESCRIPTION

[0045] In the following description, specific details are set forth such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it should be apparent to those skilled in the art that the present application can be practiced without such specific details. In other instances, well-known systems, structures, circuits, and techniques have been not been described in detail in order to avoid obscuring the description of the present application.

[0046] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0047] It should also be understood that the term “and / or” as used herein refers to any combination of one or more of the associated listed items, and that includes all possible combinations, whether explicit or not.

[0048] As used in the description of the application and the appended claims, the term “if’ can be interpreted to mean “when” or “upon” or “in response to determining” or “in response to detecting” depending on the context. Similarly, the phrase “if it is determined” or “if [a described condition or event] is detected” can be interpreted to mean “upon determining” or “in response to determining” or “upon detecting [the described condition or event]” or “in response to detecting [the described condition or event],” depending on the context.

[0049] In addition, the description in the specification of the application and the appended claims uses the terms “first,” “second,” “third,” and the like merely to distinguish different features, and does not imply or suggest relative importance.

[0050] Reference in the specification to “one embodiment” or “some embodiments” or “an embodiment” or “some implementations” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases “in one embodiment” or “in some embodiments” or “in other embodiments” or “in still other embodiments” or the like in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily referring to some, but not all, embodiments, unless otherwise indicated by the context. The terms “including,” “comprising,” “having” and variations thereof mean “including but not limited to,” unless expressly specified otherwise.

[0051] At present, in the process of image transmission, the image compression algorithm will cause information loss and reduce the image quality. The decline of image quality will cause the detection accuracy of image tamper detection model to be low, and the artificial artifacts in the image are difficult to identify.

[0052] To solve the above problems, in the embodiment of the present application, the face image to be detected is acquired, and frequency domain enhancement processing and image compression noise suppression processing are performed on the face image to be detected to obtain a first face image enhanced in frequency domain and a second face image with suppressed image compression noise. Then, the first face image and the second face image are input into a face image tampering detection model for tampering detection to obtain a detection result for prompting the face image to be detected as a tampered image or a real image. Since the frequency domain enhancement can enhance the spatial domain features, the first face image can enhance the spatial domain features of the face image to be detected and improve the important features of the face image to be detected. Meanwhile, since the second face image can reduce the influence of high-frequency noise caused by the image compression algorithm, the face image tampering detection model can improve the detection accuracy of face image tampering when performing tampering detection on the face image to be detected based on the first face image and the second face image. In addition, since the feature extraction part of the face image tampering detection model includes a multi-frequency channel attention mechanism module, the global average pooling is used to take the mean value of each channel in the feature channel as the weight to enhance the content of interest and suppress the background features in a weighted manner, thereby further improving the detection accuracy of face image tampering.

[0053] The face image tampering passive detection method provided by the present application will be exemplarily described below in combination with specific embodiments.

[0054] The face image tampering passive detection method provided by the present application will be exemplarily described below in combination with specific embodiments.

[0055] As shown in Figure 1 The face image tampering passive detection method provided by the present application includes the following steps:

[0056] Step 101, acquiring a face image to be detected.

[0057] The face image to be detected is a face image that needs to be subjected to tampering detection. Specifically, the face image to be detected can be collected by a terminal device, or can be collected by other image collection devices and then sent to a terminal device.

[0058] Step 102, performing frequency domain enhancement processing and image compression noise suppression processing on the face image to be detected to obtain a first face image enhanced in frequency domain and a second face image with suppressed image compression noise.

[0059] In some instances of this application, since frequency domain enhancement can strengthen spatial domain features, the aforementioned first face image can enhance the spatial domain features of the face image to be detected and improve the important features of the face image to be detected. The aforementioned second face image is obtained based on the basic principles of two-dimensional discrete cosine transform and image compression algorithms (such as JPEG image compression algorithm), and this second face image can reduce the impact of high-frequency noise caused by image compression algorithms such as JPEG.

[0060] Step 103: Input the first face image and the second face image into the face image tampering detection model for tampering detection and obtain the detection result.

[0061] The above detection results are used to indicate whether the face image to be detected is a tampered image or a real image, and the feature extraction part of the above face image tampering detection model includes a multi-frequency channel attention mechanism module.

[0062] It is worth mentioning that, because the first face image enhances the spatial domain features and important features of the face image to be detected, while the second face image reduces the impact of high-frequency noise introduced by image compression algorithms, the face image tampering detection model can improve the detection accuracy when performing tampering detection on the face image to be detected based on the first and second face images. Furthermore, since the feature extraction part of the face image tampering detection model includes a multi-frequency channel attention mechanism module, which uses global average pooling and uses the mean of each channel in the feature channels as weights to enhance the content of interest and suppress background features in a weighted manner, the detection accuracy of face image tampering is further improved.

[0063] The following is an exemplary description of step 102, which involves performing frequency domain enhancement processing on the face image to be detected to obtain a first face image with frequency domain enhancement, using a specific embodiment as an example.

[0064] like Figure 2 As shown, in some embodiments of this application, the specific implementation of obtaining the first face image includes the following steps:

[0065] Step 201: Convert the face image to be detected from the RGB color space to the YCbCr color space to obtain the Y channel sub-image, Cb channel sub-image, and Cr channel sub-image.

[0066] Step 202: Perform DCT transformation on the Y channel sub-image, Cb channel sub-image, and Cr channel sub-image respectively to obtain the first original DCT image corresponding to the Y channel sub-image, the second original DCT image corresponding to the Cb channel sub-image, and the third original DCT image corresponding to the Cr channel sub-image.

[0067] The DCT transformation is a discrete cosine transform (DCT).

[0068] At step 203, the first original DCT image, the second original DCT image and the third original DCT image are normalized respectively to obtain a first DCT weight image corresponding to the first original DCT image, a second DCT weight image corresponding to the second original DCT image and a third DCT weight image corresponding to the third original DCT image.

[0069] Specifically, the first DCT weight image is Y NDCT , the second DCT weight image is Cb NDCT , and the third DCT weight image is Cr NDCT .

[0070] The normalization can be a sliding window normalization (SWN). The sliding window normalization refers to dividing the original matrix into smaller matrices and performing local normalization from left to right and from top to bottom in order. For example, if the size of the original matrix is WxH (W is the size of the original matrix in the horizontal direction, and H is the size of the original matrix in the vertical direction), the original matrix is gridded according to an 8x8 grid, and the size of each sub-region is (W / 8)x(H / 8), containing (WxH) / 64 sample values. The sample values are normalized in the maximum value normalization manner. For a data sample X, the expression is as follows:

[0071]

[0072] wherein x min and x max are the minimum value and the maximum value in each group of sample values, x is the original data value to be normalized, and x scale is the normalized data value.

[0073] At step 204, the first original DCT image, the second original DCT image and the third original DCT image are divided into n x n blocks respectively.

[0074] wherein n is a power of 2, and n is greater than or equal to 8.

[0075] As a preferred example, the first original DCT image, the second original DCT image and the third original DCT image are divided into 8x8 blocks respectively.

[0076] Step 205, for each of the first original DCT image, the second original DCT image and the third original DCT image, respectively, normalizing each block in the original DCT image to obtain a frequency domain matrix weight containing n x n values, and through matrix copying and matrix expansion, obtaining a frequency domain weight image corresponding to the original DCT image.

[0077] Among them, the frequency domain weight image corresponding to the first original DCT image is the first frequency domain weight image, the frequency domain weight image corresponding to the second original DCT image is the second frequency domain weight image, and the frequency domain weight image corresponding to the third original DCT image is the third frequency domain weight image.

[0078] Specifically, the first frequency domain weight image is Y F_weight , the second frequency domain weight image is Cb F_weight , and the third frequency domain weight image is Cr F_weight .

[0079] Step 206, adding the first DCT weight image, the second DCT weight image and the third DCT weight image to the first frequency domain weight image Y, the second frequency domain weight image and the third frequency domain weight image according to the color channel correspondence to obtain the first enhanced weight image corresponding to the first original DCT image, the second enhanced weight image corresponding to the second original DCT image and the third enhanced weight image corresponding to the third original DCT image.

[0080] Specifically, the first enhanced weight image is f Y , the second enhanced weight image is f Cb , and the third enhanced weight image is f Cr .

[0081] Step 207, calculating the first product of the first enhanced weight image and the Y channel sub-image, the second product of the second enhanced weight image and the Cb channel sub-image, and the third product of the third enhanced weight image and the Cr channel sub-image.

[0082] Step 208, for each of the first product, the second product and the third product, respectively, sequentially performing inverse DCT transformation and RGB conversion on the product to obtain the frequency domain enhanced first face image.

[0083] That is, sequentially performing inverse DCT transformation and RGB conversion (which means conversion to RGB color space) on the first product, sequentially performing inverse DCT transformation and RGB conversion on the second product, and sequentially performing inverse DCT transformation and RGB conversion on the third product, that is, the first face image X FE .

[0084] It should be noted that the RGB conversion in the embodiments of the present application means conversion to the RGB color space.

[0085] The step 102 of suppressing compression noise of the face image to be detected is exemplarily described below in combination with specific embodiments.

[0086] As shown in the figure, Figure 3 the specific implementation of obtaining the second face image in some embodiments of the present application includes the following steps:

[0087] Step 301: converting the face image to be detected from an RGB color space to a YCbCr color space to obtain a Y channel sub-image, a Cb channel sub-image, and a Cr channel sub-image.

[0088] Step 302: performing DCT transformation on the Y channel sub-image, the Cb channel sub-image, and the Cr channel sub-image respectively to obtain a first original DCT graph corresponding to the Y channel sub-image, a second original DCT graph corresponding to the Cb channel sub-image, and a third original DCT graph corresponding to the Cr channel sub-image.

[0089] Step 303: for each original DCT graph in the first original DCT graph, the second original DCT graph, and the third original DCT graph, dividing the original DCT graph into a first image region including high frequency components and a second image region other than the first region, and performing overall normalization processing on the first image region and sliding window normalization processing on the second image region to obtain a fourth DCT weight graph corresponding to the first original DCT graph, a fifth DCT weight graph corresponding to the second original DCT graph, and a sixth DCT weight graph corresponding to the third original DCT graph.

[0090] Specifically, as shown in the figure, Figure 4 an original DCT graph can be divided into a region A (i.e. the second image region mentioned above) and a region B (i.e. the first image region mentioned above), the region B is a small window in the lower right corner of the matrix, the window size is 1 / 4 of the height and width of the original image, representing the high frequency components in the graph, and the region A is the remaining part; then the region A is subjected to sliding window normalization processing, and the region B is subjected to overall normalization (i.e. the normalization range of the region B is larger), so as to obtain the normalized DCT weight graph.

[0091] Specifically, the first original DCT graph is subjected to the above operation to obtain the fourth DCT weight graph Y NDCT The second original DCT graph is subjected to the above operation to obtain the fifth DCT weight graph Cb NDCT The third original DCT graph is subjected to the above operation to obtain the sixth DCT weight graph Cr NDCT .

[0092] Step 304: For each DCT weight map in the fourth, fifth, and sixth DCT weight maps, perform inverse DCT transformation and RGB conversion sequentially to obtain a second face image with suppressed image compression noise.

[0093] That is, sequentially weight the fourth DCT map Y NDCT Perform inverse DCT transformation and RGB conversion, and then sequentially adjust the weight map Cb of the fifth DCT. NDCT Perform inverse DCT transformation and RGB conversion, and then sequentially adjust the weight map Cr of the sixth DCT. NDCT Performing inverse DCT transform and RGB conversion yields the second face image X, which suppresses image compression noise. NS .

[0094] The following is an exemplary description of the face image tampering detection model with reference to specific embodiments.

[0095] In some embodiments of this application, the face image tampering detection model is an Xception classification model. The activation function of the feature extraction part of the Xception classification model is a parameterized ReLU (PReLU) activation function. All pooling layers in the feature extraction part except the last pooling layer are detail-preserving pooling layers. Each separable convolutional layer in the feature extraction part is followed by a multi-frequency channel attention mechanism module.

[0096] In some embodiments of this application, the aforementioned face image tampering detection model can be constructed based on the overall framework of the classification network Xception, which can be called a binary classification face forensics network (FENet). In related technologies, the classification network Xception is divided into three units (flows): feature extraction (Entry), optimization (Middle), and summarization (Exit). The feature extraction unit is used to downsample the input image to reduce spatial dimensionality and extract rich spatial and channel features; the optimization unit is used to further learn and optimize features; and the summarization unit is used to summarize and organize the previously extracted features. It should be noted that in some embodiments of this application, the aforementioned face image tampering detection model is obtained mainly by improving the feature extraction part.

[0097] like Figure 5 As shown, FENet takes two images as input for the feature extraction part: the first face image X. FE Second face image X NS X FEThe image input is mainly (Main) Flow, in which a multi-frequency channel attention mechanism module (which can be an adaptive multi-frequency channel attention (AFCA) module) is added after each separable convolution (Separable Conv) layer and activation function. The image is input as a matrix into the separable convolution layer and the feature map after the activation function, and then input into the AFCA module; X NS Then, the input residual (Residual) Flow is used, which uses a normal convolution (Conv) layer. Instead of directly adding the features of the previous convolution module to the current layer feature output after processing to learn the residual features, the pre-processed noise-reduced X NS As the learning object of the residual structure, the backward propagation of noise information in the residual structure can be reduced. In the feature extraction part, two image sharing convolution layers and other trainable parameter layers are used.

[0098] It should be noted that since the FENet is based on the overall framework of Xception, except for the improvements in the feature extraction part as described above, the remaining parts are the same as Xception, so in the embodiments of the present application, the Figure 5 other units in the feature extraction part are not described in detail.

[0099] Among them, the FENet retains the last max pooling layer of the feature extraction part, and replaces other max pooling (Max Pooling) layers in the feature extraction part except the last max pooling layer with a detail-preserving pooling (DPP, Detail-Preserving Pooling) layer.

[0100] The detail-preserving pooling is a pooling method that can preserve detailed features, and its expression is:

[0101]

[0102] ω α , [p, q] = a + s (I [q] - I o [p])

[0103]

[0104] Where I represents the original image, o represents the output, [q] represents the pixel value at position q, a is a bias used to ensure that the input does not disappear and can affect the output, and l is also a reward index. These two parameters are inverse bilateral weights (Inverse Bilateral Weights), and the numbers can be learned from the training data; ω α,λ [p, q] F is a neighborhood Ω pa learnable, non-standardized 2D filter, Ω p The size of the DPP is 3x3. In summary, the DPP can amplify the detailed features in the image, and the importance of various detailed components is measured by learnable parameters a, l, F, etc., so that the pooling operation can also retain the rich detailed features of the original face image, which often contain various artificial artifacts and can become important clues for classification.

[0105] It should be noted that, since the detail preserving pooling is a general detail preserving pooling technology, in the embodiments of the present application, the detail preserving pooling is not described in detail.

[0106] In the FENet, a parameterized ReLU activation function (i.e., a PReLU activation function) is used, and an AFCA module is added after each PReLU activation function and separable convolution, to extract features of different frequency components of the image.

[0107] For example, the PReLU activation function expression is:

[0108]

[0109] where y i represents the input of the activation function on the i-th channel, a i is the slope of the PReLU function in the negative number interval. As can be seen, in the case of using the PReLU function, there is a learnable parameter to control the slope in each feature channel. The PReLU function avoids the problem of complete failure of the ReLU function when the input is negative by using a small slope linear function in the negative value domain, so that the classification accuracy is improved.

[0110] It should be noted that, since the PReLU activation function is a general PReLU activation function, in the embodiments of the present application, the PReLU activation function is not described in detail.

[0111] The adaptive multi-frequency channel attention mechanism module will be described below in conjunction with specific embodiments.

[0112] Each feature channel map of the multi-frequency channel attention mechanism module needs to be processed by N different frequency components, where N is an integer that can be evenly divided by the number of channels C of the multi-frequency channel attention mechanism module.

[0113] Specifically, after the output of the PReLU activation function and the separable convolution enters the AFCA module, the AFCA module intercepts frequency block data of N frequency components as N DCT subblocks, denoted as D0, D1, … DN. N-1 .

[0114] C HxW (H is height, W is width) single-channel feature map is multiplied with N different DCT frequency components to obtain N one-dimensional weight vectors W0, W1, … WN of size 1x1xC. N-1 ; and after being transposed in the H direction (i.e. horizontal direction) and 1x1 convolution, final channel weights are obtained.

[0115] The channel weights are multiplied with the initial feature channel HxWxC to complete one attention mechanism.

[0116] It is worth mentioning that since the AFCA module processes all feature channels with different DCT frequency components respectively, learns features of different frequency components, and thus can greatly improve the accuracy of tamper detection.

[0117] The N frequency components can be obtained by using an image dataset to qualitatively study the different roles played by different image frequency components in face forensics tasks and traditional image classification tasks, extracting the top N frequency components with the highest accuracy for face forensics tasks and traditional image classification tasks, and using the N frequency components as the required N frequency components in the multi-frequency channel attention mechanism module.

[0118] The image dataset includes 10000 tampered face images (fake face images) obtained from four methods Deepfake, Face2face, Faceswap and Neuraltexture in the FaceForensics++ dataset and 40000 real face images from the Youtube website, which together form a face experimental dataset (totaling 80000).

[0119] The image dataset is divided into four groups according to different forgery methods, namely Deepfake, Face2face, Faceswap and NeuralTexture, and each dataset contains 10000 real and fake face images. The following operations are performed on the four sub-datasets:

[0120] As shown in Figure 6 , the process of obtaining N frequency components based on the image dataset is as follows: the frequency spectrum obtained by DCT transformation of the RGB picture (i.e. the original image on the left end in Figure 6 ) of each sub-dataset face image is equally divided in the horizontal and vertical directions. As an optimal example, the obtained frequency spectrum can be equally divided in the horizontal and vertical directions by 8, and the frequency spectrum is divided into 64 blocks. After generating a mask image to retain a single frequency block in the frequency spectrum and performing inverse DCT (iDCT) transformation, a local frequency component image (i.e. Figure 6The original image at the right end); the above processing is performed for each picture of the four sub-datasets, so that 64 sub-frequency component datasets are obtained on each forgery method sub-dataset; the 64 sub-frequency component datasets are subjected to face tampering recognition on a residual network (ResNet-18) to obtain the accuracy of each frequency component; finally, the top N frequency components with the highest accuracy are selected as different DCT frequency components allocated to the AFCA module.

[0121] The training and testing of the face image tampering detection model will be described below in conjunction with specific examples.

[0122] For example, the face image tampering detection model can be trained and tested using the FaceForensics++ dataset and the DFDC Preview dataset. The FaceForensics++ dataset includes three sub-datasets of different image qualities: raw (Raw, Raw Quality, uncompressed), high quality (HQ, High Quality, compression parameter 23), and low quality (LQ, Low Quality, compression parameter 40), each containing 63,000 true and false face images.

[0123] Since the DFDC Preview dataset does not contain images of different qualities, in order to better study the robustness of the proposed network to image compression, the DFDC Preview dataset is compressed using the JPEG algorithm in this embodiment. According to the JPEG compression ratio, it is divided into four sub-datasets: Raw (uncompressed), HQ (80%), middle quality (MQ, 60%), and LQ (40%), a total of 240,000 images.

[0124] The accuracy of the face image tampering passive detection method of the present application will be described below in conjunction with specific experimental data. Table 1 is the accuracy (ACC) comparison experimental data of FENet on the FaceForensics++ dataset, Table 2 is the AUC (AUC is a commonly used index for evaluating the performance of a binary classification network, and a higher AUC value generally indicates better network training results) comparison experimental data of FENet on the FaceForensics++ dataset, Table 3 is the ACC comparison experimental data of FENet on the DFDC Preview dataset, and Table 4 is the AUC comparison experimental data of FENet on the DFDC Preview dataset.

[0125]

[0126]

[0127] Table 1

[0128] Method AUC(Raw) AUC(HQ) AUC(LQ) Meso-4 70.36 66.03 54.21 Meso-Incep 79.53 73.45 69.42 HP-CNN 78.82 71.28 68.37 Constrained-Conv 95.21 92.00 88.35 AMTEN 96.68 91.78 86.82 XceptionNet 99.26 98.52 97.43 ResNet34 88.25 80.36 75.32 FENet 99.99 99.93 99.56

[0129] Table 2

[0130] Method ACC(Raw) ACC(HQ) ACC(MQ) ACC(LQ) Meso-4 53.71 60.63 58.25 54.38 Meso-Incep 58.16 64.49 59.47 58.30 HP-CNN 61.49 64.09 63.38 62.59 Constrained-Conv 81.01 83.40 81.27 80.05 AMTEN 88.83 85.69 83.96 83.76 XceptionNet 89.37 92.29 90.28 88.04 ResNet34 94.52 96.68 94.92 93.93 FENet 97.88 96.83 96.15 95.78

[0131] Table 3

[0132] Method AUC(Raw) AUC(HQ) AUC(MQ) AUC(LQ) Meso-4 55.37 53.26 52.14 51.54 Meso-Incep 65.46 64.68 59.72 55.90 HP-CNN 67.58 63.73 60.85 56.36 Constrained-Conv 87.72 85.34 83.14 80.41 AMTEN 89.25 86.13 85.57 84.25 XceptionNet 96.91 93.86 88.79 83.37 ResNet34 73.64 70.24 69.23 68.10 FENet 99.58 98.53 97.78 94.89

[0133] Table 4

[0134] As shown in Table 1 and Table 3, the accuracy of FENet is 99.96% in ACC comparison using the FaceForensics++ dataset, which is significantly higher than the accuracy of other face image tampering detection methods; the accuracy of FENet is 97.88% in ACC comparison using the DFDC Preview dataset, which is significantly higher than the accuracy of other face image tampering detection methods.

[0135] As shown in Table 2 and Table 4, the AUC value of FENet is 99.99% in AUC comparison using the FaceForensics++ dataset, which is higher than the ACU value of other face image tampering detection methods; the AUC value of FENet is 99.58% in AUC comparison using the DFDC Preview dataset, which is higher than the ACU value of other face image tampering detection methods.

[0136] The face image tampering passive detection device provided by the present application will be exemplarily described below in combination with specific embodiments.

[0137] As shown in Table 1 and Table 3, the accuracy of FENet is 99.96% in ACC comparison using the FaceForensics++ dataset, which is significantly higher than the accuracy of other face image tampering detection methods; the accuracy of FENet is 97.88% in ACC comparison using the DFDC Preview dataset, which is significantly higher than the accuracy of other face image tampering detection methods. Figure 7 The face image tampering passive detection device 700 provided by the embodiments of the present application comprises:

[0138] The first acquisition module 701 is configured to acquire a face image to be detected.

[0139] The second acquisition module 702 is configured to perform frequency domain enhancement processing and compressed noise suppression processing on the face image to be detected to obtain a first face image subjected to frequency domain enhancement and a second face image subjected to suppression of compressed noise.

[0140] The detection module 703 is configured to input the first face image and the second face image into a face image tampering detection model for tampering detection to obtain a detection result; the detection result is used to prompt the face image to be detected as a tampered image or a real image, and a feature extraction part of the face image tampering detection model comprises a multi-frequency channel attention mechanism module.

[0141] It should be noted that the information interaction, execution process and the like between the above apparatuses / units are based on the same concept as the method embodiments of the present application, and the specific functions and the technical effects brought by the same can be referred to the method embodiments part, which will not be repeated here.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for convenient distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0143] As shown in FIG. 1, the embodiment of the present application provides a terminal device, as shown in FIG. 2, the terminal device D10 of the embodiment comprises at least one processor D100 (only one processor is shown in the figure), a memory D101 and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 executes the computer program D102 to realize the steps in any of the above method embodiments. Figure 8 Figure 8 Figure 8

[0144] ​​​Specifically, the processor D100 executes the computer program D102 to obtain a to-be-detected face image, and performs frequency domain enhancement processing and compressed noise suppression processing on the to-be-detected face image to obtain a first face image enhanced in frequency domain and a second face image with suppressed compressed noise, and then inputs the first face image and the second face image into a face image tampering detection model to perform tampering detection, and obtain a detection result for prompting the to-be-detected face image as a tampered image or a real image. Since the frequency domain enhancement can enhance the spatial domain features, the first face image can enhance the spatial domain features of the to-be-detected face image and improve the important features of the to-be-detected face image. Since the second face image can reduce the influence of high-frequency noise caused by the image compression algorithm, the face image tampering detection model can improve the detection accuracy of face image tampering when performing tampering detection on the to-be-detected face image based on the first face image and the second face image. In addition, since the feature extraction part of the face image tampering detection model includes a multi-frequency channel attention mechanism module, the multi-frequency channel attention mechanism module uses global average pooling to take the mean value of each channel in the feature channel as a weight to enhance the content of interest and suppress the background features in a weighted manner, thereby further improving the detection accuracy of face image tampering.

[0145] The processor D100 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0146] The memory D101 can be an internal storage unit of the terminal device D10, such as a hard disk or a memory of the terminal device D10 in some embodiments. The memory D101 can also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device D10 in other embodiments. Further, the memory D101 can include both the internal storage unit and the external storage device of the terminal device D10. The memory D101 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of the computer program, etc. The memory D101 can also be used to temporarily store data that has been output or will be output.

[0147] It should be noted that the information interaction, execution process, etc. between the above devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the brought technical effects can be referred to the method embodiments part, and will not be repeated here.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the system can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0149] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps in each of the above method embodiments.

[0150] The embodiments of the present application provide a computer program product. When the computer program product is run on a terminal device, the terminal device is caused to implement the steps in each of the above method embodiments.

[0151] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the face image tampering passive detection device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0152] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0153] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0154] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the above-described apparatus / network device embodiments are merely schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0155] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0156] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A passive detection method for facial image tampering, characterized in that, include: Acquire the image of the face to be detected; The face image to be detected is subjected to frequency domain enhancement processing and compression noise suppression processing to obtain a first face image with frequency domain enhancement and a second face image with suppressed image compression noise; The first face image and the second face image are input into the face image tampering detection model for tampering detection, and the detection results are obtained. The detection result is used to indicate whether the face image to be detected is a tampered image or a real image. The feature extraction part of the face image tampering detection model includes a multi-frequency channel attention mechanism module. The step of performing frequency domain enhancement processing and compression noise suppression processing on the face image to be detected to obtain a first face image with frequency domain enhancement and a second face image with suppressed image compression noise includes: The face image to be detected is converted from the RGB color space to the YCbCr color space to obtain the Y channel sub-image, Cb channel sub-image and Cr channel sub-image; DCT transformation is performed on the Y channel sub-image, the Cb channel sub-image, and the Cr channel sub-image respectively to obtain the first original DCT image corresponding to the Y channel sub-image, the second original DCT image corresponding to the Cb channel sub-image, and the third original DCT image corresponding to the Cr channel sub-image. Based on the first original DCT image, the second original DCT image, and the third original DCT image, a first face image with frequency domain enhancement and a second face image with suppressed image compression noise are obtained; The step of obtaining a frequency-domain enhanced first face image based on the first original DCT image, the second original DCT image, and the third original DCT image includes: Normalize the first original DCT image, the second original DCT image, and the third original DCT image respectively to obtain the first DCT weight image corresponding to the first original DCT image, the second DCT weight image corresponding to the second original DCT image, and the third DCT weight image corresponding to the third original DCT image. The first original DCT image, the second original DCT image, and the third original DCT image are respectively divided into n n cubes; where n is a power of 2 and n is greater than or equal to 8; For each of the first, second, and third original DCT images, normalization is performed on each block in the original DCT image to obtain a result containing n The frequency domain matrix weights are obtained by copying and expanding the matrix to obtain the frequency domain weight map corresponding to the original DCT map; wherein, the frequency domain weight map corresponding to the first original DCT map is the first frequency domain weight map, the frequency domain weight map corresponding to the second original DCT map is the second frequency domain weight map, and the frequency domain weight map corresponding to the third original DCT map is the third frequency domain weight map. The first DCT weight map, the second DCT weight map, and the third DCT weight map are added to the first frequency domain weight map, the second frequency domain weight map, and the third frequency domain weight map according to the color channels to obtain the first enhancement weight map corresponding to the first original DCT map, the second enhancement weight map corresponding to the second original DCT map, and the third enhancement weight map corresponding to the third original DCT map. A first face image with frequency domain enhancement is obtained based on the first enhancement weight map, the second enhancement weight map, and the third enhancement weight map.

2. The method according to claim 1, characterized in that, The step of obtaining a frequency-domain enhanced first face image based on the first enhancement weight map, the second enhancement weight map, and the third enhancement weight map includes: Calculate the first product of the first enhanced weight map and the Y channel sub-image, the second product of the second enhanced weight map and the Cb channel sub-image, and the third product of the third enhanced weight map and the Cr channel sub-image; For each of the first product, the second product, and the third product, inverse DCT transformation and RGB conversion are performed sequentially to obtain a frequency-domain enhanced first face image.

3. The method according to claim 1, characterized in that, Based on the first original DCT image, the second original DCT image, and the third original DCT image, a second face image with suppressed image compression noise is obtained, including: For each of the first, second, and third original DCT maps, the original DCT map is divided into a first image region including high-frequency components and a second image region other than the first image region. The first image region is then subjected to overall normalization, and the second image region is subjected to sliding window normalization to obtain the fourth DCT weight map corresponding to the first original DCT map, the fifth DCT weight map corresponding to the second original DCT map, and the sixth DCT weight map corresponding to the third original DCT map. For each DCT weight map in the fourth, fifth, and sixth DCT weight maps, inverse DCT transformation and RGB conversion are performed sequentially to obtain a second face image with suppressed image compression noise.

4. The method according to claim 1, characterized in that, The face image tampering detection model is the Xception classification model. The activation function of the feature extraction part of the Xception classification model is the parameterized ReLU activation function. All pooling layers in the feature extraction part except the last pooling layer are detail-preserving pooling layers. Each separable convolutional layer in the feature extraction part is connected to a multi-frequency channel attention mechanism module.

5. The method according to claim 4, characterized in that, Each feature channel map of the multi-frequency channel attention mechanism module is processed by N different frequency components, where N is an integer divisible by the number of channels C of the multi-frequency channel attention mechanism module.

6. A passive detection device for facial image tampering, characterized in that, include: The first acquisition module is used to acquire the face image to be detected; The second acquisition module is used to perform frequency domain enhancement processing and compression noise suppression processing on the face image to be detected, so as to obtain a first face image with frequency domain enhancement and a second face image with suppressed image compression noise. The detection module is used to input the first face image and the second face image into the face image tampering detection model for tampering detection and to obtain the detection result; The detection result is used to indicate whether the face image to be detected is a tampered image or a real image. The feature extraction part of the face image tampering detection model includes a multi-frequency channel attention mechanism module. The second acquisition module is specifically used to perform the following steps: The face image to be detected is converted from the RGB color space to the YCbCr color space to obtain the Y channel sub-image, Cb channel sub-image and Cr channel sub-image; DCT transformation is performed on the Y channel sub-image, the Cb channel sub-image, and the Cr channel sub-image respectively to obtain the first original DCT image corresponding to the Y channel sub-image, the second original DCT image corresponding to the Cb channel sub-image, and the third original DCT image corresponding to the Cr channel sub-image. Based on the first original DCT image, the second original DCT image, and the third original DCT image, a first face image with frequency domain enhancement and a second face image with suppressed image compression noise are obtained; The step of obtaining a frequency-domain enhanced first face image based on the first original DCT image, the second original DCT image, and the third original DCT image includes: Normalize the first original DCT image, the second original DCT image, and the third original DCT image respectively to obtain the first DCT weight image corresponding to the first original DCT image, the second DCT weight image corresponding to the second original DCT image, and the third DCT weight image corresponding to the third original DCT image. The first original DCT image, the second original DCT image, and the third original DCT image are respectively divided into n n cubes; where n is a power of 2 and n is greater than or equal to 8; For each of the first, second, and third original DCT images, normalization is performed on each block in the original DCT image to obtain a result containing n The frequency domain matrix weights are obtained by copying and expanding the matrix to obtain the frequency domain weight map corresponding to the original DCT map; wherein, the frequency domain weight map corresponding to the first original DCT map is the first frequency domain weight map, the frequency domain weight map corresponding to the second original DCT map is the second frequency domain weight map, and the frequency domain weight map corresponding to the third original DCT map is the third frequency domain weight map. The first DCT weight map, the second DCT weight map, and the third DCT weight map are added to the first frequency domain weight map, the second frequency domain weight map, and the third frequency domain weight map according to the color channels to obtain the first enhancement weight map corresponding to the first original DCT map, the second enhancement weight map corresponding to the second original DCT map, and the third enhancement weight map corresponding to the third original DCT map. A first face image with frequency domain enhancement is obtained based on the first enhancement weight map, the second enhancement weight map, and the third enhancement weight map.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the passive detection method for face image tampering as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the passive detection method for face image tampering as described in any one of claims 1 to 5.

Images