A method and apparatus for detecting a fake image based on deep filtering

Abstract

The present application relates to the technical field of human intelligence deep learning, especially to a method and device for deep filtering-based fake image, comprising: inputting an original image, obtaining an RGB image of the original image through analysis, and then converting the RGB image into a Tensor format; a disturbance image of a fake image is calculated through deep fake image detection and adversarial attack calculation of the image in the Tensor format; a noise region map is determined according to the position of the larger absolute value in the disturbance image; noise is added to the corresponding region of the fake image according to the noise region map, and a noise-added fake image is obtained; the noise-added fake image is subjected to deep filtering processing through a filter kernel and image filtering, and a refined fake image is obtained; and the refined fake image is outputted, wherein the present application adds noise to the artifacts of the fake image and performs deep filtering processing, thereby improving the authenticity of the fake image.

Description

Technical Field

[0001] This invention relates to the field of human intelligence deep learning technology, and in particular to a method and apparatus for forging images based on depth filtering. Background Technology

[0002] In recent years, with the continuous development of artificial intelligence, the realism of deepfake images has been continuously improved. At the same time, deepfake images have been used in many technical fields. Although deepfake images look real, they can still be detected by deepfake image detection networks.

[0003] Current deepfake images contain artifacts in both the spatial and frequency domains. Once these artifacts are detected by a deepfake image detection network, the authenticity of the deepfake image can be determined based on the detection results. Based on this, the present invention is proposed. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a method for forging images based on depth filtering in order to solve the problem of artifacts in depth-forged images in the prior art.

[0005] To achieve the above and other related objectives, the present invention adopts the following solution:

[0006] Option 1

[0007] This invention provides a method for forging images based on depth filtering, characterized by comprising the following steps:

[0008] Step S1: Input the original image, analyze it to obtain the RGB image of the original image, and then convert the RGB image into Tensor format. The original image is in PNG, JPEG, or BMP format. Tensor format can be read by deep learning model frameworks.

[0009] Step S2: The perturbation map of the forged image is calculated by deep forgery image detection and adversarial attack. The deep forgery image detection is a network that has the ability to distinguish between forged and real images after training. The perturbation map of the forged image is obtained by gradient-based adversarial attack algorithm.

[0010] Step S3: Determine the noise region map based on the location of the larger absolute value in the disturbance map. The larger absolute value is defined by humans. The noise region map is a binary map, and the pixel size of the noise region map is the same as that of the forged image.

[0011] Step S4: Based on the noise region map, add noise to the corresponding region of the forged image to obtain a noisy forged image;

[0012] Step S5: The noisy forged image is processed by depth filtering through a filter kernel and image filtering to obtain a refined forged image. Each filter kernel corresponds to a pixel in the forged image.

[0013] The method for forging images based on depth filtering provided in this invention is characterized by:

[0014] Step S1 includes the following steps:

[0015] Step S1.1: After the original image is input, the original image is used to obtain an RGB image through the Python library, and then the pixel value of the RGB image with a pixel value range of [0, 255] is reduced to [0, 1].

[0016] Step S1.2: Convert the image obtained after pixel reduction into Tensor format.

[0017] The method for forging images based on depth filtering provided in this invention is characterized by:

[0018] Step S5 includes the following steps:

[0019] Step S5.1: Input the forged image, obtain the reconstructed image after filtering, and calculate the loss based on the reconstructed image and the real image;

[0020] Step S5.2: Update the model parameters based on the loss calculation;

[0021] Step S5.3: Input the noisy forgery image and obtain a refined forgery image after filtering.

[0022] Step S6: Output the refined forged image from step S5.

[0023] Option 2

[0024] This invention provides a device for forging images based on depth filtering, characterized in that it comprises:

[0025] The image reading module is used to read the RGB image of the original image and convert the RGB image into Tensor format;

[0026] The adversarial attack detection module is used to calculate the perturbation map of the forged image by detecting deepfake images and performing adversarial attacks on Tensor format images.

[0027] The noise localization module is used to determine the noise region map based on the location of the larger absolute value in the disturbance map. Based on the noise region map, noise is added to the corresponding region of the forged image to obtain a noisy forged image.

[0028] The image depth filtering module is used to perform depth filtering on the noisy forged image through a filtering kernel and image filtering to obtain a refined forged image;

[0029] The output module outputs a refined fake image.

[0030] The forged image device based on depth filtering provided by this invention is characterized in that the adversarial attack detection module includes:

[0031] The adversarial attack unit is used to determine the magnitude of the perturbation value of each pixel in the forged image;

[0032] A deepfake image detection unit is used to assist the adversarial attack unit in identifying the perturbation image.

[0033] The present invention provides a forgery device based on depth filtering, characterized in that the image depth filtering module includes:

[0034] The filter kernel generation unit is used to obtain a set of filter kernels based on a deep neural network.

[0035] The image filtering unit is used to filter the forged image to obtain a refined forged image.

[0036] This invention takes an original image as input, analyzes it to obtain its RGB image, and then converts the RGB image into a Tensor format. The Tensor format image is then used for deepfake image detection and adversarial attack calculations to generate a perturbation map of the forged image. A noise region map is determined based on the location of the largest absolute value in the perturbation map. Based on the noise region map, noise is added to the corresponding regions of the forged image to obtain a noisy forged image. This noisy forged image is then subjected to depth filtering through a filtering kernel and image filtering to obtain a refined forged image. The refined forged image is then output. Therefore, this invention adds noise and performs depth filtering on artifacts in forged images, thereby improving the realism of the forged image. Attached Figure Description

[0037] Figure 1 This is a flowchart of a depth-filter-based image forgery method according to an embodiment of the present invention:

[0038] Figure 2 This is a block diagram of a depth-filter-based image forgery device according to Embodiment 2 of the present invention;

[0039] Figure 3 This is an application diagram of an embodiment of the present invention. Detailed Implementation

[0040] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0041] Example 1

[0042] like Figure 1 As shown, this embodiment provides a method for forging images based on depth filtering, including the following steps S1 to S5.

[0043] Step S1: Input the original image, analyze it to obtain the RGB image of the original image, and then convert the RGB image into Tensor format.

[0044] In this embodiment, the original image can be in PNG, JPEG, or BMP format, and the Tensor format can be read by the deep learning model framework.

[0045] Specifically, step S1 above includes the following sub-steps:

[0046] Step S1.1: After the original image is input, the original image is used to obtain an RGB image through the Python library, and then the pixel value of the RGB image with a pixel value range of [0, 255] is reduced to [0, 1].

[0047] Step S1.2: Convert the image obtained after pixel reduction into Tensor format.

[0048] In this embodiment, the RGB image is obtained by reading image files using Python libraries such as OpenCV and Python Imaging Library.

[0049] Step S2: The perturbation map of the forged image is calculated from the Tensor format image through deep forgery image detection and adversarial attack calculation.

[0050] In this embodiment, deepfake image detection is a network that, after training, has the ability to distinguish between fake and real images. The network structure is arbitrary, and it is at least one of the basic network structures such as VGG, Resent, Densent, Mobilenet, or a complex network structure.

[0051] In this embodiment, the perturbation map of the forged image is obtained through a gradient-based adversarial attack algorithm.

[0052] Step S3: Determine the noise region map based on the location of the larger absolute value in the disturbance map.

[0053] In this embodiment, the larger absolute value is defined artificially. The absolute values ​​in the disturbance map are sorted, and the range of larger values ​​is defined artificially. Alternatively, a threshold is set for the absolute value, and absolute values ​​greater than the threshold are classified as larger absolute values. The noise region map is a binary map, and the pixel size of the noise region map and the forged image are the same.

[0054] Step S4: Based on the noise region map, add noise to the corresponding region of the forged image to obtain a noisy forged image.

[0055] In this embodiment, the method of adding noise is selected from at least one of the noise type or the noise intensity. On the one hand, the noise type is selected from common noise types such as Gaussian noise and uniform noise; on the other hand, the greater the noise intensity, the easier it is to destroy artifacts and also the easier it is to reduce the quality of the reconstructed image. When the depth detector has a detection accuracy of 10% on the reconstructed image, the noise intensity is selected to add noise at this time.

[0056] Step S5: The noisy forgery image is processed by depth filtering through a filtering kernel and image filtering to obtain a refined forgery image.

[0057] In this embodiment, one filter kernel corresponds to one pixel in the forged image.

[0058] Specifically, step S5 above includes the following sub-steps:

[0059] Step S5.1: Input the forged image, filter it to obtain the reconstructed image, and calculate the loss based on the reconstructed image and the real image;

[0060] Step S5.2: Based on the loss calculation, update the model parameters to obtain the image depth filter;

[0061] Step S5.3: Input the noisy forgery image into the image depth filter, and obtain the refined forgery image after filtering.

[0062] In this embodiment, the loss calculation for the reconstructed image and the original image is performed using L1 and L2 norms. Through loss calculation, the difference between the reconstructed image and the original image can be determined. When the loss calculation does not change at a certain value, the model parameters are stopped from being updated, and the image depth filtering is obtained.

[0063] Step S6: Output the refined forged image from step S5.

[0064] In this embodiment, as Figure 3As shown, A1 is the original image, and A2 is its spectrogram; B1 is the fake image, and B2 is its spectrogram. In Figure B2, artifacts can be clearly seen in the image; C1 is the noisy fake image, and C2 is its spectrogram. As shown in the spectrogram of Figure C2, the artifacts in B2 are suppressed; D1 is the depth-filtered image, and D2 is its spectrogram.

[0065] Example 2

[0066] like Figure 2 As shown, this embodiment provides a depth-filter-based image forgery device, comprising:

[0067] The image reading module is used to read the RGB image of the original image and convert the RGB image into Tensor format;

[0068] The adversarial attack detection module is used to calculate the perturbation map of the forged image by detecting deepfake images and performing adversarial attacks on Tensor format images.

[0069] The noise localization module is used to determine the noise region map based on the location of the larger absolute value in the disturbance map. Based on the noise region map, noise is added to the corresponding region of the forged image to obtain a noisy forged image.

[0070] The image depth filtering module is used to perform depth filtering on the noisy forged image through a filtering kernel and image filtering to obtain a refined forged image;

[0071] The output module outputs a refined fake image.

[0072] The adversarial attack detection module includes: an adversarial attack unit, used to determine the magnitude of the perturbation value of each pixel in the forged image; and a deep forged image detection unit, used to assist the adversarial attack unit in confirming the perturbation map.

[0073] The image depth filtering module includes: a filter kernel generation unit, used to obtain a set of filter kernels based on a deep neural network; and an image filtering unit, used to filter the forged image to obtain a refined forged image.

[0074] In this invention, a method and apparatus for forging images based on depth filtering has the following beneficial effects:

[0075] This invention takes an original image as input, analyzes it to obtain its RGB image, and then converts the RGB image into a Tensor format. The Tensor format image is then used for deepfake image detection and adversarial attack calculations to generate a perturbation map of the forged image. A noise region map is determined based on the location of the largest absolute value in the perturbation map. Based on the noise region map, noise is added to the corresponding regions of the forged image to obtain a noisy forged image. This noisy forged image is then subjected to depth filtering through a filtering kernel and image filtering to obtain a refined forged image. The refined forged image is then output. Therefore, this invention adds noise and performs depth filtering on artifacts in forged images, thereby improving the realism of the forged image.

[0076] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for forging images based on depth filtering, characterized in that, Includes the following steps: Step S1: Input the original image, obtain the RGB image of the original image through analysis, and then convert the RGB image into Tensor format. The original image is in PNG, JPEG, or BMP format, and the Tensor format can be read by the deep learning model framework. Step S2: The perturbation map of the forged image is calculated by deepfake image detection and adversarial attack on the Tensor format image. The deepfake image detection is a network that is trained to distinguish between forged and real images. The perturbation map of the forged image is obtained by a gradient-based adversarial attack algorithm. The network used for deepfake image detection is VGG, ResNet, DenseNet, or MobileNet. Step S3: Determine the noise region map based on the location of the larger absolute value in the perturbation map. The larger absolute value is obtained by sorting the absolute values ​​of each pixel in the perturbation map and filtering according to a preset range or preset threshold. The noise region map is a binary map, and the pixel size of the noise region map is the same as that of the forged image. Step S4: Based on the noise region map, add noise to the corresponding region of the forged image to obtain a noisy forged image; Step S5: The noisy forgery image is processed by depth filtering through a filter kernel and image filtering to obtain a refined forgery image. One filter kernel corresponds to one pixel in the forgery image. Step S5 includes the following steps: Step S5.1: Input the forged image, obtain the reconstructed image after filtering, and calculate the loss based on the reconstructed image and the real image; Step S5.2: Based on the loss calculation, update the model parameters to obtain the image depth filter; Step S5.3: Perform depth filtering on the input image to obtain a refined forged image after the noisy forgery image is processed. The addition of noise includes: When the depth detector has a detection accuracy of 10% for the reconstructed image, noise is added at the noise intensity corresponding to that accuracy. Step S6: Output the refined forged image from step S5.

2. The image forgery method based on depth filtering according to claim 1, characterized in that: in, Step S1 includes the following steps: Step S1.1: After the original image is input, the original image is used to obtain an RGB image through the Python library, and then the pixel value of the RGB image with a pixel value range of [0, 255] is reduced to [0, 1]. Step S1.2: Convert the image obtained after pixel reduction into Tensor format.

3. A forgery device based on depth filtering, characterized in that, include: The image reading module is used to input the original image, obtain the RGB image of the original image through analysis, and then convert the RGB image into Tensor format. The original image is in PNG, JPEG, or BMP format, and the Tensor format can be read by the deep learning model framework. The adversarial attack detection module is used to calculate the perturbation map of the forged image by deepfake image detection and adversarial attack calculation for Tensor format images. The deepfake image detection is a network that is trained to distinguish between forged and real images. The perturbation map of the forged image is obtained by a gradient-based adversarial attack algorithm. The network used for deepfake image detection is VGG, ResNet, DenseNet, or MobileNet. The noise localization module is used to determine the noise region map based on the location of the larger absolute value in the disturbance map. The larger absolute value is obtained by sorting the absolute values ​​of each pixel in the disturbance map and filtering them according to a preset range or a preset threshold. The noise region map is a binary map, and the pixel size of the noise region map is the same as that of the forged image. The noise addition module is used to add noise to the corresponding areas of the forged image based on the noise region map, so as to obtain a noisy forged image; The image depth filtering module is used to perform depth filtering on the noisy forgery image through filtering kernels and image filtering to obtain a refined forgery image. One filtering kernel corresponds to one pixel in the forgery image. The output module outputs a refined forged image; The image depth filtering module is specifically used to input a forged image, obtain a reconstructed image after filtering, perform loss calculation based on the reconstructed image and the real image, update the model parameters based on the loss calculation, and obtain the image depth filter; and perform input image depth filtering on the noisy forged image, and obtain a refined forged image after filtering. The noise addition module is specifically used to add noise when the depth detector has a detection accuracy of 10% for the reconstructed image, by selecting the noise intensity corresponding to the accuracy rate.

4. The image forgery device based on depth filtering according to claim 3, characterized in that, The attack detection module includes: The adversarial attack unit is used to determine the magnitude of the perturbation value of each pixel in the forged image; A deepfake image detection unit is used to assist the adversarial attack unit in identifying the perturbation image.

5. The image forgery device based on depth filtering according to claim 3, characterized in that, The image depth filtering module includes: The filter kernel generation unit is used to obtain a set of filter kernels based on a deep neural network. The image filtering unit is used to filter the forged image to obtain a refined forged image.

Images