An adversarial noise suppression method and system for image tampering localization task

Abstract

The application discloses an anti-noise suppression method and system for an image tampering positioning task. The method comprises the following steps: obtaining a target image, inputting the target image into an anti-noise suppression module to obtain a defensive disturbance corresponding to the target image; generating a defensive disturbance image corresponding to the target image according to the target image and the defensive disturbance corresponding to the target image; inputting the defensive disturbance image corresponding to the target image into a tampering positioning model to obtain a predicted tampering mask or a predicted tampering probability map corresponding to the target image. The application generates a defensive disturbance through an anti-noise suppression module to suppress the attack effect of the anti-noise, thereby assisting in improving the prediction accuracy of the tampering positioning model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an adversarial noise suppression method and system for an image tampering positioning task. BACKGROUND

[0002] With the development of image editing technology and generative artificial intelligence tools, the difficulty of generating tampered images without perceptible traces is reduced. The massive spread of such images has brought serious security risks to society. Image tampering localization is a key technology in image forensics, which aims to identify and accurately locate the tampered region in an image.

[0003] In recent years, deep learning technology has become the mainstream method for this task and has made significant progress in dealing with typical tampering types such as splicing, copy-paste, and repair. More advanced tampering localization models have the ability to handle multiple tampering types simultaneously. At the same time, as the ability of tampering localization models improves, attackers have also developed "anti-forensic" attack strategies to mislead these models. Traditional anti-forensic attack methods include JPEG compression, Gaussian noise addition, and image blurring, aiming to weaken or mask tampering traces. Although existing advanced models have strong robustness to the above traditional attacks, they still lack systematic defense against a more covert and attackable method - adversarial attack. Adversarial attack is a significant vulnerability of deep learning models. It adds carefully designed noise to the image, causing the model to produce incorrect output, although the noise is invisible to the human eye. Adversarial attacks have white-box and black-box settings: in the white-box setting, the attacker has full information about the victim model (structure, parameters, and gradients), and can generate high-precision adversarial samples; in the black-box setting, the attacker achieves the goal through a large number of queries or transfer attacks. Compared with the black-box attack, the white-box attack is more destructive and challenging.

[0004] It can be seen that although the development of deep learning has significantly improved the effect of image tampering localization, existing tampering localization models are still vulnerable to adversarial attacks: the imperceptible adversarial noise added to the tampered image can mislead these models. There is currently a lack of a defense mechanism with generalization ability that can effectively defend against multiple attack algorithms.

[0005] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0006] The technical problem solved by the present application is to provide an adversarial noise suppression method and system for image tampering positioning tasks to solve the problem that tampering positioning models in the prior art are vulnerable to adversarial attacks, resulting in adversarial noise in tampered images affecting the prediction accuracy of tampering positioning models.

[0007] The technical solution adopted by the present application to solve the problem is as follows:

[0008] In a first aspect, the present application provides an adversarial noise suppression method for image tampering positioning tasks, which comprises:

[0009] Obtaining a target image, inputting the target image into an adversarial noise suppression module to obtain a defensive perturbation corresponding to the target image;

[0010] Generating a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image;

[0011] Inputting the defensive perturbation image corresponding to the target image into a tampering positioning model to obtain a predicted tampering mask or a predicted tampering probability map corresponding to the target image.

[0012] In an embodiment, the adversarial noise suppression module is implemented using a lightweight encoder-decoder structure; the encoder uses an EfficientNet-B4 model as the backbone network; and the decoder is generated based on a U-Net network structure and includes a multi-scale feature fusion and a skip connection mechanism.

[0013] In an embodiment, the decoder includes five up-sampling modules and four feature fusion modules arranged alternately in sequence.

[0014] Each of the up-sampling modules includes an up-sampling operation with a scaling factor of 2, a 3x3 convolution layer, a batch normalization layer, and a ReLU activation function.

[0015] Each of the feature fusion modules includes two 3x3 convolution layers arranged in sequence, a batch normalization layer between the convolution layers, and a ReLU activation function at the end.

[0016] The output data of the last up-sampling layer are input into two parallel branches respectively; the first branch uses a SoftSign activation function to scale the output range; and the second branch uses a Sigmoid activation function to scale the output range.

[0017] The output data corresponding to the two parallel branches are multiplied element by element to obtain the defensive perturbation.

[0018] In an implementation, generating the defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image comprises:

[0019] performing a pixel value clipping operation on the target image and the defensive perturbation corresponding to the target image to obtain the defensive perturbation image corresponding to the target image; the target image is an original tampered image or a adversarial image corresponding to the original tampered image.

[0020] In an implementation, the adversarial noise suppression module is pre-trained, and the training method comprises:

[0021] freezing the tampering positioning model and selecting a plurality of feature extraction layers of the tampering positioning model to construct a feature extractor;

[0022] obtaining a plurality of first feature maps based on the original tampered image sample through the feature extractor; obtaining a plurality of second feature maps based on the defensive perturbation image corresponding to the original tampered image sample through the feature extractor;

[0023] for each layer, performing normalization processing on the feature maps corresponding to the layer in the first feature map set and the second feature map set to obtain two normalized feature distributions; taking the feature distribution corresponding to the original tampered image sample as a target distribution, and taking the feature distribution of the defensive perturbation image corresponding to the original tampered image sample as an to-be-optimized distribution; calculating a first loss function value according to the target distribution and the to-be-optimized distribution; and performing parameter updating on the adversarial noise suppression module that has not completed training according to the first loss function value.

[0024] In an implementation, the training method further comprises:

[0025] inputting the original tampered image sample into the tampering positioning model to obtain a first tampering probability map;

[0026] inputting the original tampered image sample into the adversarial noise suppression module that has not completed training to obtain the defensive perturbation image corresponding to the original tampered image sample; and inputting the defensive perturbation image corresponding to the original tampered image sample into the tampering positioning model to obtain a second tampering probability map;

[0027] inputting the adversarial image sample corresponding to the original tampered image sample into the adversarial noise suppression module that has not completed training to obtain the defensive perturbation image corresponding to the adversarial image sample; and inputting the defensive perturbation image corresponding to the adversarial image sample into the tampering positioning model to obtain a third tampering probability map;

[0028] performing binary conversion processing according to the first tampering probability map to obtain a supervised tampering mask;

[0029] According to the supervised tamper mask, the second tamper probability map and the third tamper probability map, a second loss function value is calculated, and a parameter of an unfinished training adversarial noise suppression module is updated according to the second loss function value.

[0030] In an implementation manner, the first loss function value is a KL divergence loss function value, and the second loss function value is a mask loss function value.

[0031] In a second aspect, an embodiment of the present application further provides an adversarial noise suppression system for an image tamper positioning task, and the system comprises:

[0032] A suppression module is configured to acquire a target image, input the target image into an adversarial noise suppression module, and obtain a defensive perturbation corresponding to the target image.

[0033] A generation module is configured to generate a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image.

[0034] A prediction module is configured to input the defensive perturbation image corresponding to the target image into a tamper positioning model, and obtain a predicted tamper mask or a predicted tamper probability map corresponding to the target image.

[0035] In a third aspect, an embodiment of the present application further provides a terminal, which comprises a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the method for the adversarial noise suppression for the image tamper positioning task according to any one of the above-mentioned embodiments; and the processor is configured to execute the programs.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded and executed by a processor to implement the steps of the method for the adversarial noise suppression for the image tamper positioning task according to any one of the above-mentioned embodiments.

[0037] The present application has the following beneficial effects: according to the embodiments of the present application, a target image is acquired, the target image is input into an adversarial noise suppression module, a defensive perturbation corresponding to the target image is obtained, a defensive perturbation image corresponding to the target image is generated according to the target image and the defensive perturbation corresponding to the target image, and the defensive perturbation image corresponding to the target image is input into a tamper positioning model to obtain a predicted tamper mask or a predicted tamper probability map corresponding to the target image. The present application generates a defensive perturbation through an anti-noise suppression module to suppress the attack effect of the adversarial noise, and further assists in improving the prediction accuracy of the tamper positioning model. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0039] Figure 1 is a flowchart of the adversarial noise suppression method provided by the embodiment of the present application for the image tampering positioning task.

[0040] Figure 2 is a training stage schematic diagram of the tampering related feature alignment method provided by the embodiment of the present application.

[0041] Figure 3 is a training stage schematic diagram of the mask guided optimization method provided by the embodiment of the present application.

[0042] Figure 4 is a test stage schematic diagram provided by the embodiment of the present application.

[0043] Figure 5 is a visualization result diagram of the tampering mask provided by the embodiment of the present application.

[0044] Figure 6 is a visualization result diagram of the feature distribution difference provided by the embodiment of the present application.

[0045] Figure 7 is a module schematic diagram of the adversarial noise suppression system for the image tampering positioning task provided by the embodiment of the present application.

[0046] Figure 8 is a principle block diagram of the terminal provided by the embodiment of the present application. DETAILED DESCRIPTION

[0047] The present application discloses an adversarial noise suppression method and system for image tampering positioning task. In order to make the purpose, technical scheme and effect of the present application more clear and explicit, the present application will be further described in detail with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0048] It will be understood by those within the art that, in this specification and in the claims that follow, relative terms are used to describe features that can be positioned with respect to one another. With respect to the terms "first" and "second" and "third" and "fourth" and "fifth" and "sixth" and similar terms, these terms are used to differentiate between entities, and are not intended to imply a physical or logical order or sequence. Such terms are used merely to distinguish between two or more entities or actions. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes" and / or "including", when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will be understood that when a quantity is said to be "connected" or "coupled" to another, it can be directly connected or coupled to the other, or intervening elements can be present. In addition, the use of "connection" or "coupling" herein also includes wireless connection or wireless coupling. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0049] It will be understood by those within the art that, unless otherwise defined, all terms used herein including technical and scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the specification and relevant art and should not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0050] In view of the above defects of the prior art, the present application provides an adversarial noise suppression method for an image tampering positioning task. The method comprises the following steps: obtaining a target image, inputting the target image into an adversarial noise suppression module to obtain a defensive perturbation corresponding to the target image; generating a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image; and inputting the defensive perturbation image corresponding to the target image into a tampering positioning model to obtain a predicted tampering mask or a predicted tampering probability map corresponding to the target image. The present application generates a defensive perturbation through an anti-noise suppression module to suppress the attack effect of adversarial noise that may exist in a tampered image, thereby assisting in improving the prediction accuracy of the tampering positioning model.

[0051] As shown in Figure 1 the method comprises:

[0052] In step S100, a target image is obtained, and the target image is input into an adversarial noise suppression module to obtain a defensive perturbation corresponding to the target image.

[0053] Specifically, the target image refers to the input image of the adversarial noise suppression module, which can be the original tampered image or the adversarial image corresponding to the original tampered image. In view of the possible imperceptible adversarial noise in the tampered image, an anti-noise suppression module (Adversarial Noise Suppression Module, ANSM) is designed in advance. The goal of the anti-noise suppression module is to generate a set of defensive perturbations to offset the possible adversarial noise in the tampered image, thereby weakening the attack effect of the adversarial noise on the tamper localization model.

[0054] In an implementation manner, the method for obtaining the adversarial image comprises:

[0055] performing a pixel value truncation operation on the original tampered image and the adversarial noise to obtain an adversarial image corresponding to the original tampered image.

[0056] Specifically, although existing white-box attack algorithms differ in the way of generating adversarial noise, such as one-time perturbation based on gradient (such as FGSM algorithm) and iterative algorithm based on optimization (such as C&W algorithm), the essence of all of them is to add adversarial noise to the original tampered image, which can be uniformly modeled as additive noise, that is:

[0057] X a = TRUNC (X o + ξ) ;

[0058] wherein X a represents the adversarial image, X o is the original tampered image, and ξ is the adversarial noise; TRUNC represents a pixel value truncation operation (such as limiting the pixel value to the range of [0, 255] or [0, 1]).

[0059] In an implementation manner, the anti-noise suppression module adopts a lightweight encoder-decoder structure; the encoder adopts an EfficientNet-B4 model as a backbone network; and the decoder is generated based on a U-Net network structure and contains a multi-scale feature fusion and a skip connection mechanism.

[0060] The anti-noise suppression module in the embodiment adopts a lightweight design to reduce the parameter size and the amount of calculation and improve the model efficiency. Specifically, a lightweight encoder-decoder structure is adopted, wherein the encoder part selects an EfficientNet-B4 model as a backbone network, the model has been pre-trained on an ImageNet dataset, and can effectively extract complex texture features and potential adversarial noise patterns in the image; the decoder part learns from the U-Net network structure, introduces a multi-scale feature fusion and a skip connection mechanism, and improves the feature extraction capability of the model on the adversarial noise.

[0061] In an implementation manner, the decoder comprises five up-sampling modules and four feature fusion modules arranged alternately in sequence.

[0062] Each of the up-sampling modules comprises an up-sampling operation with a scaling factor of 2, a 3x3 convolution layer, a batch normalization layer, and a ReLU activation function.

[0063] Each of the feature fusion modules comprises two 3x3 convolution layers arranged in sequence, a batch normalization layer arranged between the convolution layers, and a ReLU activation function at the end.

[0064] The output data of the last up-sampling layer are respectively input into two parallel branches; the first branch uses a SoftSign activation function to scale the output range; and the second branch uses a Sigmoid activation function to scale the output range.

[0065] The output data of the two parallel branches are respectively multiplied element by element to obtain the defensive perturbation.

[0066] Specifically, the decoder comprises five up-sampling modules and four feature fusion modules arranged alternately in sequence. In the decoder generated based on the U-Net network structure, after each up-sampling (restoring resolution), the feature map of the corresponding layer of the encoder is spliced (jump connection), and then the features are fused through the convolution layer. The number of up-sampling modules is equal to the number of down-sampling stages of the encoder, and the number of feature fusion modules is one less than that of the up-sampling modules, because after the last up-sampling, there can be no need for further fusion. The five up-sampling modules and the four feature fusion modules are arranged alternately, indicating that after each up-sampling to expand the size of the feature map, the feature fusion module integrates the shallow details of the encoder and the deep semantics of the decoder, gradually restores the image details and suppresses the noise. After the last up-sampling module, the output of the decoder is divided into two parallel branches, the first branch uses a SoftSign activation function to scale the output range to (-1, 1), and its output is represented as γ1∈(-1, 1) 3×H×W , where H and W are the length and width of the image, and 3 is the number of channels of the image. The second branch uses a Sigmoid activation function to scale the output range to (0, 1), represented as γ2∈(0, 1) 3×H×W , where H and W are the length and width of the image, and 3 is the number of channels of the image. The final output defensive perturbation S φ (X a ) is the element-by-element product of γ1 and γ2. The decoder in the embodiment converts the abstract features into defensive perturbations with clear physical meaning through the design of hierarchical feature fusion combined with double-branch activation control.

[0067] Step S200, generating a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image.

[0068] Specifically, the purpose of the embodiment is to adaptively correct the target image by the defensive perturbation to generate a defensive perturbation image, so as to improve the robustness of the image to noise, attacks or other interference while maintaining the semantic of the image.

[0069] In an implementation manner, the defensive perturbation image corresponding to the target image is generated according to the target image and the defensive perturbation corresponding to the target image, including:

[0070] The pixel value truncation operation is performed on the target image and the defensive perturbation corresponding to the target image to obtain the defensive perturbation image corresponding to the target image.

[0071] Specifically, the embodiment designs an anti-defense strategy, which generates a defensive perturbation image by generating a learnable defensive perturbation and counteracting the counter-attack noise.

[0072] X pa =TRUNC(X a +S φ (X a ));

[0073] Wherein, S φ represents the counter-attack noise suppression module, φ represents the parameter set, X pa represents the output defensive perturbation image; TRUNC represents the pixel value truncation operation (for example, limiting the pixel value in the range of [0, 255] or [0, 1]).

[0074] Step S300, inputting the defensive perturbation image corresponding to the target image into the tampering positioning model to obtain a predicted tampering mask or a predicted tampering probability map corresponding to the target image.

[0075] Specifically, the defensive perturbation image is input into the tampering positioning model for pixel-level anomaly evaluation. The defensive perturbation image is generated by adding the defensive perturbation to the image by the counter-attack noise suppression module, which can effectively weaken the attack effect of the counter-attack noise, thereby improving the accuracy of the predicted tampering mask data output by the tampering positioning model.

[0076] In an implementation manner, the counter-attack noise suppression module is trained in advance, and the training method includes:

[0077] Freezing the tampering positioning model and selecting a plurality of feature extraction layers of the tampering positioning model to construct a feature extractor;

[0078] Based on the original tampered image sample, the feature extractor obtains several first feature maps; based on the defensive perturbation image corresponding to the original tampered image sample, the feature extractor obtains several second feature maps.

[0079] For each layer, the feature maps corresponding to that layer in the first feature map set and the second feature map set are normalized to obtain two normalized feature distributions. The feature distribution corresponding to the original tampered image sample is taken as the target distribution, and the feature distribution of the defensive perturbation image corresponding to the original tampered image sample is taken as the distribution to be optimized. A first loss function value is calculated based on the target distribution and the distribution to be optimized, and the parameters of the adversarial noise suppression module that has not completed training are updated based on the first loss function value.

[0080] Specifically, the tamper-related features extracted from the adversarial image and the original tampered image exhibit significant distribution differences. Therefore, this embodiment employs Forgery-relevant Features Alignment (FFA) as the first-stage training strategy to optimize defensive perturbations. Taking the use of the KL (Kullback-Leibler) divergence loss function in the FFA method as an example, the FFA method minimizes the KL divergence in each channel dimension of the feature map, thereby reducing the distribution differences between the tamper-related features extracted from the adversarial image and the original tampered image.

[0081] For example, such as Figure 2 As shown, defensive perturbation image X is obtained through the Adversarial Noise Suppression Module (ANSM). pa Subsequently, the location model V was frozen and tampered with. θ Feature extractor E is constructed by selecting a portion of its feature extraction layers. θ From the original tampered image X o With defensive perturbation image X pa Extract multi-layered tampering-related features, represented as follows:

[0082]

[0083] Where n represents the number of feature layers selected, F (i) The shape is (c i ,h i ,w i ), which represent the number of channels, height, and width, respectively.

[0084] Furthermore, to reduce the computational burden of training and improve the stability of the results, this embodiment divides the feature layer into a shallow layer (first 1 / 3), a middle layer (middle 1 / 3), and a deep layer (last 1 / 3). Experiments show that the middle layer features have better alignment effects, possessing both semantic and spatial information, and are most suitable as the optimization target. Subsequently, to measure the differences between image feature distributions, the channel-dimensional KL divergence is introduced as a loss function. First, the feature map is normalized by channels to obtain the normalized feature distribution:

[0085]

[0086] Will Treat it as the target distribution, This is considered a distribution that needs optimization. The KL divergence between the two along the channel dimension is denoted as KL. C The definition is as follows:

[0087]

[0088] Where N represents the feature map size, i.e., h × w. The parameter α = 1.0 × 10 -8 Used to prevent division by zero errors. KL C Let represent the KL divergence along the channel dimension, where j represents the j-th channel.

[0089] Define loss function The average value of the KL divergence at the n-level channel level is expressed as follows:

[0090]

[0091] Calculated The value is the first loss function value, which can be used to optimize the parameters of the adversarial noise suppression module.

[0092] In one implementation, the training method also includes:

[0093] The original tampered image sample is input into the tampering localization model to obtain the first tampering probability map;

[0094] The original tampered image sample is input into the adversarial noise suppression module that has not been trained, and a defensive perturbation image corresponding to the original tampered image sample is obtained; the defensive perturbation image corresponding to the original tampered image sample is input into the tampering localization model, and a second tampering probability map is obtained.

[0095] The adversarial image sample corresponding to the original tampered image sample is input into the adversarial noise suppression module that has not been trained, to obtain the defensive perturbation image corresponding to the adversarial image sample; the defensive perturbation image corresponding to the adversarial image sample is input into the tampering localization model to obtain the third tampering probability map;

[0096] performing a binarization conversion process according to the first tampering probability map to obtain a supervised tampering mask;

[0097] According to the supervised tampering mask, the second tampering probability map and the third tampering probability map, a second loss function value is calculated, and the parameter of the unfinished training of the adversarial noise suppression module is updated according to the second loss function value.

[0098] In order to further improve the accuracy of defensive perturbation, the embodiment also proposes mask-guided refinement (MgR) as a second stage training strategy. The MgR method introduces double mask constraints, which makes the defensive perturbation adaptively maintain effectiveness on the adversarial and original tampering images, and finely restores the tampering positioning accuracy to the original level. Taking the mask loss function used in the MgR method as an example, the MgR method explicitly guides the tampering mask of the defensive perturbation image X pa to be as close as possible to the prediction result of the original tampering image X o . At the same time, considering that the input image may also be X o (the original tampering image), the same mask guidance is also performed on the corresponding defensive perturbation image X po .

[0099] For example, as shown in Figure 3 , after optimization by the FFA method, the adversarial noise suppression module has certain adversarial defense capability and can ensure the global consistency of tampering-related features. However, the FFA method cannot completely guarantee the recovery accuracy of the predicted tampering mask. To make up for this deficiency, a second stage training strategy, mask-guided refinement, is proposed.

[0100] In this stage, first, the adversarial noise suppression module is used to generate the defensive perturbation image X po of the original tampering image and the defensive perturbation image X pa corresponding to the adversarial image of the original tampering image. Then, the defensive perturbation images X po and X pa are input into the tampering positioning model V θ , to obtain the tampering probability map P po corresponding to the defensive perturbation image X po and P pa corresponding to the defensive perturbation image X pa , as the optimization target of this stage.

[0101] Then, a hybrid loss function L combined with binary cross-entropy loss (BCE) and Dice loss is constructed and defined as follows:

[0102]

[0103] where P i and M i denote the i-th value of predicted tampering probability map and supervised mask respectively, N is the total number of pixels in the image, λ bce is the weighting coefficient of binary cross-entropy loss, which is set to 0.3 in the experiment.

[0104] Then, P po and P pa are optimized simultaneously, and the loss function is defined as follows:

[0105]

[0106] It is worth noting that there are two choices for the supervised tampering mask M: one is the true label of the tampering mask M gt , and the other is the predicted mask of the original tampering image M o . Although M gt is more accurate, it does not conform to the original localization ability of the tampering localization model, and using it as a supervised tampering mask may cause the adversarial noise suppression module to learn patterns unrelated to adversarial noise, resulting in overfitting and reducing the generalization performance on other datasets. Therefore, this embodiment chooses to use M o as the supervised mask to retain the generalization ability of the original tampering localization model.

[0107] Finally, the loss function of the mask-guided optimization can be represented as:

[0108]

[0109] The calculated value of is the value of the second loss function, which can be used to optimize the parameters of the adversarial noise suppression module.

[0110] For ease of understanding, the following are the full process steps of training, testing, evaluation, and experimental verification of the adversarial noise suppression module in actual application scenarios:

[0111] For the training phase: first, prepare two types of datasets: one is a tampering image dataset without adversarial noise (i.e., a clean dataset), and the other is an adversarial sample dataset generated using a fixed intensity FGSM algorithm. Subsequently, initialize the adversarial noise suppression module S φ , and sequentially execute the two-stage optimization strategy, i.e., the FFA method and the MgR method.

[0112] In the FFA stage, first input the adversarial image X a into S φ to obtain the defensive perturbation image X pa . Then, use the frozen feature extractor Eθ from the original tampered image X o with the defensive perturbation image X pa extract n-layer tampering-related features, respectively represented as and These features are processed by SoftMax normalization in the channel dimension to obtain the normalized feature set and Subsequently, the channel-level KL divergence between the two sets of features is calculated, and a loss function is constructed accordingly After e1 iterations of optimization, the updated adversarial noise suppression module S φ' is obtained.

[0113] In the MgR stage, first input X o and X a to S φ' to obtain the defensive perturbation image X po and X pa . Subsequently, input X o , X po and X pa to the frozen tampering localization model V θ to obtain the corresponding tampering probability maps P o , P po and P pa , where P o is converted to a tampering mask M o by binaryzation, which is used as a supervised tampering mask. Finally, calculate the loss function and continue to optimize the adversarial noise suppression module S φ' . After e2 iterations are completed, the trained adversarial noise suppression module

[0114] For the test phase: as shown in Figure 4 , in the test process, only the input image X is sent to the adversarial noise suppression module for preprocessing to obtain the defensive perturbation image X p , and then the defensive perturbation image X p is input into the tampering localization model V θ for tampering mask prediction.

[0115] For the experimental phase, first select four of the most advanced image tampering localization models as protection objects:

[0116] MVSS-Net: This model combines multi-scale feature extraction and visual attention mechanisms to improve image tampering localization performance.

[0117] ​IF-OSN: This model models the noise introduced by the Facebook platform, improving the robustness of the model facing images transmitted on social media platforms.

[0118] HDF-Net: This model combines prior knowledge of steganalysis and visual features of images to improve the model's ability to capture tampering traces.

[0119] CoDE: This model is the first to introduce deep reinforcement learning into image tampering localization. By constructing task-related state, action space and reward function, it models image tampering localization as a Markov decision process, effectively improving the model's generalization and robustness.

[0120] Experimental setup: In the white-box setting, the performance of the proposed ANSM module was evaluated using six widely used adversarial attack algorithms, as follows:

[0121] FGSM (Fast Gradient Sign Method): A single-step attack method that perturbs the input image along the sign direction of the loss function gradient. This method is commonly used to test the adversarial robustness of models due to its high computational efficiency and simplicity of implementation.

[0122] C&W (Carlini & Wagner Attack): A goal-oriented attack method that aims to make the model output a tampering mask with all zeros, while imposing an L2 norm constraint on the perturbation size.

[0123] BIM (Basic Iterative Method): An iterative version of FGSM that achieves stronger attack effects through multiple small-step updates.

[0124] PGD (Projected Gradient Descent): A classic first-order attack method that starts with a randomly perturbed image and iteratively optimizes updates within a restricted region, widely used as a benchmark for adversarial robustness testing.

[0125] MI-FGSM (Momentum Iterative FGSM): Introduces a momentum term to smooth the gradient direction based on BIM, making the perturbation update more stable and thus improving the attack effect.

[0126] PGN (Perturbation Gradient Norm-penalized Attack): Regularizes the gradient norm during the attack optimization process, encouraging the perturbation to move towards the loss flat area to enhance the stability and effectiveness of the attack.

[0127] In the white-box setting, for the optimization-based attack algorithm C&W, five different sets of optimization iteration steps m ∈ {50, 100, 150, 200, 250} are set. For the remaining gradient-based methods, three levels of noise intensity are set Corresponding to gradually increasing but still imperceptible adversarial noise.

[0128] For the selection of the data set:

[0129] COLUMBIA is the first publicly available tampered image dataset, with 160 tampered images of varying sizes. This dataset uses a random patching approach to obtain tampered images, and the tampered regions are not subjected to any post-processing, and the tampered images are saved in lossless PNG format.

[0130] The CASIA dataset provides two versions, v1 and v2, containing 960 and 5,123 tampered images, respectively. Their tampering methods focus on patching and copy-paste. Some of them are carefully post-processed to hide tampering traces, such as filtering and blurring.

[0131] The IMD2020 dataset contains 2010 tampered images obtained through various operations such as patching, copy-pasting, repairing, and morphing.

[0132] The MISD dataset contains 227 digitally patched images generated using Figma software. Each tampered image involves multi-source tampering, i.e., patching objects from multiple external images onto a base image.

[0133] During the training of the ANSM module, only the CASIAv2 dataset was used, including its original tampered images and adversarial images generated using FGSM attack The remaining datasets and attack algorithms were only used for testing to comprehensively evaluate the generalization ability of the proposed ANSM module.

[0134] For evaluation indicators: This embodiment uses pixel-level F1 as the main evaluation indicator for tampering localization performance. The higher the F1, the better the model's performance in tampering region localization tasks. In addition, to more clearly reflect the performance changes of the model under adversarial attack and defense conditions, this embodiment proposes a new evaluation indicator—Residual Performance (RP). This indicator is defined as the ratio of the F1 value of the current input image to the F1 value of its corresponding original tampered image (the F1 of the original tampered image is considered to be 100%). The higher the RP, the closer the model's performance under the current input is to its original level without attack, reflecting the model's robustness and defense effect.

[0135] For defense performance: The following four tables summarize the quantitative evaluation results of the proposed counter-noise suppression module in dealing with adversarial attacks, showing the average performance of each adversarial attack algorithm under various perturbation strengths.

[0136] Table 1. Quantitative evaluation results based on the IMD20 dataset

[0137]

[0138] Table 2. Quantitative evaluation results based on the CASIAv1 dataset

[0139]

[0140] Table 3. Quantitative evaluation results based on the Columbia dataset

[0141]

[0142] Table 4. Quantitative evaluation results based on the MISD dataset

[0143]

[0144] Thanks to the two-stage training strategy—FFA method and MgR method, the optimized counter-noise suppression module can significantly restore the tampering localization performance, with RP reaching or exceeding 90%. Although the counter-noise suppression module is only trained on the CASIAv2 dataset and FGSM attack, it still shows excellent generalization ability when dealing with other adversarial algorithms.

[0145] Specifically, under the six adversarial attack algorithms FGSM, C&W, BIM, PGD, MI-FGSM and PGN, the average RP of the tampering localization model CoDE is 24.3%, 8.9%, 3.3%, 3.5%, 2.6%, and 4.2%, respectively. After applying the counter-noise suppression module, the RP is restored to 99.7%, 94.9%, 95.9%, 94.7%, 96.9%, and 95.0%, respectively. More importantly, when the original tampered image is preprocessed by the counter-noise suppression module, the average RP of the IF-OSN, MVSS-Net, HDF-Net and CoDE models reaches 94.3%, 99.8%, 98.6% and 94.8%, respectively. This fully demonstrates that the counter-noise suppression module can adaptively process tampered images without adversarial noise, largely retaining its original tampering localization performance.

[0146] For visual result analysis: Figure 5The tampering localization results corresponding to the case of using the anti-noise suppression module and the case of not using the anti-noise suppression module are shown. The visual comparison result shows that the adversarial attack significantly disrupts the localization ability of the model, resulting in inaccurate identification of the tampered region. After introducing the anti-noise suppression module, the localization result is significantly restored, and the tampering mask M o is highly consistent.

[0147] In addition, the anti-noise suppression module also has strong adaptability to the original tampered image. Even if the original image is preprocessed by the anti-noise suppression module, the corresponding tampering mask M po is also highly consistent with M o .

[0148] In order to further analyze the feature alignment effect, the embodiment adopts the UMAP dimension reduction algorithm to project the tampering related features into a three-dimensional latent space, as shown in Figure 6 . Initially, the adversarial attack causes the feature distribution to shift significantly, dividing the adversarial tampered image and the original tampered image into two distinct clusters. After introducing the anti-noise suppression module, the feature distribution of the defensive perturbation image is realigned in the latent space, and the original image is aggregated into an overlapping cluster structure. This result clearly demonstrates the ability of the anti-noise suppression module to restore the internal feature distribution disturbed by the adversarial noise.

[0149] The method of the present application has the following advantages:

[0150] 1. The phenomenon of significant distribution shift of adversarial noise in tampering related feature space is revealed, emphasizing that the defense strategy should focus on feature alignment rather than pixel restoration.

[0151] 2. The anti-noise suppression module is proposed, which effectively weakens the attack effect of adversarial noise by adding defensive perturbations to the image. The module can be embedded as a pre-processing component in the inference process without modifying the original tampering localization model structure.

[0152] 3. A two-stage defense training strategy is proposed: stage one is feature alignment of tampering related features (FFA), aiming to reduce the distribution difference between the adversarial image and the original tampered image in the feature space; stage two is mask guided optimization (MgR), which improves the pixel level localization accuracy by double mask constraint while maintaining the performance on the original image.

[0153] 4. Experimental verification shows that the method significantly restores the tamper localization accuracy of the tamper localization model on the adversarial image, and has little effect on the performance of the original image, showing superior generalization ability to various attack algorithms and data sets, and is suitable for defense needs in a white-box attack situation. The results of the comprehensive experiments based on the gradient attack algorithm and the optimization attack algorithm show that the method proposed in the embodiment can significantly restore the performance of the tamper localization model when it is used to localize tampered images. It is worth noting that even if the original tampered image is preprocessed by the adversarial noise suppression module, the tamper localization performance is almost unaffected.

[0154] Possible technical variations of the application:

[0155] 1. Module structure can be deformed: lightweight or multi-branch architecture design. The encoder-decoder structure of the adversarial noise suppression module can be modified, such as replacing the encoder with a more lightweight network such as MobileNet, ShuffleNet, etc. to adapt to the deployment requirements of edge devices; or increase the multi-branch structure, independently process different feature regions, and improve local robustness.

[0156] 2. Replacement or enhancement of feature alignment method: the KL divergence in the tamper-related feature alignment (FFA) module can be replaced by JS divergence, EarthMover's Distance, feature maximum mean difference (MMD), etc. more discriminative distribution distance measurement method to improve the accuracy and stability of adversarial feature recovery.

[0157] 3. Mask supervision mechanism extension: the tamper mask used in the mask-guided optimization (MgR) can be extended from a single source to a multi-mask fusion strategy, such as simultaneously referencing multiple model predictions or using a teacher model output for soft label guidance, enhancing the robustness of the supervision signal.

[0158] 4. Application scenario expansion: cross-modal tamper detection. Although the current model focuses on image tamper localization, the adversarial noise suppression module can also be extended to scenarios such as image-text alignment and multi-modal tamper detection. By integrating text feature disturbance suppression mechanisms in the language mode, a new type of module suitable for multi-modal model defense can be formed.

[0159] Based on the above embodiment, the application also provides an adversarial noise suppression system for an image tamper localization task, as shown in Figure 7 , the system comprises:

[0160] a suppression module 01 for obtaining a target image, inputting the target image into an adversarial noise suppression module, and obtaining a defensive disturbance corresponding to the target image;

[0161] The generating module 02 is configured to generate a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image.

[0162] The predicting module 03 is configured to input the defensive perturbation image corresponding to the target image into a tampering localization model to obtain a predicted tampering mask or a predicted tampering probability map corresponding to the target image.

[0163] Based on the above-mentioned embodiments, the present application further provides a terminal, a principle block diagram of which can be shown in Figure 8 The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. The processor of the terminal is configured to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the terminal is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement the method for suppressing adversarial noise for image tampering localization tasks. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0164] Those skilled in the art can understand that Figure 8 The principle block diagram shown in the above-mentioned embodiments is only a block diagram of part of the structure related to the present application scheme, and does not constitute a limitation on the terminal to which the present application scheme is applied. Specifically, the terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0165] In an implementation manner, the memory of the terminal stores more than one program, and is configured to execute the more than one program by more than one processor, which includes instructions for performing the method for suppressing adversarial noise for image tampering localization tasks.

[0166] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0167] In summary, the present application discloses an anti-noise suppression method and system for image tampering positioning task. The method obtains a target image, inputs the target image into an anti-noise suppression module to obtain a defensive perturbation corresponding to the target image; generates a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image; inputs the defensive perturbation image corresponding to the target image into a tampering positioning model to obtain a predicted tampering mask or a predicted tampering probability map corresponding to the target image. The present application aims at the possible anti-noise in the tampered image, generates a defensive perturbation through an anti-noise suppression module to suppress the attack effect of the anti-noise, and further assists in improving the prediction accuracy of the tampering positioning model.

[0168] It should be understood that the application of the present application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the present application.

Claims

1. An adversarial noise suppression method for an image tampering localization task, characterized in that, The method comprises: acquiring a target image, inputting the target image into an adversarial noise suppression module to obtain a defensive perturbation corresponding to the target image; the adversarial noise suppression module is implemented by using a lightweight encoder-decoder structure; the encoder uses an EfficientNet-B4 model as a backbone network; the decoder is generated based on a U-Net network structure and contains a multi-scale feature fusion and a skip connection mechanism; the decoder comprises five up-sampling modules and four feature fusion modules arranged alternately; each up-sampling module comprises: an up-sampling operation with a scaling factor of 2; a 3x3 convolution layer; a batch normalization layer; a ReLU activation function; each feature fusion module comprises: two 3x3 convolution layers arranged sequentially; a batch normalization layer is arranged between the convolution layers; and a ReLU activation function is connected at the end; the output data of the last up-sampling module are input into two parallel branches respectively; a SoftSign activation function is used to scale the output range in the first branch; a Sigmoid activation function is used to scale the output range in the second branch; the output data corresponding to the two parallel branches are multiplied element by element to obtain the defensive perturbation; generating a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image; inputting the defensive perturbation image corresponding to the target image into a tampering positioning model to obtain a predicted tampering mask or a predicted tampering probability map corresponding to the target image.

2. The method of claim 1, wherein, The method comprises: performing a pixel value truncation operation on the target image and the defensive perturbation corresponding to the target image to obtain a defensive perturbation image corresponding to the target image; the target image is an original tampered image or an adversarial image corresponding to the original tampered image.

3. The method of claim 1, wherein, The adversarial noise suppression module is pre-trained, and the training method comprises: freeze the tampering positioning model and select a plurality of feature extraction layers of the tampering positioning model to construct a feature extractor; obtain a plurality of first feature maps based on the original tampered image sample through the feature extractor; obtain a plurality of second feature maps based on the defensive perturbation image corresponding to the original tampered image sample through the feature extractor; for each layer, normalize the feature maps corresponding to the layer in the first feature map set and the second feature map set to obtain two normalized feature distributions; the feature distribution corresponding to the original tampered image sample is taken as a target distribution, and the feature distribution of the defensive perturbation image corresponding to the original tampered image sample is taken as an optimization distribution; calculate a first loss function value according to the target distribution and the optimization distribution; and update the parameters of the adversarial noise suppression module which has not been trained according to the first loss function value.

4. The method of claim 3, wherein, The training method further comprises: inputting the original tampered image sample into the tampering positioning model to obtain a first tampering probability map; inputting the original tampered image sample into an unfinished training adversarial noise suppression module to obtain a defensive perturbation image corresponding to the original tampered image sample; inputting the defensive perturbation image corresponding to the original tampered image sample into the tamper localization model to obtain a second tamper probability map; inputting the original tampered image sample into an unfinished training adversarial noise suppression module to obtain a defensive perturbation image corresponding to the original tampered image sample; inputting the defensive perturbation image corresponding to the original tampered image sample into the tamper localization model to obtain a second tamper probability map; performing a binary conversion process on the first tamper probability map to obtain a supervised tamper mask; calculating a second loss function value according to the supervised tamper mask, the second tamper probability map and the third tamper probability map, and performing parameter updating on the unfinished training adversarial noise suppression module according to the second loss function value.

5. The method of claim 4, wherein, The first loss function value is a KL divergence loss function value; and the second loss function value is a mask loss function value.

6. An adversarial noise suppression system for an image tampering localization task, the system comprising: The system comprises: The suppression module is configured to obtain a target image, input the target image into an adversarial noise suppression module, and obtain a defensive perturbation corresponding to the target image; the adversarial noise suppression module is implemented by using a lightweight encoder-decoder structure; the encoder uses an EfficientNet-B4 model as a backbone network; the decoder is generated based on a U-Net network structure and comprises a multi-scale feature fusion and a skip connection mechanism; the decoder comprises five up-sampling modules and four feature fusion modules arranged alternately; each up-sampling module comprises an up-sampling operation with a scaling factor of 2, a 3x3 convolution layer, a batch normalization layer and a ReLU activation function; each feature fusion module comprises two 3x3 convolution layers arranged sequentially, a batch normalization layer arranged between the convolution layers, and a ReLU activation function at the end; the output data of the last up-sampling module are input into two parallel branches respectively; the first branch uses a SoftSign activation function to scale the output range; the second branch uses a Sigmoid activation function to scale the output range; the output data corresponding to the two parallel branches are multiplied element by element to obtain the defensive perturbation; The generation module is configured to generate a defensive perturbation image corresponding to the target image according to the target image and the defensive perturbation corresponding to the target image. The prediction module is configured to input the defensive perturbation image corresponding to the target image into a tamper localization model to obtain a predicted tamper mask or a predicted tamper probability map corresponding to the target image.

7. A terminal, characterized by comprising: The terminal comprises a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the adversarial noise suppression method for the image tamper localization task according to any one of claims 1-5; and the processors are configured to execute the programs.

8. A computer-readable storage medium storing a plurality of instructions, characterized in that, The instructions are adapted to be loaded and executed by a processor to implement the steps of the method of suppressing adversarial noise for an image tampering localization task as claimed in any one of claims 1-5.

Images