Image tampering positioning method, system and terminal based on reinforcement learning

Abstract

The present invention discloses a method, system and terminal for image tampering location based on reinforcement learning, the method comprising: obtaining a target tampered image, initializing a tampering probability map, constructing a first initial state according to the target tampered image and the initialized tampering probability map, performing an iterative operation of a preset time step according to a trained image tampering location model, and obtaining a final state corresponding to the target tampered image; obtaining a corresponding final tampering probability map according to the final state, and thresholding the final tampering probability map to obtain a predicted mask map, and outputting it. The present invention can output an accurate predicted mask map through the trained image tampering location model, so that the tampering position of the target tampered image can be identified, ensuring the security and credibility of the image.

Description

Technical Field

[0001] The present invention relates to the technical field of image tampering positioning, and in particular to an image tampering positioning method, system, terminal and computer-readable storage medium based on reinforcement learning. Background Art

[0002] At present, the widespread use of image editing software has made it easier to tamper with digital images, so the authenticity of images has become an important issue. At present, most tampered images cannot be visually detected, so deep learning technology is used in image tampering forensics. In image tampering forensics, deep learning is widely used to build an end-to-end framework that can predict the tampering probability map of tampered images, thereby achieving pixel-level image tampering positioning.

[0003] However, although the deep learning-based image tampering forensics framework can locate image tampering, the current image tampering localization model cannot meet the requirements of locating the tampered position of images in real scenes, which poses a potential threat to the security and credibility of the images.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide an image tampering localization method, system, terminal and computer-readable storage medium based on reinforcement learning, aiming to solve the problem that the image tampering localization model in the prior art cannot meet the requirements of tampering position positioning of images in real scenes, which poses a potential threat to the security and credibility of the images.

[0006] To achieve the above object, the present invention provides an image tampering location method based on reinforcement learning, and the image tampering location method based on reinforcement learning comprises the following steps:

[0007] Acquire a target tampered image, initialize a tampering probability map, and construct a first initial state according to the target tampered image and the initialized tampering probability map;

[0008] Input the target tampered image into the trained image tampering localization model, output an initial mean, obtain an initial action according to the initial mean, obtain an initial tampering probability map according to the initial action and the initialized tampering probability map, and construct a second initial state according to the target tampered image and the initial tampering probability map;

[0009] According to the second initial state, an iterative operation of a preset number of time steps is performed to obtain a final state, wherein in the iterative operation of each time step, the state corresponding to the current time step is output according to the trained image tampering localization model and the intelligent agent;

[0010] A corresponding final tampering probability map is obtained according to the final state, and the final tampering probability map is thresholded to obtain a predicted mask map, and output it.

[0011] Optionally, the training process of the image tampering localization model specifically includes:

[0012] Obtain a training image, construct a state of the current time step according to the training image and the corresponding tampering probability map, input the state of the current time step into the image tampering localization model, and output the training variance, training mean and value corresponding to the current time step;

[0013] The corresponding action is calculated according to the training variance and training mean corresponding to the current time step, the action corresponding to the current time step and the tampering probability map are input into the agent, the updated tampering probability map is output, and the state corresponding to the next time step is constructed based on the updated tampering probability map and the training image;

[0014] According to the state corresponding to the next time step, obtain the state and value corresponding to each time step within the preset time step number;

[0015] According to the states and values ​​corresponding to all time steps within the preset number of time steps, the total loss function is calculated in reverse order, and the parameters of the image tampering localization model are updated using the total loss function;

[0016] According to the training image set, multiple training images in the training image set are obtained, and the image tampering localization model is iteratively trained according to the multiple training images. When the number of training times reaches a preset number of training times, the image tampering localization model obtained from the last training is used as the trained image tampering localization model.

[0017] Optionally, the image tampering localization model includes a dual-stream state encoder, an actor network and a critic network, and the dual-stream state encoder includes a tampered image encoder and a tampering probability map encoder;

[0018] The state of the current time step is constructed according to the training image and the corresponding tampering probability map, the state of the current time step is input into the image tampering localization model, and the training variance, training mean and value corresponding to the current time step are output, specifically including:

[0019] Construct the state of the current time step based on the training image and the corresponding tampering probability map;

[0020] Inputting the training image and the tampering probability map in the state of the current time step into the tampered image encoder and the tampering probability map encoder in the image tampering localization model respectively, the tampered image encoder inputs the output into the actor network and the critic network respectively, the tampering probability map encoder outputs a first feature map and a second feature map, and inputs the first feature map into the actor network, and inputs the second feature map into the critic network;

[0021] The actor network outputs the training variance and training mean corresponding to the current time step, and the critic network outputs the value corresponding to the current time step.

[0022] Optionally, the total loss function is calculated in reverse order according to the states and values ​​corresponding to all time steps within the preset number of time steps, and the parameters of the image tampering localization model are updated using the total loss function, specifically including:

[0023] Calculate the reward graph corresponding to each time step based on the states corresponding to all time steps within the preset number of time steps;

[0024] According to the reward graph and value corresponding to each time step, calculate the strategy loss and value loss corresponding to each time step;

[0025] According to the strategy loss and value loss corresponding to each time step, the total loss function is calculated in reverse order, and the parameters of the image tampering localization model are updated using the total loss function.

[0026] Optionally, the step of calculating the reward graph corresponding to each time step according to the states corresponding to all time steps within a preset number of time steps specifically includes:

[0027] Obtain a tampering probability graph in the state corresponding to all time steps within a preset number of time steps;

[0028] According to all the tampering probability maps within the preset time steps, the defined reward function is used to calculate the reward map corresponding to each time step.

[0029] Optionally, the agent obtains an initial tampering probability map according to the initial action and the initialized tampering probability map, specifically including:

[0030] Acquiring the initial action and the initialized tampering probability map;

[0031] The intelligent agent performs a sum operation on the initial action and the initialized tampering probability map to obtain an initial tampering probability map.

[0032] Optionally, thresholding the final tampering probability map to obtain a predicted mask map specifically includes:

[0033] Obtaining element values ​​of elements in the final tampering probability map;

[0034] According to the preset threshold, the element values ​​in the final tampering probability map are updated to complete the thresholding of the final tampering probability map and obtain the predicted mask map.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides an image tampering positioning system based on reinforcement learning, wherein the image tampering positioning system based on reinforcement learning includes:

[0036] An initial state construction module, used to obtain a target tampered image, initialize a tampering probability map, and construct a first initial state according to the target tampered image and the initialized tampering probability map;

[0037] a second initial state construction module, configured to input the target tampered image into the trained image tampering localization model, output an initial mean, obtain an initial action according to the initial mean, obtain an initial tampering probability map according to the initial action and the initialized tampering probability map, and construct a second initial state according to the target tampered image and the initial tampering probability map;

[0038] A final state output module is used to perform an iterative operation of a preset number of time steps according to the second initial state to obtain a final state, wherein in the iterative operation of each time step, the state corresponding to the current time step is output according to the trained image tampering localization model and the intelligent agent;

[0039] The result output module is used to obtain the corresponding final tampering probability map according to the final state, and threshold the final tampering probability map to obtain a predicted mask map, and output it.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a reinforcement learning-based image tampering localization program stored in the memory and run on the processor, and when the reinforcement learning-based image tampering localization program is executed by the processor, the steps of the reinforcement learning-based image tampering localization method as described above are implemented.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image tampering localization program based on reinforcement learning, and when the image tampering localization program based on reinforcement learning is executed by a processor, the steps of the image tampering localization method based on reinforcement learning as described above are implemented.

[0042] In the present invention, a target tampered image is obtained, and an iterative operation of a preset number of time steps is performed according to a trained image tampering positioning model to obtain a final state corresponding to the target tampered image; a corresponding final tampering probability map is obtained according to the final state, and the final tampering probability map is thresholded to obtain a predicted mask map, and output. In the process of implementing the above method in the present invention, the predicted mask map corresponding to the target tampered image can be output through the trained image tampering positioning model, so that the tampered position of the target tampered image can be identified, and pixel-level image tampering positioning can be achieved. At the same time, since the trained image tampering positioning model is based on the set reward function during training, the predicted mask map corresponding to the present application is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of a preferred embodiment of the image tampering location method based on reinforcement learning of the present invention;

[0044] Figure 2 Schematic diagram of three common image tampering methods provided by the present invention;

[0045] Figure 3 It is a schematic diagram of the image tampering localization framework structure corresponding to the image tampering localization method based on reinforcement learning of the present invention;

[0046] Figure 4 It is a schematic diagram of an implementation method of the iterative process of tampering probability graph of the present invention;

[0047] Figure 5 It is a structural diagram of a preferred embodiment of the image tampering localization system based on reinforcement learning of the present invention;

[0048] Figure 6 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] The widespread use of image editing software has made it easier to tamper with digital images, so the authenticity of images has become an important issue, especially in scenarios where accuracy and credibility are crucial. The problem is further exacerbated by the endless number of tampering methods that are emerging today.

[0051] In general, if Figure 2As shown in the figure, tampered images can be divided into three types, namely, stitching, copy and paste, and erasure. Stitching is a method of merging two or more images into one image. In the generated image, edges and blurred areas will appear at the stitching location. By using image editing tools, these areas can be merged with the image, making the tampering imperceptible to the human eye; copy and paste is one of the simplest and most common methods of image manipulation. It is to copy one or more parts of an image and paste them to other locations in the same image. The main purpose of this method is to hide some important information or insert some incorrect information into the image. Since the copied part belongs to the same image, it is compatible with the entire image in terms of structure and texture; erasure is a destructive tampering because it may change the semantic content of the image. There are two ways of erasure: copy and paste and image restoration, in which copy and paste removes the desired object by copying an area from the main image or another image and pasting it to the area of ​​the removed object. Due to the simplicity of copy and paste, it is widely used for erasure; image restoration was initially achieved by filling with surrounding pixels. With the development of deep generation technology, it can fill the erased area in a generative way while maintaining the consistency of texture structure and the rationality of semantics.

[0052] Adobe Photoshop (PS) is the most commonly used software for splicing and copy-paste operations, as well as other related tools including GIMP, After Effects Pro, etc. With the rapid development of deep learning, image restoration techniques based on generative adversarial networks and diffusion models can effectively generate highly reasonable and credible texture structures and semantic content for erasing specific objects, watermarks, faces, label information, etc. As tampering technology becomes more and more powerful, most tampered images cannot be visually detected. Therefore, this requires the development of methods that can capture more subtle tampering features to detect and even locate image tampering areas. Traditional methods usually use manually designed filters to extract underlying features from images, such as texture, color, and edge information, and then apply machine learning algorithms such as SVM (support vector machine) to classify whether the image has been tampered with. However, they may be ineffective for complex tampering operations and require relevant prior knowledge.

[0053] With the surge in the application of deep learning technology in various pattern recognition tasks, image tampering forensics is no exception. In image tampering forensics, deep learning is widely used to build an end-to-end framework that can predict the tampering probability map of tampered images, thereby achieving pixel-level image tampering localization. Among them, MVSS-Net combines multi-scale feature extraction and visual attention mechanism to improve tampering localization performance; PSCC-Net proposes an image spatiotemporal correlation module, which models the correlation of tampering features between image space and channels; in order to defend against adversarial attacks, SAT-IFL uses adversarial samples generated by FGSM (fast gradient sign method) for adversarial training; CAT-Net proposes a two-stream network to capture visual artifacts in the RGB (red, green, and blue) domain and compression artifacts in the DCT (discrete cosine transform) domain.

[0054] However, although significant progress has been made in the research of image tampering detection, there are still two major shortcomings: (1) Limited generalization ability. Generalization ability is mainly related to the performance of the model across multiple datasets. If the data distribution of the test set is very different from that of the training set, the performance of the model may be significantly reduced. To alleviate this problem, some methods propose to fine-tune the pre-trained model using a subset of the target dataset, but this method may only be effective for the specific dataset used for fine-tuning and not for other datasets. (2) Insufficient robustness. Robustness mainly refers to the performance of the model in the face of degraded images. For example, various post-processing attacks such as Gaussian noise, scaling, Gaussian blur, JPEG compression, etc. are applied to the tampered images to evaluate the performance of the model under image degradation. These are single post-processing attacks, while for more complex scenarios, for example, tampered images transmitted through online social networks such as WeChat and Facebook will undergo various known or unknown image processing operations, which brings greater challenges to locating the tampered area, because after such an image processing pipeline, the tampering traces may be severely weakened or even erased.

[0055] Therefore, although the deep learning-based image tampering forensics framework can locate image tampering, the current image tampering localization model cannot meet the requirements of locating the tampered position of images in real scenes, which poses a potential threat to the security and credibility of the images.

[0056] In order to solve one or more of the above problems, the present application proposes an image tampering localization method based on reinforcement learning, namely CoDE (Construct Decision-making Environment). The trained image tampering localization model can output a predicted mask map corresponding to the target tampered image, so that the tampered position of the target tampered image can be identified, and pixel-level image tampering localization can be achieved. At the same time, since the trained image tampering localization model is based on the set reward function during training, the predicted mask map corresponding to the present application is more accurate.

[0057] Moreover, the present application models the tampering localization task as a Markov decision process, in which each pixel is equipped with an agent, which performs continuous actions based on Gaussian distribution to iteratively update the respective tampering probabilities, that is, the process of updating the tampering probability map, thereby realizing pixel-level image tampering localization. In order to construct the state transition within the Markov decision process, the present application proposes a two-stream state encoder to process the tampered image and its corresponding state, so as to obtain an accurate predicted mask map.

[0058] The image tampering location method based on reinforcement learning described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the image tampering positioning method based on reinforcement learning includes the following steps:

[0059] Step S10: Acquire a target tampered image, initialize a tampering probability map, and construct a first initial state according to the target tampered image and the initialized tampering probability map.

[0060] It should be noted that, in the present application, before the method described in the present application is implemented, the elements of the Markov decision process are predefined, that is, part of the definition in the image tampering localization method based on reinforcement learning described in the present application.

[0061] First, define the state space, that is, the state described in this application. The state space contains the most relevant information to the task. For the image tampering location task in this application, in order to capture its subtle tampering features to guide the model to make accurate decisions, the state corresponds to the target tampered image I f , in order to update the tampering probability graph P f , and multiple time steps need to be iterated, so the tampering probability map of the t-th time step is also included in the state space. As shown in formula (1), the state space, that is, the state s (t) It consists of two main parts:

[0062]

[0063] It should be noted that I f and All values ​​must be divided by 255 to normalize them to [0, 1].

[0064] Define the action space, which is designed according to the change of state. Since the goal of this application is to iteratively update the tampering probability of all pixels of the target tampered image (i.e., the tampering probability map P f ), so the addition and subtraction of the tampering probability will be involved in the operation of the whole method. In this application, a continuous action space based on Gaussian distribution is adopted. In the process of training the image tampering location model in this application, the sampling expression of the continuous action based on Gaussian distribution is shown in formula (2):

[0065] a (t) =N(λμ (t) ,(σ 2 ) (t) ); (2)

[0066] Among them, a (t) is the action graph taken by the agent at the tth time step, μ (t) and (σ 2 ) (t ) are the mean and variance of Gaussian distribution, respectively, and N represents Gaussian distribution (normal distribution). In addition, this application introduces a scaling factor λ=0.5 to limit μ (t) This can prevent the agent from over-exploring, reduce its extreme or unrealistic actions, and more effectively enable the agent to learn the optimal strategy.

[0067] However, the sampling of formula (2) is not differentiable, so it cannot be applied to the back propagation of the image tampering localization model. In order to solve this problem, this application reparameterizes it, as shown in formula (3), sampling a variable δ from the standard normal distribution N(0, 1):

[0068] δ~N(0,1);(3)

[0069] Then, use δ to reparameterize formula (2), which is specifically as shown in formula (4):

[0070] a (t) =N(λμ (t) +δ(σ 2 ) (t) ); (4)

[0071] By formula (4), μ (t) and (σ 2 ) (t) Relative to action a (t) The gradient of can be calculated and back-propagated during training.

[0072] In the specific reasoning stage, the agent no longer needs to explore the strategy by sampling, but uses the trained optimal strategy. The expression of the action is shown in formula (5):

[0073] a (t) =λμ (t) ; (5)

[0074] Define the reward function. Specifically, the environment rewards the actions performed by the agent through the reward function, and the reward is the only feedback information that the agent can obtain. The agent is trained and optimized based on this feedback information. Therefore, the reward function should be designed according to the characteristics of different tasks. The goal of this application is to update the tampering probability of each pixel, so a mapping function is defined to give the mapping value of the tampering probability, and then the reward is calculated based on the mapping value before and after the tampering probability is updated. The specific mapping function is shown in formula (6):

[0075]

[0076] Among them, M gt Represents the truth map, ∈1×10 -8 is a constant used to constrain the value range of the mapping function. The reward obtained is the difference between the two mapping values ​​before and after, as shown in formula (7):

[0077]

[0078] On the basis of defining the state, the first initial state is constructed according to the target tampered image and the initialized tampering probability map, that is, the state in formula (1) is constructed according to the target tampered image and the initialized tampering probability map; it should be noted that the target tampered image is the tampered image, and the initialized tampering probability map is 0.

[0079] Step S20: input the target tampered image into the trained image tampering localization model, output the initial mean, obtain the initial action according to the initial mean, the intelligent agent obtains the initial tampering probability map according to the initial action and the initialized tampering probability map, and constructs the second initial state according to the target tampered image and the initial tampering probability map.

[0080] It should be noted that, in the present application, before executing the above image tampering localization method based on reinforcement learning, the image tampering localization model must be trained first.

[0081] Specifically, the structure of the image tampering localization model is as follows Figure 3As shown in Figure 1, the image tampering localization model includes a two-stream state encoder, an actor network, and a critic network. The purpose of the two-stream state encoder is to extract state features and satisfy state transitions. It is divided into two parts: a tampered image encoder and a tampered probability map encoder. Figure 3 It can be seen that the tampered image encoder contains 5 encoder blocks for extracting multi-layer feature maps of tampered images. In order to speed up the training speed, this application uses EffentNet-b4 pre-trained on the ImageNet dataset as the backbone network of the tampered image encoder, and the tampered probability map encoder encodes the tampered probability map. The tampered probability map encoder consists of two tampered probability map convolution blocks, such as Figure 3 As shown in the tampering probability graph convolution block, the first is a 1x1 convolution layer without bias value, a combination of a normalization layer and a ReLU activation function, followed by two 3x3 convolution layers, a combination of a normalization layer and a ReLU activation function, where the last 3x3 convolution layer has a 2x2 step size for downsampling, and the feature map size is halved after each downsampling.

[0082] Furthermore, the input of the tampered image encoder is the corresponding tampered image, and the output is five layers of feature maps; while the input of the tampered probability map encoder is the tampered probability map, and the output is two parallel feature maps, namely the first feature map and the second feature map in this application. In addition, the tampered image encoder is divided into five tampered image encoder blocks, which are set to e1, e2, e3, e4 and e5 in the order of processing. The input of the first encoder block e1 is the corresponding tampered image, and the output is the first layer feature map. The input of the second encoder block e2 is the first layer feature map output by the first encoder block, and the output is the second layer feature map. The subsequent encoder blocks all use the output of the previous encoder block as their input. The tampered probability map encoder is divided into two tampered probability map encoder blocks, and their inputs are all tampered probability maps. One of the tampered probability map encoder blocks is connected to the actor network, and the output is a layer of feature map, namely the first feature map, which is input into the actor network; the other tampered probability map encoder block is connected to the critic network, and the output is also a layer of feature map, namely the second feature map, which is input into the critic network. It should be noted that the first feature map input to the actor network is used simultaneously by the last two convolutional blocks of the actor network.

[0083] The actor network creates an independent Gaussian distribution for each pixel, that is, the output μ (t) and (σ 2 ) (t) ; then the agent can generate actions a based on them (t) .like Figure 3 As shown, the actor network contains 4 basic decoder blocks, 1 mean block and 1 variance block.

[0084] The basic decoder block upsamples the features output by the tampered image encoder four times, and the size of the feature map is doubled after each upsampling. In addition, in order to accelerate the convergence of the model, the present application connects the first four tampered image encoder blocks with the above four basic decoder blocks. In the basic decoder block, there is first an upsampling layer using nearest neighbor interpolation, followed by a 3x3 convolution layer, a combination of a normalization layer and a ReLU activation function, and then after the connection, it is followed by two 3x3 convolution layers, a combination of a normalization layer and a ReLU activation function.

[0085] Specifically, the input of the actor network includes two parts, one is the five-layer feature map output by the tampered image encoder, and the other is the first feature map output by the tampered probability map encoder block. The output is the mean map and variance map. In the actor network, assuming that the four basic decoder blocks are numbered d1, d2, d3 and d4 in the processing order, then the input of d1 is the output of e5 and e4, and the output is a layer of feature map; the input of d2 is the output of d1 and e3, and the output is a layer of feature map; the input of d3 is the output of d2 and e2, and the output is a layer of feature map; the input of d4 is the output of d3 and e1, and the output is a layer of feature map. That is, each basic decoder block will pass the smaller of the two input feature maps through a 3x3 convolution layer, a normalization layer, and a ReLU activation function to obtain a new feature map, and then the new feature map is spliced ​​with the output of the corresponding tampered image encoder block in the channel dimension to obtain a further feature map, and then the further feature map is passed through two 3x3 convolution layers, a normalization layer, and a ReLU activation function to obtain the output of the current output basic decoder block. Through this connection, the feature maps output by encoder blocks and decoder blocks at different levels can be fully integrated, further improving the feature expression ability of the model and accelerating the convergence of the model.

[0086] The mean block fuses the features output by the tampering probability map encoder and the basic decoder block, and then outputs μ (t) In the mean block, the first is a 3x3 convolution layer, a combination of a normalization layer and a ReLU activation function, followed by an upsampling layer using nearest neighbor interpolation, followed by a 3x3 convolution layer, a combination of a normalization layer and a ReLU activation function, and finally a 3x3 convolution layer, a combination of a normalization layer and a Softsign activation function. The variance block fuses the features output by the tampering probability map encoder and the basic decoder block, and then outputs μ (t) In the variance block, there is first a 3x3 convolutional layer, a combination of a normalization layer and a ReLU activation function, followed by an upsampling layer using nearest neighbor interpolation, followed by a 3x3 convolutional layer, a combination of a normalization layer and a ReLU activation function, and finally a 3x3 convolutional layer, a combination of a normalization layer and a Softplus activation function.

[0087] The critic network is used to estimate the value V of the current state (t) .like Figure 3 As shown in Figure 2, the critic network contains 4 basic decoder blocks and 1 value block; the basic decoder block is the same as that used by the actor network; and the value block fuses the features output by the tampering probability map encoder and the basic decoder block, and then outputs V (t) In the value block, there is first a combination of a 3x3 convolutional layer, a normalization layer, and a ReLU activation function, followed by an upsampling layer with nearest neighbor interpolation, and finally a combination of a 3x3 convolutional layer, a normalization layer, and a Sigmoid activation function.

[0088] Specifically, the input of the critic network consists of two parts: one is the five-layer feature map output by the tampered image encoder, and the other is the second feature map output by the tampered probability map encoder block. The overall output is the value map. The critic network has a similar structure to the actor network, except that the last mean block and variance block are replaced by a value block.

[0089] Furthermore, the training process of the image tampering localization model specifically includes:

[0090] Obtain a training image, construct a state of the current time step according to the training image and the corresponding tampering probability map, input the state of the current time step into the image tampering localization model, and output the training variance, training mean and value corresponding to the current time step;

[0091] The corresponding action is calculated according to the training variance and training mean corresponding to the current time step, the action corresponding to the current time step and the tampering probability map are input into the agent, the updated tampering probability map is output, and the state corresponding to the next time step is constructed based on the updated tampering probability map and the training image;

[0092] According to the state corresponding to the next time step, obtain the state and value corresponding to each time step within the preset time step number;

[0093] According to the states and values ​​corresponding to all time steps within the preset number of time steps, the total loss function is calculated in reverse order, and the parameters of the image tampering localization model are updated using the total loss function;

[0094] According to the training image set, multiple training images in the training image set are obtained, and the image tampering localization model is iteratively trained according to the multiple training images. When the number of training times reaches a preset number of training times, the image tampering localization model obtained from the last training is used as the trained image tampering localization model.

[0095] Specifically, before using a training image for training, the model parameters are initialized and the tampering probability map P f Set to 0, recorded as The initial reward r is also set to 0, because no state transition has occurred at this time, so it is recorded as r (0) .

[0096] When training according to the training image, at the tth time step, the current state s (t) Input into the model, the actor network outputs μ (t) and (σ 2 ) (t) , then according to formula (4) we can get action a (t) ; The corresponding critic network outputs V (t) , the agent's tampering probability map Add a (t) , get the tampering probability map of the next time step According to the tampering probability map and training image of the next time step, the state corresponding to the next time step can be constructed; using the updated state s (t+1) Repeat the process again to complete the repeated operation within the preset time step number, and obtain the state and value corresponding to each time step within the preset time step number. In a preferred embodiment of the present application, the preset time step number is set to 3.

[0097] Then, the total loss function is calculated in reverse order through the states and values ​​corresponding to all time steps within the preset number of time steps, and the parameters of the image tampering localization model are updated using the total loss function.

[0098] Further, the image tampering localization model includes a dual-stream state encoder, an actor network and a critic network, and the dual-stream state encoder includes a tampered image encoder and a tampering probability map encoder;

[0099] The state of the current time step is constructed according to the training image and the corresponding tampering probability map, the state of the current time step is input into the image tampering localization model, and the training variance, training mean and value corresponding to the current time step are output, specifically including:

[0100] Construct the state of the current time step based on the training image and the corresponding tampering probability map;

[0101] Inputting the training image and the tampering probability map in the state of the current time step into the tampered image encoder and the tampering probability map encoder in the image tampering localization model respectively, the tampered image encoder inputs the output into the actor network and the critic network respectively, the tampering probability map encoder outputs a first feature map and a second feature map, and inputs the first feature map into the actor network, and inputs the second feature map into the critic network;

[0102] The actor network outputs the training variance and training mean corresponding to the current time step, and the critic network outputs the value corresponding to the current time step.

[0103] Specifically, based on the structure of the image tampering localization model, the tampering probability map encoder outputs the first feature map and the second feature map, the actor network outputs the training variance and the training mean corresponding to the current time step, and the critic network outputs the value corresponding to the current time step.

[0104] Furthermore, the total loss function is calculated in reverse order according to the states and values ​​corresponding to all time steps within the preset number of time steps, and the parameters of the image tampering positioning model are updated using the total loss function, which specifically includes:

[0105] Calculate the reward graph corresponding to each time step based on the states corresponding to all time steps within the preset number of time steps;

[0106] According to the reward graph and value corresponding to each time step, calculate the strategy loss and value loss corresponding to each time step;

[0107] According to the strategy loss and value loss corresponding to each time step, the total loss function is calculated in reverse order, and the parameters of the image tampering localization model are updated using the total loss function.

[0108] Specifically, the reward graph corresponding to each time step is calculated according to formula (7), and the strategy loss and value loss corresponding to each time step are calculated according to the reward graph and value corresponding to each time step. For each time step, the strategy loss function is shown in formula (8):

[0109]

[0110] Where β = 0.01, f(a (t) ) is to get the action a corresponding to the corresponding time step (t) The probability is as shown in formula (9):

[0111]

[0112] A (t) is the advantage function. When t is the last time step, A (t) =V (t) , when t is not the last time step, A (t) =r (t+1) +γV (t+1) , and γ=0.95.

[0113] The value loss function is shown in formula (10):

[0114]

[0115] According to the strategy loss and value loss corresponding to each time step, when calculating the total loss function in reverse order, the total loss function is initialized to 0 and calculated in reverse order from the last time step and And update the total loss function. The specific update method is shown in formula (11):

[0116]

[0117] Among them, L` total is the total loss function before updating, L″ total is the updated total loss function, and the final total loss function L is obtained total After that, the gradient is calculated according to formula (12) and back propagation is performed, and the network parameters θ are updated at the same time, where the network parameters here are general, indicating that all parameters in the image tampering localization model are updated.

[0118] Wherein, formula (12) is as follows:

[0119] θ=θ+τd θ (L total ); (12)

[0120] Among them, τ is the learning rate, which is initialized to 0.0001, d θ represents the gradient of the parameters.

[0121] Further, in one embodiment of the present application, the agent interacts with the environment for multiple time steps, and the corresponding iterative process is as follows: Figure 4 As shown, the state, action and reward together form a tuple (s (t) , a (t) , r (t+1) ). When s(t) is a terminal state, the complete interaction process between the agent and the environment can be represented as a tuple sequence ((s (0) , a (0) , r (1) ), (s (1) , a (1) , r (2) ),...,(s (t) )).

[0122] The step of calculating the reward graph corresponding to each time step according to the states corresponding to all time steps within the preset number of time steps specifically includes:

[0123] Obtain a tampering probability graph in the state corresponding to all time steps within a preset number of time steps;

[0124] According to all the tampering probability maps within the preset time steps, the defined reward function is used to calculate the reward map corresponding to each time step.

[0125] Specifically, according to the tampering probability graph in the state corresponding to all time steps within the preset time step number, the reward graph r corresponding to each time step is calculated using formula (7), that is, the defined reward function (t+1) .

[0126] After the image tampering localization model is trained, the image tampering localization method based on reinforcement learning in the present application can be executed. In the trained image tampering localization model, in the inference stage, the critic network does not participate in the output process of predicting the mask map.

[0127] Furthermore, in step S2, the agent obtains an initial tampering probability map according to the initial action and the initialized tampering probability map, specifically including:

[0128] Acquiring the initial action and the initialized tampering probability map;

[0129] The intelligent agent performs a sum operation on the initial action and the initialized tampering probability map to obtain an initial tampering probability map.

[0130] Specifically, the intelligent agent adds the initial action and the initialized tampering probability map to obtain a corresponding new tampering probability map, that is, the initial tampering probability map.

[0131] Step S30: According to the second initial state, perform an iterative operation of a preset number of time steps to obtain a final state, wherein in the iterative operation of each time step, output the state corresponding to the current time step according to the trained image tampering localization model and the agent.

[0132] It should be noted that in the present application, the iterative operation of the preset time step is to input the second initial state into the trained image tampering localization model, and update the tampering probability map through the intelligent agent to obtain the corresponding first state, and then cycle to obtain the second state, etc., until the preset time step is reached, and the last state obtained is taken as the final state.

[0133] In a preferred embodiment of the present application, the preset time step number is set to 3.

[0134] Step S40: acquiring a corresponding final tampering probability map according to the final state, performing thresholding on the final tampering probability map to obtain a predicted mask map, and outputting the predicted mask map.

[0135] Specifically, thresholding the final tampering probability map to obtain a predicted mask map specifically includes:

[0136] Obtaining element values ​​of elements in the final tampering probability map;

[0137] According to the preset threshold, the element values ​​in the final tampering probability map are updated to complete the thresholding of the final tampering probability map and obtain the predicted mask map.

[0138] In one embodiment of the present application, the threshold is preset to 0.2, that is, in the final tampering probability map, each element value greater than 0.2 is set to 1, and each element value less than or equal to 0.2 is set to 0, and the corresponding result is the predicted mask map.

[0139] Furthermore, in order to verify the performance of the image tampering localization method based on reinforcement learning in this application, this application conducted an experimental comparison. Four currently state-of-the-art image tampering localization methods were selected for comparison: (1) MVSS-Net, which combines multi-scale feature extraction and visual attention mechanism to improve the image tampering localization performance; (2) PSCC-Net, which proposes an image spatiotemporal correlation module, which models the tampering feature correlation between image space and channels; (3) IF-OSN, which models the noise introduced by social media platforms and improves the robustness of the model in the face of image transmission on social media platforms; (4) CAT-Net, which combines RGB and DCT domains to locate the tampered area. The network captures the image visual artifacts in the RGB domain and the compression artifacts in the DCT domain. It learns the distribution of DCT coefficients without losing spatial information, thereby finely locating the tampered area.

[0140] The experimental data sets are set as follows: COLUMBIA, which is the first public tampered image data set. The number of tampered images is 160, and the sizes are different. This data set uses a random splicing method to obtain tampered images. The tampered area has not undergone any post-processing, and the tampered images are saved in lossless PNG format; COVERAGE is a replication data set containing 100 tampered images. All tampered images have been post-processed to hide traces of manipulation; The CASIA data set provides two versions v1 and v2, containing 960 and 5,123 tampered images respectively. Its tampering methods focus on splicing and copy-paste. Some of them have been carefully post-processed such as filtering and blurring to hide traces of tampering; The NIST16 data set contains 564 images with three types of tampering, namely splicing, copy-paste and removal. The tampered area has been post-processed or not, and the images are JPEG compressed with different quality factors; The IMD2020 data set contains 2010 images that have undergone various operations such as splicing, copy-paste, repair and deformation The tampered images obtained; TampCOCO dataset contains 400,000 tampered images generated by COCO dataset, using two types of tampering: splicing and copy and paste; DSO is a splicing dataset focusing on face tampering, containing 100 tampered images; OSNs dataset contains tampered images from COLUMBIA, NIST16, DSO and CASIAv1, all of which have been transmitted through multiple social media platforms; OpenForensics is a highly challenging dataset for face tampering, and its test set applies various perturbations, such as changes in hue, brightness and saturation, JPEG compression, sharpening, adding noise, etc. to better simulate the environment in natural scenes; CocoGlide dataset is generated using a diffusion model called GLIDE, it contains 512 tampered images; Face2Face, NeuralTextures and FaceSwap datasets are generated by three GAN-based models, they focus on face tampering, each dataset contains 200 tampered images.

[0141] In this experiment, the training set contains 0.5% of the three datasets of TampCOCO, CASIA v2 and IMD2020, with a total of 12,144 tampered images. Our test set includes Columbia, Coverage, NIST16, CASIAv1, DSO, OSNs, CoCoGlide, Face2Face, NeuralTextures and FaceSwap, a total of 9 test datasets. The number of tampered images in each dataset is 160, 100, 564, 920, 100, 1744, 512, 200, 200 and 200 respectively.

[0142] In image tampering localization, F1 and IoU are usually used as evaluation indicators. F1 is the harmonic mean of precision and recall, which is used to measure the accuracy and completeness of the localization results. Its value ranges from 0 to 1, where F1 = 1 indicates perfect localization performance. IoU is the ratio of the intersection and union between the real tampered area and the predicted tampered area, and its value range is also from 0 to 1, where IoU = 1 indicates perfect localization performance.

[0143] Through experiments, the following two tables are obtained.

[0144] Table 1. Generalization performance comparison;

[0145]

[0146] Table 2. Comparison of robustness against image degradation caused by four social networking platforms.

[0147]

[0148] For generalization performance, the pre-trained models provided by other methods are used to evaluate their performance on several test datasets. Among them, the training and test sets used by all methods do not overlap. As shown in Table 1, the CoDE method is significantly better than the state-of-the-art methods; on average, compared with CAT-Net, the method of this application has achieved significant improvements, with F1 increased by 28.2% and IoU increased by 25.1%. In addition, in this experiment, the training set does not contain any data based on generative adversarial networks or diffusion models. Compared with other methods, the CoDE method of this application has stronger generalization performance.

[0149] Online social networking platforms present new and challenging scenarios, where images are often compressed and resized, and may contain unknown noise, resulting in the attenuation of tampering traces in the image. As can be seen from Table 2, the CoDE of this application achieves the highest F1 and IoU in all platforms. In terms of average F1, after the tampered images are transmitted through various social media, CoDE drops by 0.015, 0.015, 0.053, and 0.014, respectively. Compared with other methods, the performance of this application drops very little. In comprehensive comparison, when the tampered images are transmitted through online social networking platforms, the positioning accuracy of other models is greatly reduced. In the face of transmission on different online social networking platforms, the CoDE proposed in this application can still provide reliable tampering positioning results.

[0150] The image tampering localization model completed through training in the present invention can output a predicted mask map corresponding to the target tampered image, so as to identify the tampered position of the target tampered image and realize pixel-level image tampering localization. At the same time, since the trained image tampering localization model is based on the set reward function during training, the predicted mask map corresponding to the present application is more accurate.

[0151] Moreover, the dual-stream state encoder in the image tampering localization model not only significantly improves the performance and greatly accelerates the inference time of the model, but also defines a continuous action space based on Gaussian distribution for exploration, which significantly improves the robustness of the model to various image post-processing attacks, and can effectively cope with them even without corresponding data enhancement; in addition, the reward function also enables the agent to effectively learn the optimal strategy from the feedback reward; and through experimental verification, it can be known that the method proposed in this application is significantly superior to existing methods in resisting image degradation caused by online social network transmission, showing excellent robustness, and has better generalization performance on data based on generative adversarial models or diffusion models.

[0152] Furthermore, if Figure 5 As shown, based on the above-mentioned image tampering positioning method based on reinforcement learning, the present invention also provides an image tampering positioning system based on reinforcement learning, wherein the image tampering positioning system based on reinforcement learning includes:

[0153] An initial state construction module 51 is used to obtain a target tampered image, initialize a tampering probability map, and construct a first initial state according to the target tampered image and the initialized tampering probability map;

[0154] A second initial state construction module 52 is used to input the target tampered image into the trained image tampering localization model, output an initial mean, obtain an initial action according to the initial mean, obtain an initial tampering probability map according to the initial action and the initialized tampering probability map, and construct a second initial state according to the target tampered image and the initial tampering probability map;

[0155] A final state output module 53 is used to perform an iterative operation of a preset number of time steps according to the second initial state to obtain a final state, wherein in the iterative operation of each time step, the state corresponding to the current time step is output according to the trained image tampering localization model and the intelligent agent;

[0156] The result output module 54 is used to obtain the corresponding final tampering probability map according to the final state, and threshold the final tampering probability map to obtain a predicted mask map, and output it.

[0157] Furthermore, if Figure 6As shown, based on the above-mentioned image tampering positioning method and system based on reinforcement learning, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 6 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0158] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, an image tampering positioning program 40 based on reinforcement learning is stored on the memory 20, and the image tampering positioning program 40 based on reinforcement learning can be executed by the processor 10, thereby realizing the image tampering positioning method based on reinforcement learning in the present application.

[0159] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the image tampering localization method based on reinforcement learning.

[0160] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0161] In one embodiment, when the processor 10 executes the image tampering localization program 40 based on reinforcement learning in the memory 20, the steps in the above image tampering localization method based on reinforcement learning are implemented.

[0162] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image tampering localization program based on reinforcement learning, and when the image tampering localization program based on reinforcement learning is executed by a processor, the steps of the image tampering localization method based on reinforcement learning as described above are implemented.

[0163] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or terminal including the element.

[0164] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0165] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for locating image tampering based on reinforcement learning, characterized in that: The image tampering positioning method based on reinforcement learning includes: Acquire a target tampered image, initialize a tampering probability map, and construct a first initial state according to the target tampered image and the initialized tampering probability map; Input the target tampered image into the trained image tampering localization model, output an initial mean, obtain an initial action according to the initial mean, obtain an initial tampering probability map according to the initial action and the initialized tampering probability map, and construct a second initial state according to the target tampered image and the initial tampering probability map; According to the second initial state, an iterative operation of a preset number of time steps is performed to obtain a final state, wherein in the iterative operation of each time step, the state corresponding to the current time step is output according to the trained image tampering localization model and the intelligent agent; Obtaining a corresponding final tampering probability map according to the final state, thresholding the final tampering probability map to obtain a predicted mask map, and outputting the predicted mask map; The training process of the image tampering localization model specifically includes: Obtain a training image, construct a state of a current time step according to the training image and a corresponding tampering probability map, input the state of the current time step into an image tampering localization model, and output a training variance, a training mean, and a value corresponding to the current time step; The corresponding action is calculated according to the training variance and training mean corresponding to the current time step, the action corresponding to the current time step and the tampering probability map are input into the agent, the updated tampering probability map is output, and the state corresponding to the next time step is constructed based on the updated tampering probability map and the training image; According to the state corresponding to the next time step, obtain the state and value corresponding to each time step within the preset time step number; According to the states and values ​​corresponding to all time steps within the preset number of time steps, the total loss function is calculated in reverse order, and the parameters of the image tampering localization model are updated using the total loss function; According to the training image set, a plurality of training images in the training image set are obtained, and the image tampering localization model is iteratively trained according to the plurality of training images, and when the number of training times reaches a preset number of training times, the image tampering localization model obtained by the last training is used as the trained image tampering localization model; The image tampering localization model includes a two-stream state encoder, an actor network and a critic network, and the two-stream state encoder includes a tampered image encoder and a tampering probability map encoder; The state of the current time step is constructed according to the training image and the corresponding tampering probability map, the state of the current time step is input into the image tampering localization model, and the training variance, training mean and value corresponding to the current time step are output, specifically including: Construct the state of the current time step based on the training image and the corresponding tampering probability map; Inputting the training image and the tampering probability map in the state of the current time step into the tampered image encoder and the tampering probability map encoder in the image tampering localization model respectively, the tampered image encoder inputs the output into the actor network and the critic network respectively, the tampering probability map encoder outputs a first feature map and a second feature map, and inputs the first feature map into the actor network, and inputs the second feature map into the critic network; The actor network outputs the training variance and training mean corresponding to the current time step, and the critic network outputs the value corresponding to the current time step.

2. The image tampering location method based on reinforcement learning according to claim 1 is characterized in that: The method further comprises: calculating the total loss function in reverse order according to the states and values ​​corresponding to all time steps within the preset number of time steps, and using the total loss function to update the parameters of the image tampering positioning model. Specifically, the method comprises: Calculate the reward graph corresponding to each time step based on the states corresponding to all time steps within the preset number of time steps; According to the reward graph and value corresponding to each time step, calculate the strategy loss and value loss corresponding to each time step; According to the strategy loss and value loss corresponding to each time step, the total loss function is calculated in reverse order, and the parameters of the image tampering localization model are updated using the total loss function.

3. The image tampering location method based on reinforcement learning according to claim 2 is characterized in that: The step of calculating the reward graph corresponding to each time step according to the states corresponding to all time steps within the preset number of time steps specifically includes: Obtain a tampering probability graph in the state corresponding to all time steps within a preset number of time steps; According to all the tampering probability maps within the preset time steps, the defined reward function is used to calculate the reward map corresponding to each time step.

4. The image tampering location method based on reinforcement learning according to claim 1 is characterized in that: The agent obtains an initial tampering probability map according to the initial action and the initialized tampering probability map, specifically including: Acquiring the initial action and the initialized tampering probability map; The intelligent agent performs a sum operation on the initial action and the initialized tampering probability map to obtain an initial tampering probability map.

5. The image tampering location method based on reinforcement learning according to claim 1 is characterized in that: Thresholding the final tampering probability map to obtain a predicted mask map specifically includes: Obtaining element values ​​of elements in the final tampering probability map; According to the preset threshold, the element values ​​in the final tampering probability map are updated to complete the thresholding of the final tampering probability map and obtain the predicted mask map.

6. An image tampering location system based on reinforcement learning, characterized in that: The image tampering positioning system based on reinforcement learning includes: An initial state construction module, used to obtain a target tampered image, initialize a tampering probability map, and construct a first initial state according to the target tampered image and the initialized tampering probability map; a second initial state construction module, configured to input the target tampered image into the trained image tampering localization model, output an initial mean, obtain an initial action according to the initial mean, obtain an initial tampering probability map according to the initial action and the initialized tampering probability map, and construct a second initial state according to the target tampered image and the initial tampering probability map; A final state output module is used to perform an iterative operation of a preset number of time steps according to the second initial state to obtain a final state, wherein in the iterative operation of each time step, the state corresponding to the current time step is output according to the trained image tampering localization model and the intelligent agent; A result output module is used to obtain a corresponding final tampering probability map according to the final state, and threshold the final tampering probability map to obtain a predicted mask map, and output it; The training process of the image tampering localization model specifically includes: Obtain a training image, construct a state of the current time step according to the training image and the corresponding tampering probability map, input the state of the current time step into the image tampering localization model, and output the training variance, training mean and value corresponding to the current time step; The corresponding action is calculated according to the training variance and training mean corresponding to the current time step, the action corresponding to the current time step and the tampering probability map are input into the agent, the updated tampering probability map is output, and the state corresponding to the next time step is constructed based on the updated tampering probability map and the training image; According to the state corresponding to the next time step, obtain the state and value corresponding to each time step within the preset time step number; According to the states and values ​​corresponding to all time steps within the preset number of time steps, the total loss function is calculated in reverse order, and the parameters of the image tampering localization model are updated using the total loss function; According to the training image set, a plurality of training images in the training image set are obtained, and the image tampering localization model is iteratively trained according to the plurality of training images, and when the number of training times reaches a preset number of training times, the image tampering localization model obtained by the last training is used as the trained image tampering localization model; The image tampering localization model includes a two-stream state encoder, an actor network and a critic network, and the two-stream state encoder includes a tampered image encoder and a tampering probability map encoder; The state of the current time step is constructed according to the training image and the corresponding tampering probability map, the state of the current time step is input into the image tampering localization model, and the training variance, training mean and value corresponding to the current time step are output, specifically including: Construct the state of the current time step based on the training image and the corresponding tampering probability map; Inputting the training image and the tampering probability map in the state of the current time step into the tampered image encoder and the tampering probability map encoder in the image tampering localization model respectively, the tampered image encoder inputs the output into the actor network and the critic network respectively, the tampering probability map encoder outputs a first feature map and a second feature map, and inputs the first feature map into the actor network, and inputs the second feature map into the critic network; The actor network outputs the training variance and training mean corresponding to the current time step, and the critic network outputs the value corresponding to the current time step.

7. A terminal, characterized in that: The terminal includes: a memory, a processor, and a reinforcement learning-based image tampering localization program stored in the memory and executable on the processor. When the reinforcement learning-based image tampering localization program is executed by the processor, the steps of the reinforcement learning-based image tampering localization method as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an image tampering localization program based on reinforcement learning, and when the image tampering localization program based on reinforcement learning is executed by a processor, the steps of the image tampering localization method based on reinforcement learning as described in any one of claims 1 to 5 are implemented.

Images