A sustainable learning image tampering localization method and system integrating edge features

Abstract

The invention discloses a sustainable learning image tampering positioning method integrating edge features, comprising: constructing a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder; obtaining a target image and inputting it into the backbone network for multiple feature extraction to obtain multiple extracted feature maps; inputting all extracted feature maps into the filtering module and the edge feature fusion module in sequence to obtain a predicted tampering region edge mask; inputting the extracted feature maps obtained for the first and last time into the decoder to obtain a predicted tampering region mask; constructing a joint loss function based on the predicted tampering region edge mask and the predicted tampering region mask and training the basic positioning model to obtain an initial positioning model; obtaining a data set and training the initial positioning model for a preset number of times based on the final objective function to obtain a tampering positioning model; inputting the image to be detected into the tampering positioning model to obtain a tampering positioning result map. The detection accuracy and generalization ability are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and more specifically to a sustainable learning image tampering positioning method and system integrating edge features. Background Art

[0002] At present, image forensics technology is committed to detecting and locating tampering traces and areas in images; image tampering is the use of specific image editing techniques and tools to change and forge the content of real images; common image tampering methods mainly include image stitching, copy-move and erase, among which image stitching refers to pasting a certain area in an image into another image, copy-move refers to pasting a certain area in an image to another position in the image, and erase refers to erasing a specified image area and estimating the new pixel value of the area based on the image background. Image forensics methods based on deep learning can automatically learn complex features at different levels of images, can extract more discriminative features, and have significantly better performance than traditional manual features, making image forensics methods based on deep learning gradually become the mainstream method in the field of image forensics.

[0003] However, the effectiveness of deep learning-based image forensics methods in locating tampered areas in images depends to a certain extent on the dataset used during training. When the distribution of the test dataset is inconsistent with the training dataset or there is a domain shift, there is a problem of poor generalization ability.

[0004] In this regard, the existing technology fine-tunes the trained image tampering localization model. Although this helps to improve the effect of the image tampering localization model on the test dataset, it also causes a significant decrease in its effect on the original dataset, that is, catastrophic forgetting occurs. Catastrophic forgetting refers to the fact that the neural network loses the knowledge learned from the previous old tasks during the learning process due to adapting to the new task, which seriously affects its performance on the old task.

[0005] Therefore, how to improve the detection accuracy and generalization ability of image tampering localization while avoiding catastrophic forgetting is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a sustainable learning image tampering localization method and system integrating edge features, which improves the detection accuracy and generalization ability of image tampering localization while avoiding catastrophic forgetting.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A sustainable learning image tampering localization method integrating edge features, comprising:

[0009] Construct a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder;

[0010] Acquire a target image and input it into the backbone network for multiple feature extractions to obtain multiple extracted feature maps;

[0011] Inputting all the extracted feature maps into the filtering module and the edge feature fusion module in sequence to obtain the edge mask of the predicted tampered area;

[0012] Inputting the extracted feature maps obtained by the first and last extractions into the decoder to obtain a predicted tampered area mask;

[0013] Based on the predicted tampered area edge mask and the predicted tampered area mask, a joint loss function is constructed and the basic positioning model is trained to obtain an initial positioning model;

[0014] Acquire a data set of images to be tested and train the initial positioning model for a preset number of times based on a final objective function, and finally obtain a tampering positioning model;

[0015] The image to be detected is obtained and input into the tampering localization model to obtain a tampering localization result map.

[0016] Preferably, the step of obtaining a plurality of extracted feature maps specifically includes:

[0017] The backbone network includes: a first convolutional layer, a maximum pooling layer and four feature extraction modules;

[0018] The target image is sequentially input into the first convolutional layer and the maximum pooling layer for processing to obtain an input feature map;

[0019] The input feature map is input into a first feature extraction module to obtain a first extracted feature map;

[0020] The first extracted feature map is input into a second feature extraction module to obtain a second extracted feature map;

[0021] The second extracted feature map is input into a third feature extraction module to obtain a third extracted feature map;

[0022] The third extracted feature map is input into a fourth feature extraction module to obtain a fourth extracted feature map.

[0023] Preferably, the filtering module comprises four filtering layers of the same structure;

[0024] The first extracted feature map, the second extracted feature map, the third extracted feature map and the fourth extracted feature map are respectively input into one of the filtering layers for feature extraction, and a first edge feature map, a second edge feature map, a third edge feature map and a fourth edge feature map are correspondingly obtained.

[0025] Preferably, obtaining the predicted tampered area edge mask specifically includes:

[0026] The edge feature fusion module includes four processing modules and three fusion modules;

[0027] The first edge feature map is input into a first processing module to obtain a first processed feature map;

[0028] The second edge feature map is input into a second processing module to obtain a second processed feature map;

[0029] The third edge feature map is input into a third processing module to obtain a third processing feature map;

[0030] The fourth edge feature map is input into a fourth processing module to obtain a fourth processing feature map;

[0031] The first processed feature map and the second processed feature map are input into a first fusion module to obtain a first fused feature map;

[0032] The first fused feature map and the third processed feature map are input into a second fusion module to obtain a second fused feature map;

[0033] The second fused feature map and the fourth processed feature map are input into a third fusion module to obtain a third fused feature map;

[0034] The third fused feature map performs an activation function operation to obtain the predicted tampering area edge mask.

[0035] Preferably, the first processing module, the second processing module, the third processing module and the fourth processing module have the same structure, and all include: a second convolution layer, a third convolution layer, a fourth convolution layer and a first upsampling layer;

[0036] The corresponding edge feature map is input into the second convolutional layer to obtain an intermediate result;

[0037] The intermediate results are sequentially input into the third convolutional layer and the fourth convolutional layer to obtain the final result;

[0038] The intermediate result and the final result are fused and input into the first upsampling layer to obtain corresponding processing features.

[0039] Preferably, the first fusion module, the second fusion module and the third fusion module have the same structure, and all include: a fifth convolution layer, a sixth convolution layer and a seventh convolution layer;

[0040] Input the corresponding two types of feature maps for fusion, and input the fusion result into the fifth convolutional layer to obtain the process result;

[0041] Inputting the process results into the sixth convolutional layer and the seventh convolutional layer in sequence to obtain output results;

[0042] The process result and the output result are fused to obtain a corresponding fusion feature map.

[0043] Preferably, the decoder comprises: a pyramid pooling module, an eighth convolutional layer, a second upsampling layer and an output module;

[0044] The fourth extracted feature map is input into the pyramid pooling module to obtain a global fusion feature map;

[0045] The global fusion feature map is input into the second upsampling layer to obtain a first result;

[0046] The first extracted feature map is input into the eighth convolutional layer to obtain a second result;

[0047] The first result and the second result are connected to obtain a connection feature graph;

[0048] The connection feature map is input into the output module to obtain the predicted tampering area mask.

[0049] Preferably, the pyramid pooling module includes: a ninth convolutional layer, a first atrous convolutional layer, a second atrous convolutional layer, a third atrous convolutional layer, a tenth convolutional layer and a global feature extraction module;

[0050] The fourth extracted feature map is respectively input into the ninth convolutional layer, the first atrous convolutional layer, the second atrous convolutional layer and the third atrous convolutional layer, and a first scale feature map, a second scale feature map, a third scale feature map and a fourth scale feature map are correspondingly obtained;

[0051] The first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map are concatenated and input into the tenth convolutional layer to obtain a local feature map;

[0052] The fourth extracted feature is input into the global feature extraction module to obtain a global feature map;

[0053] The local feature map and the global feature map are fused to obtain the global fused feature map.

[0054] Preferably, the global feature extraction module includes: a global average pooling layer, a third upsampling layer and an eleventh convolutional layer;

[0055] The fourth extracted feature is sequentially input into the global average pooling layer, the third upsampling layer and the eleventh convolutional layer to obtain the global feature map.

[0056] A sustainable learning image tampering positioning system integrating edge features, comprising: a model building module, a feature extraction module, an edge mask output module, a regional mask output module, an initial training module, a continuous learning module and a result output module;

[0057] The model building module is used to build a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder;

[0058] The feature extraction module is used to obtain the target image and input it into the backbone network for multiple feature extractions to obtain multiple extracted feature maps;

[0059] The edge mask output module is used to input all the extracted feature maps into the filtering module and the edge feature fusion module in sequence to obtain the edge mask of the predicted tampered area;

[0060] A region mask output module, used for inputting the extracted feature maps obtained by the first and last extractions into the decoder to obtain a predicted tampered region mask;

[0061] The initial training module is used to construct a joint loss function and train the basic positioning model based on the predicted tampered area edge mask and the predicted tampered area mask to obtain an initial positioning model;

[0062] The continuous learning module is used to obtain the image data set to be tested and train the initial positioning model for a preset number of times based on the final objective function to finally obtain a tampering positioning model;

[0063] The result output module is used to obtain the image to be detected and input it into the tampering positioning model to obtain a tampering positioning result map.

[0064] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a sustainable learning image tampering location method and system integrating edge features, which has the following beneficial effects:

[0065] 1. The present invention extracts edge information from the hierarchical feature map of the deep network and calculates the edge loss. By superimposing it with the predicted tampered area loss for joint optimization, the edge information is used to correct the prediction result, thereby further improving the detection accuracy of tampered area positioning.

[0066] 2. The present invention adopts different sampling strategies to store representative samples of previous tasks to alleviate stubbornness and improve the adaptability of the model to new tasks, thereby solving the catastrophic forgetting problem in image tampering localization.

[0067] 3. The present invention adopts a continuous learning method to optimize the training of the image tampering localization model. During the optimization process, the parameter update of the image tampering localization model on the new data set is constrained to retain the knowledge learned by the model on the old data set, so that the model can achieve relatively good results on different data sets, and the image tampering localization model can continuously obtain better generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0069] Figure 1 A flow chart of a sustainable learning image tampering location method integrating edge features provided by the present invention.

[0070] Figure 2 This is a schematic diagram of the basic positioning model structure provided by the present invention.

[0071] Figure 3 This is a schematic diagram of the backbone network structure provided by the present invention.

[0072] Figure 4 This is a schematic diagram of the edge feature fusion module structure provided by the present invention.

[0073] Figure 5 This is a schematic diagram of the processing module structure provided by the present invention.

[0074] Figure 6 This is a schematic diagram of the fusion module structure provided by the present invention.

[0075] Figure 7 This is a schematic diagram of the decoder structure provided by the present invention.

[0076] Figure 8 A schematic diagram of the structure of the pyramid pooling module provided by the present invention.

[0077] Fig. 9 A schematic diagram of the structure of a sustainable learning image tampering location system that integrates edge features provided by the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] Example 1

[0080] Continuous learning (CL) allows the model to continuously learn new knowledge from incoming tasks while retaining the knowledge learned from old tasks as much as possible, so that the model has the ability to adapt to different tasks at the same time, thus helping to solve the problem of catastrophic forgetting.

[0081] To solve the problems existing in the background technology, such as Figure 1 As shown, the embodiment of the present invention discloses a sustainable learning image tampering location method integrating edge features, including:

[0082] Construct a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder;

[0083] Obtain the target image and input it into the backbone network for multiple feature extractions to obtain multiple extracted feature maps;

[0084] All extracted feature maps are sequentially input into the filtering module and the edge feature fusion module to obtain the edge mask of the predicted tampered area;

[0085] The extracted feature maps obtained by the first and last extractions are input into the decoder to obtain the predicted tampered area mask;

[0086] Based on the predicted tampered area edge mask and the predicted tampered area mask, a joint loss function is constructed and the basic positioning model is trained to obtain an initial positioning model;

[0087] Obtain the image data set to be tested and train the initial positioning model for a preset number of times based on the final objective function, and finally obtain the tampering positioning model;

[0088] The image to be detected is input into the tampering localization model to obtain the tampering localization result map.

[0089] Example 2

[0090] The embodiment of the present invention discloses a sustainable learning image tampering location method integrating edge features, including:

[0091] like Figure 2 As shown, a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder is constructed.

[0092] Get the target image and input it into the backbone network for multiple feature extractions to obtain multiple extracted feature maps:

[0093] Preferably, the target image is the image to be detected. In this embodiment, the RGB domain image I∈R 3×H×W , where H and W represent the height and width of the RGB domain image respectively, and 3 represents the number of channels.

[0094] Preferably, a plurality of extracted feature maps are obtained, specifically including:

[0095] like Figure 3 As shown, the backbone network includes: the first convolutional layer, the maximum pooling layer and four feature extraction modules;

[0096] The target image is sequentially input into the first convolutional layer and the maximum pooling layer to obtain the input feature map F0, F0∈R 64 ×(H / 4)×(w / 4) ;

[0097] The input feature map F0 is input into the first feature extraction module to obtain the first extracted feature map F1;

[0098] The first extracted feature map F1 is input into the second feature extraction module to obtain the second extracted feature map F2;

[0099] The second extracted feature map F2 is input into the third feature extraction module to obtain the third extracted feature map F3;

[0100] The third extracted feature map F3 is input to the fourth feature extraction module to obtain the fourth extracted feature map F4.

[0101] Preferably, the first feature extraction module includes three first residual blocks connected in sequence, all of which have the same structure, and the input feature map F0 is sequentially input into the three first residual blocks for processing to obtain a first extracted feature map F1.

[0102] Preferably, the second feature extraction module includes four second residual blocks connected in sequence, all of which have the same structure, and the first extracted feature map F1 is sequentially input into the four second residual blocks for processing to obtain the second extracted feature map F2.

[0103] Preferably, the third feature extraction module includes twenty-three third residual blocks connected in sequence, all of which have the same structure, and the second extracted feature map F2 is sequentially input into the twenty-three third residual blocks for processing to obtain the third extracted feature map F3.

[0104] The fourth feature extraction module includes three fourth residual blocks connected in sequence, all of which have the same structure, and the third extracted feature map F3 is sequentially input into the three fourth residual blocks for processing to obtain a fourth extracted feature map F4.

[0105] All extracted feature maps are input into the filtering module and edge feature fusion module in sequence to obtain the edge mask of the predicted tampered area:

[0106] Preferably, the filter module includes four filter layers with the same structure;

[0107] The first extracted feature map F1, the second extracted feature map F2, the third extracted feature map F3 and the fourth extracted feature map F4 are respectively input into a filter layer for feature extraction, and the first edge feature map, the second edge feature map, the third edge feature map and the fourth edge feature map are correspondingly obtained.

[0108] Preferably, the filtering module comprises a first filtering layer, a second filtering layer, a third filtering layer and a fourth filtering layer;

[0109] The first extracted feature map F1 is input into the first filtering layer to obtain the first edge feature map B1;

[0110] The second extracted feature map F2 is input to the second filtering layer to obtain a second edge feature map B2;

[0111] The third extracted feature map F3 is input to the third filtering layer to obtain a third edge feature map B3;

[0112] The fourth extracted feature map F4 is input to the fourth filtering layer to obtain a fourth edge feature map B4.

[0113] Preferably, in this embodiment, the first filter layer, the second filter layer, the third filter layer and the fourth filter layer all use the Sobel operator based on Gaussian filtering; the edge feature map has the same feature dimension as the extracted feature map.

[0114] Preferably, obtaining the predicted tampered area edge mask specifically includes:

[0115] like Figure 4 As shown, the edge feature fusion module includes four processing modules and three fusion modules;

[0116] The first edge feature map B1 is input into the first processing module to obtain the first processing feature map

[0117] The second edge feature map B2 is input to the second processing module to obtain the second processing feature map

[0118] The third edge feature map B3 is input to the third processing module to obtain the third processing feature map

[0119] The fourth edge feature map B4 is input to the fourth processing module to obtain the fourth processing feature map

[0120] First processing feature map and the second processing feature map Input to the first fusion module to obtain the first fusion feature map R1;

[0121] The first fusion feature map R1 and the third processing feature map Input to the second fusion module to obtain the second fusion feature map R2;

[0122] The second fusion feature map R2 and the fourth processing feature map Input to the third fusion module to obtain the third fusion feature map R3;

[0123] The third fused feature map R3 performs the activation function operation to obtain the predicted tampered area edge mask M edge .

[0124] Preferably, the activation function of this embodiment adopts the Sigmoid activation function.

[0125] Preferably, the first processing module, the second processing module, the third processing module and the fourth processing module have the same structure. Figure 5 As shown, they all include: the second convolution layer, the third convolution layer, the fourth convolution layer and the first upsampling layer

[0126] The corresponding edge feature map is input into the second convolutional layer to obtain the intermediate result

[0127]

[0128] Intermediate results Input into the third convolutional layer and the fourth convolutional layer in sequence to get the final result

[0129] Intermediate results and the final result Fusion is performed and input into the first upsampling layer to obtain the corresponding processing features This embodiment uses element-level addition operation for fusion:

[0130]

[0131] Among them, Conv represents the convolution operation, It represents element-level addition operation and Upsample represents upsampling operation.

[0132] Preferably, the first fusion module, the second fusion module and the third fusion module have the same structure, such as Figure 6 As shown, they all include: a fifth convolutional layer, a sixth convolutional layer and a seventh convolutional layer;

[0133] Input the corresponding two sets of processed feature maps for fusion, and input the fusion result into the fifth convolutional layer to obtain the process result

[0134] The process results Input to the sixth convolutional layer and the seventh convolutional layer in turn to get the output result

[0135] The process results And the output result Fusion is performed to obtain the corresponding fusion feature map R i .

[0136] Preferably, the first processing feature map and the second processing feature map After concatenation, the concatenated feature vector is obtained and combined with the first processed feature map Perform element-wise multiplication and then Perform element-wise addition to obtain the process result:

[0137]

[0138] Among them, Concat represents the connection operation, represents an element-wise multiplication operation, and this process is called merging, and the same applies to the following.

[0139] Preferably, the first processing feature map and the second processing feature map Merge to get the process results And input to the sixth convolutional layer and the seventh convolutional layer to get the output result Process Results And the output result Merge to obtain the corresponding first fusion feature map R1; the first fusion feature map R1 and the third processing feature map Fusion to get process results And input to the sixth convolutional layer and the seventh convolutional layer to get the output result Process Results And the output result The corresponding second fused feature map R2 is obtained by merging; the other fused feature maps are obtained in the same way as above and are not listed here one by one.

[0140] The extracted feature maps obtained by the first and last extractions are input into the decoder to obtain the predicted tampered area mask:

[0141] Preferably, Figure 7 As shown, the decoder includes: a pyramid pooling module, an eighth convolutional layer, a second upsampling layer and an output module;

[0142] The fourth extracted feature map F4 is input into the pyramid pooling module to obtain the global fusion feature map F M , F M The graph has both local features and global features at different scales. M ∈R 256×(H / 16)×(w / 16) ;

[0143] Global fusion feature map F M Input to the second upsampling layer to obtain the first result;

[0144] The first extracted feature map F1 is input into the eighth convolutional layer to obtain the second result;

[0145] The first result and the second result are connected to obtain the connection feature graph

[0146] Connection feature map Input to the output module to obtain the predicted tampered area mask M area .

[0147] Preferably, Figure 8 As shown, the pyramid pooling module includes: a ninth convolutional layer, a first atrous convolutional layer, a second atrous convolutional layer, a third atrous convolutional layer, a tenth convolutional layer and a global feature extraction module;

[0148] The fourth extracted feature map F4 is input to the ninth convolution layer, the first hole convolution layer, the second hole convolution layer and the third hole convolution layer respectively, and the first scale feature map F4 is obtained accordingly. (1) , the second scale feature map F4 (2) , the third scale feature map F4 (3) And the fourth scale feature map F4 (4) ;

[0149] The first scale feature map F4 (1) , the second scale feature map F4 (2) , the third scale feature map F4 (3) And the fourth scale feature map F4 (4) After concatenation, it is input into the tenth convolutional layer to obtain the local feature map F A ;

[0150] The fourth extracted feature map F4 is input into the global feature extraction module to obtain the global feature map F4 (5) ;

[0151] Local feature map F A And the global feature map F4 (5) Fusion obtains the global fusion feature map F M .

[0152] Preferably, in this embodiment, the first scale feature map F4 (1) , the second scale feature map F4 (2) , the third scale feature map F4 (3) , the fourth scale feature map F4 (4) And the global feature map F4 (5) ∈R 256×(H / 16)×(w / 16) , the fusion uses element-wise addition operation.

[0153] Preferably, the first scale feature map F4 (1) , the second scale feature map F4 (2) , the third scale feature map F4 (3) , the fourth scale feature map F4 (4) The receptive fields have different scales and capture contextual information from different scales.

[0154] Preferably, in this embodiment, the convolution kernels of the ninth convolution layer and the tenth convolution layer are both 1×1; the convolution kernel sizes of the first atrous convolution layer, the second atrous convolution layer, and the third atrous convolution layer are all 3×3, and the atrous rates are 6, 12, and 18, respectively.

[0155] Preferably, the first extracted feature map F1 is input to the eighth convolutional layer to perform a convolution operation to reduce the number of feature channels, in order to weaken its relative to F M importance, so as not to connect the feature graph Overly influenced by shallow features.

[0156] Preferably, the global feature extraction module includes: a global average pooling layer, a third upsampling layer and an eleventh convolutional layer;

[0157] The fourth extracted feature map F4 is sequentially input into the global average pooling layer, the third upsampling layer and the eleventh convolution layer to obtain the global feature map F4 (5) .

[0158] Preferably, the output module includes a twelfth convolutional layer, a thirteenth convolutional layer, a fourteenth convolutional layer and a fourth upsampling layer;

[0159] Connection feature map Input to the twelfth convolutional layer, the thirteenth convolutional layer and the fourteenth convolutional layer in turn to obtain the feature map Feature Map The input is processed in the fourth upsampling layer and then the activation function is performed to obtain the predicted tampered area mask M.area .

[0160] Preferably, in this embodiment, the convolution kernel sizes of the twelfth convolution layer, the thirteenth convolution layer and the fourteenth convolution layer are respectively 3×3, 3×3 and 1×1, and the activation function adopts the Sigmoid activation function.

[0161] Based on the predicted tampered area edge mask and the predicted tampered area mask, a joint loss function is constructed and the basic positioning model is trained to obtain the initial positioning model N0:

[0162] Preferably, the edge loss function is calculated based on the predicted tampered area edge mask: edge =f1(G edge ,M edge );Based on the predicted tampered area mask, the tampered area loss function is calculated: L area =f2(G area ,M area );Build a joint loss function based on the edge loss function and the tampered area loss function: L total =aL edge +βL area , where G edge represents the true value mask of the edge of the tampered area, G area represents the true value mask of the tampered area, f1 represents the similarity coefficient loss function, which is used to measure the similarity between the segmentation result generated by the model and the true segmentation label. f2 represents the cross entropy loss function, which is used to measure the difference between the predicted edge and the true edge mask and the similarity between the predicted tampered area mask and the true tampered area. α and β represent the edge loss weight coefficient and the area loss weight coefficient, respectively.

[0163] Obtain the image dataset to be tested and train the initial positioning model for a preset number of times based on the final objective function, and finally obtain the tampering positioning model:

[0164] Preferably, define a task set T = {T1, T2, ..., T n}, obtain the image data set to be tested D = {D1, D2, ..., D n}, the image dataset to be tested includes the image to be tested and its corresponding true value mask, and the trained model set N = {N1, N2, ..., N n}, where n is the number of tasks, D i For the i-th task T i The corresponding image dataset to be tested, N i For the i-th task T i After the model is completed, task T i (i≥2) is expressed as: for the previous task T i-1 The obtained model Ni-1 Use the image dataset D to be tested i And the final objective function is trained to obtain the new model N i , and model N i-1 Compared with Model N i Not only in dataset D i It can achieve good results on datasets D1, D2, ..., D i-1 The above effect will not be affected.

[0165] Preferably, the final objective function L z for:

[0166]

[0167] in, It represents the loss function for the kth task, which is a function of the model parameter θ. P represents the number of model parameters, and λ represents the regularization strength, which controls the degree of influence on the overall objective function. The Fisher information matrix is ​​shown as follows: represents the parameter importance score, θ i represents the i-th model parameter, It represents the value of the i-th element of the model parameter θ after the k-1-th task training is completed.

[0168] Preferably, the final objective function includes the loss of the new task and a regularization term, which aims to maintain knowledge of previous tasks. In order to solve the rigidity problem, different sampling strategies are also used here to store a small number of representative samples, which are used when training new tasks to help the model recall information from old tasks. Continuous learning through the above steps helps the model reduce forgetting and improve its adaptability to new tasks.

[0169] Preferably, for each model N i , define a corresponding parameter θ, for a neural network with parameter θ, for the dataset D of task T i , the Fisher information matrix F can be approximately calculated by the following formula:

[0170]

[0171] F θ represents the Fisher information matrix corresponding to the parameter θ, F θ，i represents the Fisher information of the i-th parameter, Represents the change in the i-th parameter.

[0172] Preferably, KL divergence is used to measure the distribution difference between new and old tasks. For the new dataset D of task TT and the parameters θ of the previous task T-1 , the purpose is to minimize the following objective function:

[0173]

[0174] Among them, l T (θ) represents the loss function of the new task T, λ represents the regularization strength, and D KL represents the KL divergence, It means that after the T-1th task training is completed, the model parameter is θ T-1 The conditional probability distribution when p θ Represents the conditional probability distribution p after the model parameters are updated to θ θ (y|x).

[0175] Preferably, for each parameter update θ i(t) to θ i(t+1) , calculate parameter importance scores

[0176]

[0177] in, represents the change in loss caused by the change in parameter θ, represents the Fisher information of the i-th parameter at time t, Δθ i (t) 2 It represents the square of the change of the i-th parameter at time step t during the parameter update process, and ∈ represents a small positive number to ensure numerical stability.

[0178] The image to be detected is input into the tampering localization model to obtain the tampering localization result map.

[0179] Preferably, the predicted tampered region edge mask is used to identify the edge position in the image. The predicted tampered region edge mask is a binary mask, in which a pixel value of 1 indicates that the pixel is located at the edge, and a pixel value of 0 indicates that it is not an edge; the predicted tampered region mask is used to identify the tampered region in the image. The predicted tampered region mask is a binary mask, in which a pixel value of 1 indicates that the pixel is located in the tampered region, and a pixel value of 0 indicates that it is not a tampered region. The image to be tested is marked by the predicted tampered region mask to obtain a tampering positioning result map.

[0180] Example 3

[0181] like Fig. 9 As shown, a sustainable learning image tampering positioning system integrating edge features includes: a model building module, a feature extraction module, an edge mask output module, a regional mask output module, an initial training module, a continuous learning module and a result output module;

[0182] A model building module is used to build a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder;

[0183] A feature extraction module is used to obtain the target image and input it into the backbone network for multiple feature extractions to obtain multiple extracted feature maps;

[0184] The edge mask output module is used to input all the extracted feature maps into the filtering module and the edge feature fusion module in sequence to obtain the edge mask of the predicted tampered area;

[0185] The region mask output module is used to input the extracted feature maps obtained by the first and last extractions into the decoder to obtain the predicted tampered region mask;

[0186] An initial training module is used to construct a joint loss function and train the basic positioning model based on the predicted tampered area edge mask and the predicted tampered area mask to obtain an initial positioning model;

[0187] A continuous learning module is used to obtain a dataset of images to be tested and train the initial positioning model for a preset number of times based on the final objective function, and finally obtain a tampering positioning model;

[0188] The result output module is used to obtain the image to be detected and input it into the tampering localization model to obtain the tampering localization result map.

[0189] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a sustainable learning image tampering location method and system integrating edge features, which has the following beneficial effects:

[0190] 1. The present invention extracts edge information from the hierarchical feature map of the deep network and calculates the edge loss. By superimposing it with the predicted tampered area loss for joint optimization, the edge information is used to correct the prediction result, thereby further improving the detection accuracy of tampered area positioning.

[0191] 2. The present invention adopts different sampling strategies to store representative samples of previous tasks to alleviate stubbornness and improve the adaptability of the model to new tasks, thereby solving the catastrophic forgetting problem in image tampering localization.

[0192] 3. The present invention adopts a continuous learning method to optimize the training of the image tampering localization model. During the optimization process, the parameter update of the image tampering localization model on the new data set is constrained to retain the knowledge learned by the model on the old data set, so that the model can achieve relatively good results on different data sets, and the image tampering localization model can continuously obtain better generalization performance.

[0193] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0194] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A sustainable learning image tampering location method integrating edge features, characterized in that: include: Construct a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder; Acquire a target image and input it into the backbone network for multiple feature extractions to obtain multiple extracted feature maps; The obtaining of multiple extracted feature maps specifically includes: The backbone network includes: a first convolutional layer, a maximum pooling layer and four feature extraction modules; The target image is sequentially input into the first convolutional layer and the maximum pooling layer for processing to obtain an input feature map; The input feature map is input into a first feature extraction module to obtain a first extracted feature map; The first extracted feature map is input into a second feature extraction module to obtain a second extracted feature map; The second extracted feature map is input into a third feature extraction module to obtain a third extracted feature map; The third extracted feature map is input into a fourth feature extraction module to obtain a fourth extracted feature map; Inputting all the extracted feature maps into the filtering module and the edge feature fusion module in sequence to obtain the edge mask of the predicted tampered area; The filtering module includes four filtering layers with the same structure; The first extracted feature map, the second extracted feature map, the third extracted feature map and the fourth extracted feature map are respectively input into one of the filter layers for feature extraction, and a first edge feature map, a second edge feature map, a third edge feature map and a fourth edge feature map are correspondingly obtained; The step of obtaining the predicted tampered area edge mask specifically includes: The edge feature fusion module includes four processing modules and three fusion modules; The first edge feature map is input into a first processing module to obtain a first processed feature map; The second edge feature map is input into a second processing module to obtain a second processed feature map; The third edge feature map is input into a third processing module to obtain a third processing feature map; The fourth edge feature map is input into a fourth processing module to obtain a fourth processing feature map; The first processed feature map and the second processed feature map are input into a first fusion module to obtain a first fused feature map; The first fused feature map and the third processed feature map are input into a second fusion module to obtain a second fused feature map; The second fused feature map and the fourth processed feature map are input into a third fusion module to obtain a third fused feature map; The third fused feature map performs an activation function operation to obtain an edge mask of the predicted tampered area; Inputting the extracted feature maps obtained by the first and last extractions into the decoder to obtain a predicted tampered area mask; Based on the predicted tampered area edge mask and the predicted tampered area mask, a joint loss function is constructed and the basic positioning model is trained to obtain an initial positioning model; Acquire a data set of images to be tested and train the initial positioning model for a preset number of times based on a final objective function, and finally obtain a tampering positioning model; The image to be detected is obtained and input into the tampering localization model to obtain a tampering localization result map.

2. The method for locating image tampering by continuously learning and integrating edge features according to claim 1 is characterized in that: The first processing module, the second processing module, the third processing module and the fourth processing module have the same structure, and all include: a second convolution layer, a third convolution layer, a fourth convolution layer and a first upsampling layer; The corresponding edge feature map is input into the second convolutional layer to obtain an intermediate result; The intermediate results are sequentially input into the third convolutional layer and the fourth convolutional layer to obtain the final result; The intermediate result and the final result are fused and input into the first upsampling layer to obtain corresponding processing features.

3. The method for locating image tampering by continuously learning and integrating edge features according to claim 1, characterized in that: The first fusion module, the second fusion module and the third fusion module have the same structure, and all include: a fifth convolution layer, a sixth convolution layer and a seventh convolution layer; Input the corresponding two types of feature maps for fusion, and input the fusion result into the fifth convolutional layer to obtain the process result; Inputting the process results into the sixth convolutional layer and the seventh convolutional layer in sequence to obtain output results; The process result and the output result are fused to obtain a corresponding fusion feature map.

4. The method for locating image tampering by continuously learning and integrating edge features according to claim 1, characterized in that: The decoder comprises: a pyramid pooling module, an eighth convolutional layer, a second upsampling layer and an output module; The fourth extracted feature map is input into the pyramid pooling module to obtain a global fusion feature map; The global fusion feature map is input into the second upsampling layer to obtain a first result; The first extracted feature map is input into the eighth convolutional layer to obtain a second result; The first result and the second result are connected to obtain a connection feature graph; The connection feature map is input into the output module to obtain the predicted tampering area mask.

5. The method for locating image tampering by continuously learning and integrating edge features according to claim 4 is characterized in that: The pyramid pooling module includes: a ninth convolutional layer, a first atrous convolutional layer, a second atrous convolutional layer, a third atrous convolutional layer, a tenth convolutional layer and a global feature extraction module; The fourth extracted feature map is respectively input into the ninth convolutional layer, the first atrous convolutional layer, the second atrous convolutional layer and the third atrous convolutional layer, and a first scale feature map, a second scale feature map, a third scale feature map and a fourth scale feature map are correspondingly obtained; The first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map are concatenated and input into the tenth convolutional layer to obtain a local feature map; The fourth extracted feature is input into the global feature extraction module to obtain a global feature map; The local feature map and the global feature map are fused to obtain the global fused feature map.

6. The method for locating image tampering by continuously learning and integrating edge features according to claim 5, characterized in that: The global feature extraction module includes: a global average pooling layer, a third upsampling layer and an eleventh convolutional layer; The fourth extracted feature is sequentially input into the global average pooling layer, the third upsampling layer and the eleventh convolutional layer to obtain the global feature map.

7. A sustainable learning image tampering location system integrating edge features, characterized in that: A sustainable learning image tampering positioning method based on any one of claims 1 to 6 that integrates edge features, comprising: a model building module, a feature extraction module, an edge mask output module, a regional mask output module, an initial training module, a continuous learning module and a result output module; The model building module is used to build a basic positioning model including a backbone network, a filtering module, an edge feature fusion module and a decoder; The feature extraction module is used to obtain the target image and input it into the backbone network for multiple feature extractions to obtain multiple extracted feature maps; The edge mask output module is used to input all the extracted feature maps into the filtering module and the edge feature fusion module in sequence to obtain the edge mask of the predicted tampered area; A region mask output module, used for inputting the extracted feature maps obtained by the first and last extractions into the decoder to obtain a predicted tampered region mask; The initial training module is used to construct a joint loss function and train the basic positioning model based on the predicted tampered area edge mask and the predicted tampered area mask to obtain an initial positioning model; The continuous learning module is used to obtain the image data set to be tested and train the initial positioning model for a preset number of times based on the final objective function to finally obtain a tampering positioning model; The result output module is used to obtain the image to be detected and input it into the tampering positioning model to obtain a tampering positioning result map.

Images