A training method of a forged image detection model based on a key forgetting mechanism

Abstract

A training method for a forgery detection model based on a key forgetting mechanism is disclosed. The forgery detection model includes at least an attribute extractor and a key forgetting module, comprising: acquiring a training sample set containing multiple sample images with true / false labels; using the attribute extractor to extract attributes from the sample images to obtain attribute feature maps, which represent information related to forgery traces in the sample images; using the key forgetting module to obtain the influence degree of each pixel in the attribute feature map on the prediction result, determining key pixels in the attribute feature map based on the influence degree of each pixel, and weakening the influence degree of key pixels; using a classifier in the key forgetting module to predict the true / false status of the weakened attribute feature map to obtain predicted labels; and training the forgery detection model based on the difference between the predicted labels and the true / false labels.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of image processing technology, and in particular relate to a training method for a forged image detection model based on a key forgetting mechanism. Background Technology

[0002] With the development of artificial intelligence (AI) technology, AI-based content generation technologies are becoming increasingly sophisticated, producing more and more realistic content that is increasingly difficult to distinguish from its authenticity. For example, deepfake technology uses AI to generate seemingly real fake images or videos. Malicious actors may use deepfake technology to generate fake images or videos for malicious purposes such as impersonating others, creating fake news, and committing fraud. Therefore, a method to detect fake images is needed.

[0003] The accuracy of related technologies in detecting forged images often depends on the richness of the forgery techniques covered in the training set. When new forgery methods emerge, the detection effect is poor. Summary of the Invention

[0004] The purpose of this invention is to provide a training method and a forgery detection method for a forgery image detection model based on a key forgetting mechanism. By forcing the model to learn the features of common forgery traces, the detection performance of the model is improved when facing different types of forgery images.

[0005] The first aspect of this specification provides a training method for a forged image detection model based on a key forgetting mechanism. The forged image detection model includes at least an attribute extractor and a key forgetting module, comprising:

[0006] Obtain a training sample set, which contains multiple sample images. Each sample image has a real / fake label, which is used to indicate whether a sample image is a fake image.

[0007] The attribute extractor extracts attributes from the sample image to obtain an attribute feature map, which is used to represent information related to forgery traces in the sample image.

[0008] The key forgetting module obtains the degree of influence of each pixel in the attribute feature map on the prediction result, determines the key pixels in the attribute feature map based on the degree of influence of each pixel, and weakens the degree of influence of the key pixels.

[0009] The classifier in the key forgetting module performs true / false prediction on the weakened attribute feature map to obtain the predicted label;

[0010] The forged image detection model is trained based on the difference between predicted labels and genuine / fake labels.

[0011] The second aspect of this specification provides a method for detecting forged images based on a key forgetting mechanism, including:

[0012] The target image to be detected and the forged image detection model trained based on the method of the first aspect are obtained. The forged image detection model includes at least an attribute extractor and a key forgetting module.

[0013] The attribute extractor extracts attributes from the target image to obtain a target attribute feature map, which is used to represent information related to forgery traces in the target image.

[0014] The classifier in the key forgetting module performs a true / false prediction on the target attribute feature map to obtain the detection result, which is used to indicate whether the target image is a forged image.

[0015] A third aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method as described in the first or second aspect.

[0016] The technical solution provided in this specification provides a training method for a forged image detection model based on a key forgetting mechanism. Through a key forgetting module, based on the degree of influence of each pixel in the attribute feature map on the prediction result, key pixels in the attribute feature map are weakened. This forces the model to forget the key parts of the attribute feature map related to forgery during the training phase. These key parts correspond to the forgery techniques used in the sample images in the training sample set, thereby forcing the model to learn more common and universal features of forgery traces, improving the model's generalization ability, and enabling the model to achieve better detection results when facing different types of forged images. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a training method for a forged image detection model based on a key forgetting mechanism, as described in one embodiment of this specification.

[0019] Figure 2 This is a schematic diagram of a spoofing detection model in one embodiment of this specification;

[0020] Figure 3 This is a schematic diagram of a spoofing detection model in one embodiment of this specification;

[0021] Figure 4 This is a schematic diagram of a spoofing detection model in one embodiment of this specification;

[0022] Figure 5 This is a schematic diagram of a spoofing detection model in one embodiment of this specification;

[0023] Figure 6 This is a schematic diagram of the architecture of a forgery detection model in one embodiment of this specification;

[0024] Figure 7 This is a flowchart illustrating another training method for a forged image detection model based on a key forgetting mechanism in one embodiment of this specification.

[0025] Figure 8 This is a flowchart illustrating a forged image detection method based on a key forgetting mechanism in one embodiment of this specification.

[0026] Figure 9 This is a schematic diagram of a training device for a forged image detection model based on a key forgetting mechanism, as described in one embodiment of this specification.

[0027] Figure 10 This is a schematic diagram of a forged image detection device based on a key forgetting mechanism in one embodiment of this specification. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0029] As mentioned earlier, with the development of artificial intelligence, images forged by Deepfake technology are becoming increasingly realistic. Current forgery detection methods focus on learning the specific forgery types contained in the training set, which results in poor generalization performance of the trained models and poor detection entries for other forgery types outside the training set. However, it is foreseeable that with the continuous advancement of Deepfake technology, forged images will incorporate more complex and diverse forgery methods, thus necessitating an improvement in the generalization performance of current forgery detection models.

[0030] Based on the above analysis, considering that current forgery detection models easily capture specific forgery traces in the training set by extracting features during training, thus causing the model to tend to learn the specific forgery types contained in the training set, this specification proposes a training method for a forgery image detection model based on a key forgetting mechanism. This method forces the model's classifier to forget the significant features captured during training. These significant features are often related to specific forgery types in the training set, thereby enabling the model to reveal features of more common and generalized forgery traces that were previously overlooked. These features are widely present in various types of forgery images, greatly improving the model's generalization ability.

[0031] The following is combined Figure 1 The training process of the forged image detection model in the embodiments of this specification is described. Figure 1 This is a flowchart of the training method for the forged image detection model based on the key forgetting mechanism in the embodiments of this specification. This training process can be executed by any device, platform, or device cluster with computing and processing capabilities, including steps S101-S105 as shown below. Figure 2 The diagram shows the network structure of the forgery detection model. The forgery detection model in the embodiments of this specification includes at least an attribute extractor and a key forgetting module.

[0032] like Figure 1 As shown, in step S101, a training sample set is obtained. The training sample set contains multiple sample images. The sample images have real and fake labels, which are used to indicate whether the sample image is a fake image.

[0033] In practice, the training sample set contains multiple sample images, including real images and fake images. Fake images can be created through image editing, deepfake techniques such as deepfake networks, etc. The real image can be labeled as "real", and the fake image can be labeled as "fake".

[0034] This embodiment does not limit the objects that are forged in the forged image. For example, the forged image may contain forged or replaced faces, forged scenes, forged actions, or forged objects.

[0035] Next, in step S102, the attribute extractor extracts attributes based on the sample image to obtain an attribute feature map, which is used to represent information related to forgery traces in the sample image.

[0036] The feature information in the sample image can be divided into information related to forgery traces and information unrelated to forgery traces. Among them, the attribute extractor is used to separate the information related to forgery traces in the sample image.

[0037] This embodiment does not limit the network structure of the attribute extractor; for example, it can use a convolutional neural network, a deep autoencoder, or other similar structures.

[0038] Next, in step S103, the key forgetting module obtains the degree of influence of each pixel in the attribute feature map on the prediction result, determines the key pixels in the attribute feature map based on the degree of influence of each pixel, and weakens the degree of influence of the key pixels.

[0039] In this context, each pixel in the attribute feature map represents a forgery trace captured from the sample image. For example, when an image is stitched or tampered with, the edges at the stitching points may not be smooth, causing the pixel values ​​of some pixels in the attribute feature map to reflect abnormal edges or inconsistent textures in that area. The prediction result refers to the output of the classifier in the key forgetting module, which performs a true / false prediction.

[0040] Since specific forgery traces in the training sample set are easily captured—meaning that compared to subtle, common forgery traces—the specific forgery traces on the sample images in the training sample set have a relatively large impact on the prediction results—we can determine the key pixels in the attribute feature map related to the specific forgery traces in the training sample set by analyzing the degree of influence of each pixel on the prediction results during training. Specifically, the degree of influence on the prediction results refers to the impact on the accuracy of the prediction result when the value of the channel corresponding to a pixel changes. For example, when the sample image is an RGB image, the channel value corresponding to a pixel can refer to the value of each pixel in the image on different color channels (Red, Green, Blue).

[0041] In practice, the influence of each pixel in the attribute feature map on the prediction result can be quantified as the gradient of each pixel in the attribute feature map during backpropagation. The magnitude of the influence is represented by the magnitude of the gradient. The larger the gradient, the greater the influence. Since specific forgery traces in the training sample set are easy to capture, they are usually represented by the pixels with the largest gradients in the attribute feature map during backpropagation.

[0042] In one implementation, this step can specifically be based on the difference between the prediction result and the true and false labels to determine the gradient corresponding to each pixel in the attribute feature map during the backpropagation process. For each pixel in the attribute feature map, if the gradient of the pixel is higher than a preset threshold, the pixel is determined as a key pixel, and the value of the channel corresponding to the key pixel in the attribute feature map is set as a weakened value.

[0043] Specifically, the prediction result refers to the prediction result output by the classifier of the key forgetting module during the forward propagation process of the previous iteration of training. The loss function is calculated by the difference between the prediction result and the true and false labels. In this embodiment, the gradient corresponding to each pixel represents the derivative of the loss function with respect to the input of each neuron (i.e., each pixel in the attribute feature map can be regarded as the input of each corresponding neuron in this layer of the network). For example, the formula for calculating the gradient G corresponding to each pixel in the attribute feature map is as follows:

[0044]

[0045] Where N is the number of sample images in a batch of the training sample set, and y i p is the true / false label corresponding to the i-th sample image. i F is the prediction result of the classifier of the key detection module corresponding to the i-th sample image. a For the attribute feature map, the loss function used is the BCE (Binary Cross-Entropy) loss function, i.e., -[y i log(p i )+(1-y i log(1-p) i This formula can be derived by applying the chain rule to F. a The gradient of each pixel is obtained by taking the partial derivative of each pixel.

[0046] It should be noted that the gradient G used in this embodiment is the same as the gradient of the loss function used in gradient descent during backpropagation with respect to each parameter (such as weight w). Not entirely the same; their relationship can be viewed as a chain rule.

[0047] For attribute feature map F a The pixels in the attribute feature map can be sorted according to the magnitude of the gradient. Then, by setting a preset threshold, the pixels in the attribute feature map are filtered, and the channel values ​​corresponding to key pixels are set to weakened values ​​to force the forgetting of some pixels with greater influence in the attribute feature map. For example, this weakening process can be achieved by using a mask to mask the attribute feature map. The masking process is shown in the following formula:

[0048]

[0049] Wherein, Sort represents performing F according to the magnitude of gradient G. a Sorting of each pixel in the image This represents the dot product operation, F′ aThis represents the attribute feature map after weakening, also known as the forgotten feature map, M. σ This is a 0-1 matrix, which can be viewed as a mask. σ is a hyperparameter used to control the forgetting ratio. θ is a preset threshold selected from the sorted gradients based on the hyperparameter σ. i,j represent the pixel positions in the attribute feature map. Indicates mask M σ The value at position i,j in the middle. For example, M σ＝0.3 This means that in the attribute feature map, the top 30% of pixels in terms of influence (gradient) are identified as key pixels, and the channel values ​​corresponding to the key pixels are set to a weakened value of 0, while the channel values ​​corresponding to the remaining pixels in the attribute feature map remain unchanged. In this way, by masking channels whose gradients exceed a preset threshold, in other examples, 0 in formula (3) can also be set to other values ​​between 0 and 1 to weaken the influence of key pixels on the prediction results.

[0050] In other implementations, the weakening of key pixels can also be the enhancement of other pixels in the attribute feature map, such as setting 0 in formula (3) to 1 and setting 1 to other values ​​greater than 1, so as to enhance the influence of other pixels on the prediction result.

[0051] Continuing, in step S104, the classifier in the key forgetting module performs true / false prediction on the weakened attribute feature map to obtain the predicted label.

[0052] In this step, the weakened attribute feature map F a The input is a classifier, and the output is a predicted label. The classifier is used to identify which label {fake, real} the weakened attribute feature map belongs to. The predicted label represents the probability that the sample image is a fake image or a real image. For example, the predicted label can be a value between 0 and 1. The closer the predicted label is to 0, the more likely the model thinks the sample image is a fake image. The closer the predicted label is to 1, the more likely the model thinks the sample image is a real image.

[0053] To avoid excessive forgetting in the critical forgetting module potentially making the classifier insensitive to fake types in the training sample set, thus affecting the model's performance in detecting fake types within the training sample set, one approach is to concatenate the attribute feature map and a weakened attribute feature map before using the classifier for prediction. Then, the concatenated attribute feature map is used for true / false prediction to obtain the predicted label. The attribute feature map F... a Compared with the weakened forgetting feature map F a Instead of using the forgotten feature map F alone, the feature maps F are connected together and used as input to the classifier. aThis approach involves forgery classification, enabling the model to learn features specific to the forgery type within the training sample set while simultaneously learning overlooked, common forgery traces. Furthermore, this process effectively generates two attribute feature maps with varying detection difficulty for each sample image, which can also be viewed as a feature-level data augmentation strategy.

[0054] Finally, in step S105, the forged image detection model is trained based on the difference between the predicted label and the real / fake label.

[0055] Specifically, a loss function can be calculated based on the difference between the predicted label and the true / false label, and the forged image detection model can be trained using the loss function.

[0056] This embodiment does not restrict the specific loss function used. After determining the loss function, the parameters of the forged image detection model can be adjusted through backpropagation based on that loss function. Network training ends when the network iterations reach the training termination condition. This embodiment does not restrict the training termination condition. This condition can be reaching a certain number of iterations or the loss function value falling below a certain threshold.

[0057] For example, the loss function can be a classification loss function, such as the BCE loss function, when the concatenated attribute feature map F is used in step S104. a With forgetting feature map F a When used as input to a classifier, the loss function can be expressed as follows:

[0058]

[0059] Where Con represents the splicing operation, BCE represents the BCE loss function, and y∈{fake, real} are the real and fake labels of the sample images.

[0060] The training method for a forgery detection model based on a key forgetting mechanism provided in this specification uses a key forgetting module to weaken key pixels in the attribute feature map based on the degree of influence of each pixel on the prediction result. This forces the model to forget the key parts of the attribute feature map related to forgery during the training phase. These key parts correspond to the forgery techniques used in the sample images in the training sample set, thereby forcing the model to learn more common and universal features of forgery traces, avoiding overfitting of the model to the sample data in the training set, improving the model's generalization ability, and enabling the model to achieve better detection results when facing different types of forgery images.

[0061] In the context of forgery detection, decoupling aims to separate forgery-related attribute information from the original image and content information unrelated to forgery, thereby effectively identifying forgery images through attribute information. For example, in current forgery detection methods based on decoupled representations, attribute features and content features are extracted separately for a pair of real and forged images. Then, the content features of one image are cross-reconstructed with the attribute features of the other image to ensure that the attribute features are related to forgery, while the content features are unrelated. Besides the poor generalization performance of the trained model, this method also has other problems. For instance, using information-intensive images as the decoupling target increases the difficulty of decoupling; this method assumes that information unrelated to reconstruction is related to forgery, and the extracted attribute features tend to be unrelated to image reconstruction rather than related to forgery information.

[0062] As one implementation method, such as Figure 3 As shown, the forgery image detection model also includes an encoder. To reduce the difficulty of decoupling, the encoder can be used to convert the sample image into a feature representation, thereby decoupling based on the feature representation instead of the sample image. In step 102, the encoder can extract features from the sample image to obtain the feature representation; the attribute extractor can then extract attributes from the feature representation to obtain an attribute feature map. Compared to the dense and complex information in the sample image, the feature representation contains less information, which is more conducive to the separation of attribute feature maps related to forgery.

[0063] As one implementation method, such as Figure 4 As shown, the forgery image detection model also includes a content extractor. The method may further include: the content extractor extracting content from the feature representation to obtain a content feature map, which represents information unrelated to forgery traces in the sample image. To ensure that the attribute features separated from the sample image are more likely to be related to forgery, rather than just unrelated to image reconstruction, and to avoid information unrelated to forgery dominating attribute features, step S105 may train the forgery image detection model based on the difference between predicted labels and genuine / fake labels, as well as the irrelevance between the content feature map and the attribute feature map.

[0064] In practice, the loss function can include not only the classification loss function calculated based on the difference between predicted labels and true / false labels, but also the irrelevance loss function calculated based on the irrelevance between content feature maps and attribute feature maps. The stronger the irrelevance between content feature maps and attribute feature maps, the smaller the value of the irrelevance loss function, and the weaker the irrelevance, the larger the value of the irrelevance loss function. By continuously reducing the value of the loss function through training, the content feature maps and attribute feature maps can be separated as much as possible.

[0065] For example, when calculating irrelevance, the irrelevance between content feature maps and attribute feature maps can be calculated using methods such as Kullback-Leibler Divergence or Spearman's Rank Correlation. This embodiment does not limit the specific calculation method used.

[0066] For example, the Pearson correlation coefficient can be introduced in the decoupling phase and used to distinguish between attribute feature maps that are related to forgery and content feature maps that are not related to forgery. This makes it easier for the decoupler to isolate content feature maps and attribute feature maps. The classification loss and the Pearson correlation coefficient loss will force information related to forgery to dominate the attribute feature map, while information unrelated to forgery will gradually shift to the content feature map.

[0067] As one implementation method, such as Figure 5 As shown, the forged image detection model also includes a decoder. In order to ensure that the information in the feature representation is completely extracted into the attribute feature map and the content feature map respectively, the method may further include: inputting the attribute feature map and the content feature map into the decoder for reconstruction to obtain the reconstructed image; step S105 may be to train the forged image detection model based on the difference between the reconstructed image and the sample image, as well as the difference between the predicted label and the real and fake labels.

[0068] By using a decoder to reconstruct the attribute feature map and content feature map into a reconstructed image, it can be ensured that the attribute feature map and content feature map are a complete representation of the sample image, ensuring that the information in the sample image is fully utilized. Specifically, based on the difference between the reconstructed image and the sample image, a reconstruction loss function can be calculated. The smaller the difference between the reconstructed image and the sample image, the smaller the value of the reconstruction loss function; conversely, the larger the difference between the reconstructed image and the sample image, the larger the value of the reconstruction loss function. By using the reconstruction loss function and the classification loss function as a whole loss function for model training, the reconstructed image becomes increasingly closer to the sample image during training.

[0069] Considering that single-level decoupling methods cannot capture enough discriminative information to separate attribute feature maps and content feature maps, this specification proposes another training method for a forgery image detection model based on a key forgetting mechanism. This method combines multi-level decoupling methods to separate attribute feature maps and content feature maps, so that the extracted attribute feature maps can better gather information related to forgery in the image.

[0070] Below, in conjunction with Figure 6 The framework diagram of the forged image detection model illustrates the complete training process of the model. Figure 7 This is another flowchart illustrating the training method for the forged image detection model based on a key forgetting mechanism in the embodiments of this specification. This method can be executed by any device, platform, or cluster of devices with computing and processing capabilities, including steps S701-S708 as shown below.

[0071] like Figure 7 As shown, in step S701, a training sample set is obtained. The training sample set contains multiple sample images. The sample images have real and fake labels, which are used to indicate whether the sample image is a fake image.

[0072] Unlike existing forgery detection methods based on decoupled representations that require pairs of images as input, the method in this specification's embodiments inputs a single sample image x to the model each time.

[0073] Next, in step S702, the encoder extracts features from the sample image to obtain feature representations at multiple levels. The feature representations at different levels are used to reflect different aspects of image feature information.

[0074] Among them, encoder E n It can segment dense information in sample images and extract feature representations at multiple levels f. i It can independently focus on different aspects of image features such as facial contours, low-frequency information, texture, color, or edges. For example, encoder E... n The output can be shown below:

[0075] f1,…,f n =E n (x) (5)

[0076] Among them, f i ,i∈[1,…,n] represents the n feature representations of the network outputs of different layers in the encoder. For example, the feature representation of the first layer network output near the input end in the encoder is f1, and the feature representation of the second layer network output is f2.

[0077] Next, in step S703, for each level of feature representation, the attribute extractor corresponding to that level extracts attributes from the feature representation to obtain the initial attribute feature map corresponding to that level. For each level of feature representation, the content extractor corresponding to that level extracts content from the feature representation to obtain the initial content feature map corresponding to that level.

[0078] This embodiment extends the decoupling of forgery representation from a single level to multiple levels. The forged image detection model includes multiple levels of attribute extractors, content extractors, and decoders, with the same number of attribute extractors, content extractors, and decoders. For example... Figure 6As shown, the encoder extracts three layers of feature representations, and the model contains three attribute extractors, content extractors, and decoders.

[0079] The decoupler at level i contains an attribute extractor. and a content extractor For example, they can have the same network structure but do not share parameters. The content extractor is used to extract initial content feature maps unrelated to forgery. Attribute extractors are used to separate initial attribute feature maps associated with forgery. As shown in the formula below:

[0080]

[0081] As one implementation approach, the Pearson correlation coefficient can be used as a separation constraint between the initial content feature map and the initial attribute feature map to calculate their independence. A smaller Pearson correlation coefficient indicates a greater difference in direction and magnitude between the two feature maps; a sufficiently large difference can avoid potential correlations between attribute and content features. The Pearson correlation coefficient loss function for each level can be expressed as:

[0082]

[0083] Where Cov represents the covariance between the initial attribute feature map and the initial content feature map, and σ represents the variance. By constraining the loss function using the Pearson correlation coefficient, iterative training can ensure that information related to forgery dominates the attribute feature map, while information unrelated to forgery dominates the content feature map.

[0084] As one implementation method, it can be done in each initial attribute feature map. Each subsequent step connects to a corresponding classifier to ensure its relevance to forgery, further enhancing the correlation between the attribute feature map and the forged information. The forged image detection model also includes classifiers corresponding to multiple levels; for the initial attribute feature map at each level, the classifier corresponding to that level performs a true / false prediction on the initial attribute feature map to obtain the initial predicted label.

[0085] For example, the classifier corresponding to each level will predict the initial predicted label from the initial attribute feature map of that level. The initial classification loss function for each level is calculated based on the difference between the initial predicted label and the true and false labels, as shown below:

[0086]

[0087] Where BCE is the cross-entropy loss function, and y∈{fake, real} are the real and fake labels of the sample image x.

[0088] Next, in step S704, attribute feature maps are obtained based on multiple initial attribute feature maps corresponding to multiple levels.

[0089] like Figure 6 As shown, it can be and Perform a connection to obtain the attribute feature map F. a In other embodiments, multiple initial attribute feature maps can be weighted and then concatenated to obtain the attribute feature map.

[0090] In step S705, for each level, the initial attribute feature map and initial content feature map corresponding to that level are input into the decoder corresponding to that level for reconstruction to obtain reconstructed features or reconstructed image.

[0091] The purpose of the decoder is to ensure that the initial attribute feature map and initial content feature map obtained by decoupling are complete representations of the sample image or feature representation. Different constraints can be applied to different levels of decoupling, and the reconstruction loss can be extended from the image level to the feature level, thereby extending the decoupling of the forged representation from a single level to multiple levels, rather than using only image-level reconstruction constraints.

[0092] Specifically, each level of feature representation has a corresponding decoder. decoder The aim is to ensure the initial attribute feature map and initial content feature map The information in the image is the complete representation of the feature representation at that level, not the sample image, unless i=1. That is, when i=1, the initial attribute feature map and the initial content feature map are input into the encoder of the first level for reconstruction to obtain the reconstructed image. When i>1, the initial attribute feature map and the initial content feature map are input into the encoder of the corresponding level for reconstruction to obtain the reconstructed features. In other words, except for the sample image, the reconstruction target of other levels is the feature representation of that level. The reconstruction loss function calculated based on the differences between the reconstructed features and feature representations of multiple levels, and the reconstruction loss function calculated based on the differences between the reconstructed image and the sample image, can be expressed as:

[0093]

[0094] Here, Mse represents the Mean Square Error loss function.

[0095] In addition, in step S706, the key forgetting module obtains the degree of influence of each pixel in the attribute feature map on the prediction result, determines the key pixels in the attribute feature map based on the degree of influence of each pixel, and weakens the degree of influence of the key pixels.

[0096] For an explanation of step S706, please refer to the relevant explanation of step S103 above, which will not be repeated here.

[0097] In step S707, the classifier in the key forgetting module performs true / false prediction on the weakened attribute feature map to obtain the predicted label.

[0098] like Figure 6 As shown in the key forgetting module, the forgetting feature map F is obtained after weakening processing. a After that, the attribute feature map F can be... a With forgetting feature map F a The final attribute feature map F is obtained by concatenating the features. a ", Use a classifier for F" a "Make a prediction and obtain the prediction label."

[0099] For an explanation of step S707, please refer to the relevant explanation of step S104 above, which will not be repeated here.

[0100] In step S708, the forged image detection model is trained based on the difference between the predicted label and the real / fake label.

[0101] The classification loss function calculated based on the difference between the predicted label and the true / false label can be shown in formula (4). This step can be implemented in multiple ways, and different overall loss functions can be calculated based on different aspects of the loss function.

[0102] One approach is to train a fake image detection model based on the differences between the reconstructed image and the sample image, the differences in reconstructed features and feature representations at multiple levels, and the differences between predicted labels and true / false labels.

[0103] The reconstruction loss function, calculated based on the differences between the reconstructed image and the sample image, and the differences in reconstruction features and feature representations at multiple levels, can be shown in Equation (9). Alternatively, the classification loss function and the reconstruction loss function can be weighted and summed to obtain the overall loss function, which is then used to train the forged image detection model.

[0104] One approach is to train a fake image detection model based on the difference between the predicted label and the real / fake label, as well as the difference between the initial predicted label and the real / fake label.

[0105] The initial classification loss function for each level, calculated based on the difference between the initial predicted labels and the true / false labels at multiple levels, can be shown in Equation (8). Alternatively, the classification loss function and the initial classification loss function can be weighted and summed to obtain the overall loss function, which is then used to train the fake image detection model.

[0106] One approach is to train a fake image detection model based on the differences between predicted and genuine / fake labels, as well as the independence between multiple levels of content feature maps and the initial attribute feature map.

[0107] The irrelevance loss function, calculated based on the irrelevance between the content feature maps at multiple levels and the initial attribute feature map, can be shown in Equation (7). It can be a weighted sum of the classification loss function and the irrelevance loss function to obtain the overall loss function, and the forged image detection model can be trained based on this overall loss function.

[0108] As one implementation approach, contrastive learning can be introduced to further improve generalization performance. For multiple sample images with the same true / false labels, the intra-class similarity between the weakened attribute feature maps corresponding to each sample image is calculated; for multiple sample images with different true / false labels, the inter-class similarity between the weakened attribute feature maps corresponding to each sample image is calculated. Contrastive learning maximizes the intra-class similarity *s*. p Minimize inter-class similarity s n The similarity can be calculated using the weakened attribute feature map F. a The distance between classes is used to measure the similarity of features within a class and the dissimilarity of features between classes. For example, given a batch of training samples with K pairs of similar samples (i.e., image pairs with the same true / false labels) and L pairs of different samples (i.e., image pairs with different true / false labels), the contrastive learning loss function can be expressed as:

[0109]

[0110] Where γ is the scaling factor and m is the threshold used for similarity.

[0111] The forgery detection model is trained based on intra-class similarity, inter-class similarity, and the difference between predicted labels and genuine / fake labels. This can be achieved by weighted summation of the classification loss function and the contrastive learning loss function to obtain an overall loss function, which is then used to train the forgery detection model.

[0112] As one implementation, the overall loss function used in the training process is a weighted sum of the four loss functions listed above and the classification loss function:

[0113]

[0114] Among them, λ1, λ2, λ3 and λ4 are hyperparameters used to measure the impact of each loss function on the overall loss function.

[0115] In other implementations, the above loss function can also be combined in other ways to obtain the overall loss function.

[0116] The training method for the forgery detection model based on a key forgetting mechanism, as described in this specification, firstly uses image feature information focusing on different aspects of the image as the decoupling target, rather than the image itself, to simplify the decoupling process. Next, the decoupling of the forgery representation is extended to the decoupling of multi-level feature representations, and a Pearson correlation coefficient is introduced to better separate the attribute feature map from the content feature map, ensuring that the attribute feature map is as relevant to the forgery as possible. Subsequently, a key forgetting mechanism is used to force the model to pay more attention to previously overlooked common subtle forgery traces, thereby enhancing the model's generalization ability to unseen forgery types.

[0117] Below, in conjunction with Figure 8 The application stages of the forged image detection model are explained. Figure 8 This is a flowchart of a forged image detection method based on a key forgetting mechanism proposed in the embodiments of this specification. The method can be executed by any device, platform, or cluster of devices with computing and processing capabilities, including steps S801-S803 as shown below.

[0118] like Figure 8 As shown, in step S801, the target image to be detected and the forged image detection model trained based on any of the above implementation methods are obtained. The forged image detection model includes at least an attribute extractor and a key forgetting module.

[0119] The target image to be detected is a single image, which may be a potentially forged image or a video frame from a potentially forged video.

[0120] In step S802, the attribute extractor extracts attributes based on the target image to obtain a target attribute feature map, which is used to represent information related to forgery traces in the target image.

[0121] Among them, the attribute extractor is used to separate information related to forgery traces in the target image.

[0122] As one implementation method, such as Figure 4As shown, the forgery image detection model also includes an encoder. To reduce the difficulty of decoupling, the encoder can be used to convert the target image into a target feature representation, thereby decoupling based on the target feature representation rather than the target image. In this step, the encoder can extract features from the target image to obtain the target feature representation, and the attribute extractor can extract attributes from the feature representation to obtain the target attribute feature map. Compared to the dense and complex information in the target image, the feature representation contains less information, which is more conducive to the separation of the target attribute feature map related to forgery.

[0123] As one implementation method, a multi-level decoupling approach can be adopted to improve the correlation between the target attribute feature map obtained by decoupling and the forged information. When the encoder extracts features from the target image, it can obtain target feature representations at multiple levels. The feature representations at different levels are used to reflect different aspects of image feature information. For each level of target feature representation, the attribute extractor corresponding to that level extracts attributes from the target feature representation to obtain the initial target attribute feature map corresponding to that level. Based on the multiple initial target attribute feature maps corresponding to multiple levels, the target attribute feature map is obtained.

[0124] In step S803, the classifier in the key forgetting module performs a true / false prediction on the target attribute feature map to obtain a detection result, which is used to indicate whether the target image is a forged image.

[0125] In practice, unlike the training phase, the target attribute feature map in the key forgetting module does not require weakening. Instead, the classifier in the key forgetting module directly predicts whether the target attribute feature map is real or fake, obtaining the detection result. The classifier output can be a value between 0 and 1, where the value represents the probability that the target image is a fake or a real image. For example, the closer the value is to 0, the more likely the model considers the target image to be a fake image; the closer the value is to 1, the more likely the model considers the target image to be a real image. A threshold can be set, such as 0.7. When the value is greater than 0.7, the detection result is determined to be a real image; when the value is less than 0.7, the detection result is determined to be a fake image.

[0126] One implementation approach is to connect a corresponding classifier after each initial target attribute feature map. For each level's initial target attribute feature map, the corresponding classifier predicts whether the initial target attribute feature map is genuine or fake, obtaining an initial detection result. Each level's initial detection result can contain a probability that the target image belongs to a fake or real image. Alternatively, the initial detection result and the probabilities in the detection result can be weighted and summed to obtain a final probability value. Based on this probability value, it can be determined whether the target image is a fake image, further improving the accuracy of fake image detection.

[0127] The forgery image detection training method based on the key forgetting mechanism provided in the embodiments of this specification uses a forgery image detection model that includes at least an attribute extractor and a key forgetting module. Since the model learns how to identify more common and general features of forgery traces during the training phase, and the extracted significant features are still retained in the target attribute feature map, it can achieve better detection results for different types of forgery images.

[0128] Figure 9 This is a schematic diagram of the training device for a forged image detection model based on a key forgetting mechanism, as described in the embodiments of this specification. The forged image detection model includes at least an attribute extractor and a key forgetting module. This device can be applied to any device, platform, or device cluster with computing and processing capabilities. The device includes:

[0129] The sample acquisition unit 901 is configured to acquire a training sample set, which contains multiple sample images. The sample images have real and fake labels, which are used to indicate whether the sample images are fake images.

[0130] The feature extraction unit 902 is configured to perform attribute extraction based on the sample image by the attribute extractor to obtain an attribute feature map, which is used to represent information related to forgery traces in the sample image.

[0131] The weakening processing unit 903 is configured to obtain the degree of influence of each pixel in the attribute feature map on the prediction result by the key forgetting module, determine the key pixels in the attribute feature map based on the degree of influence of each pixel, and weaken the degree of influence of the key pixels.

[0132] Predictive classification unit 904 is configured to use the classifier in the key forgetting module to predict the true or false of the weakened attribute feature map to obtain the predicted label.

[0133] Iterative training unit 905 is configured to train the forgery detection model based on the difference between predicted labels and real / fake labels.

[0134] In some embodiments, the forged image detection model further includes an encoder; a feature extraction unit 902 is specifically configured to: extract features from the sample image by the encoder to obtain a feature representation; and extract attributes from the feature representation by an attribute extractor to obtain an attribute feature map.

[0135] In some embodiments, the forgery image detection model further includes a content extractor; the feature extraction unit 902 is further configured to: extract content from the feature representation by the content extractor to obtain a content feature map, the content feature map being used to represent information unrelated to forgery traces in the sample image;

[0136] The iterative training unit 905 is specifically configured to train the forged image detection model based on the differences between predicted labels and true / false labels, as well as the independence between content feature maps and attribute feature maps.

[0137] In some embodiments, the forged image detection model further includes a decoder, a feature extraction unit 902, which is further configured to: input attribute feature maps and content feature maps into the decoder for reconstruction to obtain a reconstructed image; and an iterative training unit 905, which is specifically configured to: train the forged image detection model based on the differences between the reconstructed image and the sample image, as well as the differences between the predicted label and the true / false label.

[0138] In some embodiments, the forged image detection module includes multiple layers of attribute extractors; the feature extraction unit 902 is specifically configured to: extract features from the sample image by the encoder to obtain multiple layers of feature representations, the feature representations at different layers being used to reflect different aspects of image feature information; for each layer of feature representation, the attribute extractor corresponding to the layer extracts attributes from the feature representation to obtain an initial attribute feature map corresponding to the layer; and obtain an attribute feature map based on the multiple initial attribute feature maps corresponding to the multiple layers respectively.

[0139] In some embodiments, the forged image detection model further includes multiple levels of content extractors and multiple levels of decoders, with the same number of attribute extractors, content extractors, and decoders; the feature extraction unit 902 is specifically configured to: for each level of feature representation, the content extractor corresponding to the level extracts content from the feature representation to obtain an initial content feature map corresponding to the level; for each level, the initial attribute feature map and the initial content feature map corresponding to the level are input into the decoder corresponding to the level for reconstruction to obtain reconstructed features or a reconstructed image; the iterative training unit 905 is specifically configured to: train the forged image detection model based on the differences between the reconstructed image and the sample image, the differences between the reconstructed features and feature representations at multiple levels, and the differences between the predicted labels and the true / false labels.

[0140] In some embodiments, the forged image detection model further includes multiple classifiers corresponding to multiple levels; the prediction classification unit 904 is further configured to: for each level's initial attribute feature map, the level-corresponding classifier performs a true / false prediction on the initial attribute feature map to obtain an initial predicted label; the iterative training unit 905 is specifically configured to: train the forged image detection model based on the difference between the predicted label and the true / false label, as well as the difference between the initial predicted label and the true / false label.

[0141] In some embodiments, the prediction classification unit 904 is specifically configured to: concatenate the attribute feature map and the weakened attribute feature map; perform true / false prediction on the concatenated attribute feature map to obtain the predicted label.

[0142] In some embodiments, the iterative training unit 905 is further configured to: calculate the intra-class similarity between the weakened attribute feature maps corresponding to the multiple sample images with the same true or false labels; calculate the inter-class similarity between the weakened attribute feature maps corresponding to the multiple sample images with different true or false labels; and train the forged image detection model based on the intra-class similarity, inter-class similarity, and the difference between the predicted label and the true or false label.

[0143] In some embodiments, the degree of influence of each pixel on the prediction result is the gradient corresponding to each pixel during backpropagation. The weakening processing unit 903 is specifically configured to: determine the gradient corresponding to each pixel in the attribute feature map during backpropagation based on the difference between the prediction result and the true and false labels; for each pixel in the attribute feature map, if the gradient of the pixel is higher than a preset threshold, then the pixel is determined as a key pixel; and set the value of the channel corresponding to the key pixel in the attribute feature map as a weakening value.

[0144] Figure 10 This is a schematic diagram of the forged image detection device based on a key forgetting mechanism, as described in the embodiments of this specification. The forged image detection model includes at least an attribute extractor and a key forgetting module. This device can be applied to any device, platform, or device cluster with computing and processing capabilities. The device includes:

[0145] Image acquisition unit 001 is configured to acquire a target image to be detected and a forged image detection model trained based on the method of any one of claims 1-10, wherein the forged image detection model includes at least an attribute extractor and a key forgetting module.

[0146] The attribute extraction unit 002 is configured to: extract attributes from the target image using an attribute extractor to obtain a target attribute feature map, which is used to represent information related to forgery traces in the target image;

[0147] The detection classification unit 003 is configured to: use the classifier in the key forgetting module to predict the authenticity of the target attribute feature map and obtain the detection result, which is used to indicate whether the target image is a forged image.

[0148] This specification also provides one or more embodiments of a computer-readable storage medium storing a computer program that can be used to perform the above-described actions. Figure 1 or Figure 7 The training method for the forged image detection model based on the key forgetting mechanism provided in any embodiment, or the method described above. Figure 8 Any embodiment provides a forged image detection method based on a key forgetting mechanism.

[0149] One or more embodiments of this specification also provide a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the above-described... Figure 1 or Figure 7 The training method for the forged image detection model based on the key forgetting mechanism provided in any embodiment, or the method described above. Figure 8 Any embodiment provides a forged image detection method based on a key forgetting mechanism.

[0150] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0151] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0152] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0153] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes the elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0154] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0155] This invention is described with reference to flowchart illustrations and / or block diagrams of methods according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0159] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0160] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0161] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0163] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0164] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A training method for a forged image detection model based on a key forgetting mechanism, wherein the forged image detection model includes at least an attribute extractor and a key forgetting module, the method comprising: Obtain a training sample set, which contains multiple sample images. Each sample image has a real / fake label, which is used to indicate whether the sample image is a fake image. The attribute extractor extracts attributes based on the sample image to obtain an attribute feature map. The attribute feature map is used to represent information related to forgery traces in the sample image. The feature information in the sample image is divided into information related to forgery traces and information unrelated to forgery traces. The attribute extractor is used to separate the information related to forgery traces in the sample image. Each pixel in the attribute feature map represents a forgery trace captured from the sample image. The key forgetting module obtains the degree of influence of each pixel in the attribute feature map on the prediction result, determines the key pixels in the attribute feature map based on the degree of influence of each pixel, and weakens the degree of influence of the key pixels. The degree of influence of each pixel on the prediction result is the gradient corresponding to each pixel in the backpropagation process. The weakening process includes setting the value of the channel corresponding to the key pixel in the attribute feature map to a weakening value. The classifier in the key forgetting module performs true / false prediction on the weakened attribute feature map to obtain the predicted label. The forged image detection model is trained based on the difference between the predicted label and the genuine / fake label.

2. The method according to claim 1, wherein, The forged image detection model further includes an encoder; the step of extracting attributes from the sample image using the attribute extractor to obtain an attribute feature map includes: The encoder extracts features from the sample image to obtain a feature representation; The attribute extractor extracts attributes from the feature representation to obtain the attribute feature map.

3. The method according to claim 2, wherein, The forged image detection model further includes a content extractor; the method further includes: The content extractor extracts content from the feature representation to obtain a content feature map, which is used to represent information unrelated to forgery traces in the sample image. The step of training the forged image detection model based on the difference between the predicted label and the genuine / fake label includes: The forged image detection model is trained based on the differences between the predicted labels and the true / false labels, as well as the independence between the content feature map and the attribute feature map.

4. The method according to claim 3, wherein, The forged image detection model further includes a decoder, and the method further includes: The attribute feature map and the content feature map are input into the decoder for reconstruction to obtain the reconstructed image; Based on the difference between the predicted label and the genuine / fake label, the forged image detection model is trained, including: The forgery detection model is trained based on the differences between the reconstructed image and the sample image, as well as the differences between the predicted label and the true / false label.

5. The method according to claim 2, wherein, The forged image detection module includes multiple levels of attribute extractors; The step of extracting features from the sample image by the encoder to obtain a feature representation includes: The encoder extracts features from the sample image to obtain feature representations at multiple levels. The feature representations at different levels are used to reflect different aspects of image feature information. The step of extracting attributes from the feature representation using the attribute extractor to obtain the attribute feature map includes: For each level of feature representation, the attribute extractor corresponding to that level extracts attributes from the feature representation to obtain the initial attribute feature map corresponding to that level. The attribute feature map is obtained based on the multiple initial attribute feature maps corresponding to multiple levels.

6. The method according to claim 5, wherein, The forged image detection model further includes multiple layers of content extractors and multiple layers of decoders, with the same number of attribute extractors, content extractors, and decoders; the method further includes: For each level of feature representation, the content extractor corresponding to that level extracts the content of the feature representation to obtain the initial content feature map corresponding to that level. For each level, the initial attribute feature map and the initial content feature map corresponding to the level are input into the decoder corresponding to the level for reconstruction to obtain reconstructed features or reconstructed images. Based on the difference between the predicted label and the genuine / fake label, the forged image detection model is trained, including: The forgery detection model is trained based on the differences between the reconstructed image and the sample image, the differences between the reconstructed features and the feature representations at multiple levels, and the differences between the predicted labels and the true / false labels.

7. The method according to claim 5, wherein, The forged image detection model further includes multiple levels of corresponding classifiers; the method further includes: For each level of the initial attribute feature map, the classifier corresponding to that level performs a true / false prediction on the initial attribute feature map to obtain the initial predicted label; Based on the difference between the predicted label and the genuine / fake label, the forged image detection model is trained, including: The forged image detection model is trained based on the difference between the predicted label and the real / fake label, as well as the difference between the initial predicted label and the real / fake label.

8. The method according to claim 1, wherein, The step of performing true / false prediction on the weakened attribute feature map to obtain the predicted label includes: The attribute feature map and the weakened attribute feature map are spliced ​​together; The spliced ​​attribute feature map is used to predict whether it is true or false, and the predicted label is obtained.

9. The method according to claim 1, wherein, The method further includes: For multiple sample images with the same true or false labels, calculate the intra-class similarity between the weakened attribute feature maps corresponding to the multiple sample images; For multiple sample images with different true and false labels, calculate the inter-class similarity between the weakened attribute feature maps corresponding to the multiple sample images; Based on the difference between the predicted label and the genuine / fake label, the forged image detection model is trained, including: The forged image detection model is trained based on the intra-class similarity, the inter-class similarity, and the difference between the predicted label and the real / fake label.

10. The method according to claim 1, wherein, The process of obtaining the degree of influence of each pixel in the attribute feature map on the prediction result, determining key pixels in the attribute feature map based on the degree of influence of each pixel, and weakening the influence of the key pixels includes: Based on the difference between the prediction results and the true and false labels, the gradient corresponding to each pixel in the attribute feature map during the backpropagation process is determined. For each pixel in the attribute feature map, if the gradient of the pixel is higher than a preset threshold, then the pixel is determined as a key pixel. Set the value of the channel corresponding to the key pixel in the attribute feature map to a weakened value.

11. A method for detecting forged images based on a key forgetting mechanism, the method comprising: The method comprises acquiring a target image to be detected and a forged image detection model trained based on the method described in any one of claims 1-10, wherein the forged image detection model includes at least an attribute extractor and a key forgetting module. The attribute extractor performs attribute extraction based on the target image to obtain a target attribute feature map, which is used to represent information related to forgery traces in the target image. The classifier in the key forgetting module performs a true / false prediction on the target attribute feature map to obtain a detection result, which is used to indicate whether the target image is a forged image.

12. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-10 or the method of claim 11.

Images