A method, apparatus and equipment for image forgery detection

Abstract

The one or more embodiments of the specification disclose a method, device and equipment for image forgery detection. The method comprises: performing feature extraction on each pixel point in a to-be-detected image to obtain image features corresponding to each pixel point in the to-be-detected image; inputting the image features corresponding to each pixel point in the to-be-detected image into a forgery detection model to perform forgery feature similarity matching between the image features corresponding to any two pixel points in the to-be-detected image, and obtain a forgery feature similarity value between any two pixel points in the to-be-detected image; determining, according to the forgery feature similarity value, a forgery probability map of the to-be-detected image corresponding to each pixel point in the to-be-detected image; and in a case where a proportion of pixel points with a forgery probability value exceeding a preset probability in the forgery probability map exceeds a first preset proportion, determining a forgery region in the to-be-detected image according to the proportion of pixel points with the forgery probability value exceeding the preset probability in the forgery probability map.

Description

Technical Field

[0001] This specification relates to the field of data processing technology, and in particular to a method, apparatus and equipment for image forgery detection. Background Technology

[0002] In recent years, Generative Adversarial Networks (GANs) have developed rapidly in the field of computer vision. Deepfakes, based on this technology, have sparked a global craze for forgery. At the same time, the malicious application of forgery techniques poses significant risks to social security and the protection of personal privacy data (including sensitive and private information). Detecting deepfake data is of paramount importance for promoting the healthy development of cyberspace and social stability.

[0003] Currently, methods for deepfake detection of text images, face images, or videos using Convolutional Neural Networks (CNNs) deep learning models have attracted extensive research. However, these CNN-based deepfake detection models can only predict the probability of whether an image or video has been forged, but cannot pinpoint the specific forged region. Therefore, there is an urgent need to provide a better training scheme for image forgery detection. Summary of the Invention

[0004] This specification provides a method, apparatus, and device for image forgery detection, in order to provide an image forgery detection solution that meets the expectations of image review personnel.

[0005] In a first aspect, one or more embodiments of this specification provide a method for image forgery detection, comprising: extracting features from each pixel in an image to be detected to obtain image features corresponding to each pixel in the image to be detected; inputting the image features corresponding to each pixel in the image to be detected into a forgery detection model, performing forgery feature similarity matching between image features corresponding to any two pixels in the image to be detected to obtain a forgery feature similarity value between any two pixels in the image to be detected; determining a forgery probability map of the image to be detected corresponding to each pixel in the image to be detected based on the forgery feature similarity value; and determining a forgery region in the image to be detected based on the proportion of pixels in the forgery probability map whose forgery probability value exceeds a preset probability when the proportion of pixels in the forgery probability map whose forgery probability value exceeds the preset probability.

[0006] Secondly, embodiments of this application provide an image forgery detection apparatus, comprising: extracting features from each pixel in an image to be detected to obtain image features corresponding to each pixel in the image to be detected; inputting the image features corresponding to each pixel in the image to be detected into a forgery detection model, performing forgery feature similarity matching between image features corresponding to any two pixels in the image to be detected to obtain a forgery feature similarity value between any two pixels in the image to be detected; determining a forgery probability map of the image to be detected corresponding to each pixel in the image to be detected based on the forgery feature similarity value; and determining a forgery region in the image to be detected based on the proportion of pixels in the forgery probability map whose forgery probability value exceeds a preset probability when the proportion of pixels in the forgery probability map whose forgery probability value exceeds the preset probability.

[0007] Thirdly, embodiments of this application provide an electronic device, comprising: a processor and a memory arranged to store computer-executable instructions, wherein when the executable instructions are executed, the processor is capable of: extracting features from each pixel in an image to be detected to obtain image features corresponding to each pixel in the image to be detected; inputting the image features corresponding to each pixel in the image to be detected into a forgery detection model, performing forgery feature similarity matching between image features corresponding to any two pixels in the image to be detected to obtain a forgery feature similarity value between any two pixels in the image to be detected; determining a forgery probability map of the image to be detected corresponding to each pixel in the image to be detected based on the forgery feature similarity value; and determining a forgery region in the image to be detected based on the proportion of pixels in the forgery probability map whose forgery probability value exceeds a preset probability when the proportion of pixels in the forgery probability map whose forgery probability value exceeds the preset probability.

[0008] Fourthly, embodiments of this specification provide a storage medium for storing a computer program, which can be executed by a processor to implement the following process: extracting features from each pixel in an image to be detected to obtain image features corresponding to each pixel in the image to be detected; inputting the image features corresponding to each pixel in the image to be detected into a forgery detection model, performing forgery feature similarity matching between image features corresponding to any two pixels in the image to be detected to obtain a forgery feature similarity value between any two pixels in the image to be detected; determining a forgery probability map of the image to be detected corresponding to each pixel in the image to be detected based on the forgery feature similarity value; and determining a forgery region in the image to be detected based on the proportion of pixels in the forgery probability map whose forgery probability value exceeds a preset probability when the proportion of pixels in the forgery probability map whose forgery probability value exceeds the preset probability. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of an image forgery detection method according to an embodiment of this specification.

[0011] Figure 2 This is a schematic diagram illustrating an application scenario of an image forgery detection method according to an embodiment of this specification.

[0012] Figure 3 This is a schematic flowchart illustrating the training process of a forgery detection model according to an embodiment of this specification.

[0013] Figure 4 This is a schematic diagram of an image forgery detection device according to an embodiment of this specification.

[0014] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this specification. Detailed Implementation

[0015] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments in this specification. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0016] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this specification can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0017] The image forgery detection method, apparatus, and device provided in this specification will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0018] Deepfake is a combination of computer "deep learning" and "fake," a technique that uses artificial intelligence and deep learning algorithms to manipulate and forge data such as audio and video media. Deepfake technology can synthesize images (including text images and facial images), audio, or video that have never appeared in the real world, and the synthesized images, audio, or video are not easily distinguishable by humans.

[0019] In recent years, deepfake-related attacks have frequently come to the public's attention. These attacks include: forgery of documents containing key information such as contracts, certificates, letters, and notices; forgery of facial images / videos such as face replacement, facial attribute editing, and facial expression manipulation; and forgery of audio such as synthesizing the voice of a person who does not exist in the real world or imitating or cloning the voice of a target person. To date, the social harm caused by deepfakes has been enormous, prompting countries around the world to pay close attention to its governance.

[0020] Advances in deepfake technology have prompted researchers to explore more effective deepfake detection algorithms. However, traditional deepfake image recognition systems typically use convolutional neural networks to perform binary classification predictions on the entire image. A single image is input into the model, which predicts the probability of whether the image has been forged. This entire process lacks interpretability and cannot guide image reviewers to pinpoint the specific forged area, making the verification of whether an image has been forged quite difficult.

[0021] Figure 1 This illustration shows an embodiment of an image forgery detection method provided by the present invention. The method can be executed by an electronic device, which may include a server and / or a terminal device, wherein the terminal device may be, for example, an in-vehicle terminal or a mobile phone terminal. In other words, the method can be executed by software or hardware installed in the aforementioned electronic device, and the method includes the following steps:

[0022] S102: Extract features from each pixel in the image to be detected to obtain the image features corresponding to each pixel in the image to be detected.

[0023] The image to be detected is an image that may contain forged areas. Specifically, the image to be detected can be an image containing text, or an image containing real objects such as people or animals. This specification does not impose specific limitations on the content of the image to be detected, which can be determined according to the actual situation.

[0024] To better illustrate the present invention and highlight its main points, the specific embodiments described in this specification use raw training data of image type as the object of description. Those skilled in the art should understand that this specification can also be implemented in a similar manner for raw training data of other data types. For example, for raw training data of audio type, the spectrum of the audio can be converted into a graphic first, and then the image forgery detection method described in this specification can be applied to the graphic corresponding to the audio to determine whether the audio has been forged, and thus determine the forged region in the audio.

[0025] In step S102, a neural network can be used to extract image features corresponding to multiple pixels in the image to be detected. Specifically, a convolutional neural network can be used. This specification does not specify a dimension for the extracted image features; it can be selected according to the actual situation.

[0026] S104: Input the image features corresponding to each pixel in the image to be detected into the forgery detection model, perform forgery feature similarity matching between the image features corresponding to any two pixels in the image to be detected, and obtain the forgery feature similarity value between any two pixels in the image to be detected.

[0027] Forgery feature similarity is a measure of whether the forgery probability of two pixels is similar. Specifically, it can be determined by the distance or angle between image features (forgery features) related to the forgery probability of two pixels. When the similarity value is greater than a threshold, it can be determined that the forgery probability of the two compared pixels is similar, that is, they may both be forged or not forged. Methods for measuring forgery feature similarity include Euclidean distance, cosine similarity, and Pearson correlation coefficient. This specification does not restrict which forgery feature similarity measurement method the specific forgery detection model should use.

[0028] Specifically, a forgery detection model can be used to calculate the forgery similarity value between any two pixels in the image to be detected. The specific implementation and training method of the forgery detection model are not limited in this embodiment and can be flexibly selected according to the actual situation.

[0029] S106: Based on the forgery feature similarity value, determine the forgery probability map of the image to be detected corresponding to each pixel in the image to be detected.

[0030] The forgery probability map is used to indicate the probability that each pixel in the image to be detected is forged. Specifically, for each pixel in the image to be detected, the forgery feature similarity value between that pixel and any other pixel in the image can be obtained. This forgery feature similarity value reflects the probability that the pixel is forged. In one example, the forgery probability value corresponding to each pixel can be determined based on the forgery feature similarity. The method for determining the forgery probability value corresponding to each pixel based on the forgery feature similarity can be found in subsequent embodiments and will not be elaborated here. After obtaining the forgery probability value corresponding to each pixel, the forgery probability values ​​can be arranged according to the positions of the pixels related to the forgery probability values ​​to obtain the forgery probability map corresponding to that pixel. Specifically, each pixel in the image to be detected corresponds to one forgery probability map.

[0031] S108: If the proportion of pixels with a forgery probability value exceeding a preset probability in the forgery probability map exceeds a first preset proportion, the forgery region in the image to be detected is determined based on the proportion of pixels with a forgery probability value exceeding the preset probability in the forgery probability map.

[0032] The preset probability is used to determine whether a pixel has been forged. After obtaining the forgery probability map, the forgery probability of the image to be detected can be determined by the number of pixels whose forgery probability value exceeds the preset probability. In one example, the proportion of pixels whose forgery probability value exceeds the preset probability in all pixels of the image to be detected can be determined. When this proportion exceeds a first preset proportion, it can be determined that the image to be detected is forged; when this proportion does not exceed the first preset proportion, it indicates that the probability of the image to be detected being forged is low. After determining that the image to be detected is forged, the forgery probability value of the image to be detected can be determined based on multiple or all forgery probability maps. For example, the average of the forgery probability values ​​corresponding to multiple / all pixel forgery probability maps can be used to determine the forgery probability value of the image to be detected.

[0033] A forged region is a region in the image to be detected that has been forged. Since the forged regions indicated in the forgery probability map corresponding to each pixel indicate potentially forged parts in the image to be detected, differing only at their edges, in one example, the forged regions in the image to be detected can be determined using one or more forgery probability maps, depending on the required computational precision. Specifically, when rapid determination of forged regions in the image is required and the accuracy requirement for forged regions is not high, the forged regions in the image to be detected can be determined based on the forgery probability map corresponding to only one pixel; when the accuracy requirement for forged regions is high and the time requirement for acquiring forged regions in the image to be detected is not high, the forged regions in the image to be detected can be determined based on the forgery probability maps corresponding to multiple or all pixels.

[0034] In the embodiments of this specification, a forgery detection model is used to determine the forgery feature similarity between any two pixels in an image, and a forgery probability map corresponding to each pixel is determined. Then, the forgery region of the image to be detected can be determined based on the positional probability map. This process, based on the correlation between forgery probability values ​​and forgery feature similarity, transforms the process of determining the forgery region in the image to be detected into the process of determining the forgery feature similarity between any two pixels. This effectively reduces the difficulty of determining the forgery region and improves the accuracy of forgery region determination, thereby enhancing the robustness of the image forgery detection process.

[0035] Figure 2 A schematic diagram illustrating an application scenario for an image forgery detection method is provided, such as... Figure 2 As shown, the user sends a service request to the server via a mobile phone or other terminal to perform image forgery detection on the image to be detected. After receiving the service request, the server uses the image forgery detection method described in this manual to obtain the forged area in the image to be detected, and sends the forged area in the image to be detected to the terminal for the user's reference.

[0036] In one implementation, the forgery detection model is trained using training images and corresponding forgery clue maps, where forgery regions in the forgery clue maps are labeled pixel by pixel.

[0037] The forgery clue map is a map used to represent the location of forged regions in the training image. The forgery clue map corresponding to the training image can be determined based on the identified forged regions. This specification does not specify a particular form for the forgery clue map; it can be selected according to the actual situation. In one implementation, the forgery clue map is a mask image representing the forged regions in the training image. Specifically, the mask image can use two values, 0 and 1, to distinguish whether each pixel in the training image belongs to a forged or non-forged region, thus providing a visual annotation of the forged regions in the image to be detected, thereby improving the speed of forged region determination. In one example, contour lines can also be used to distinguish between forged and non-forged regions in the image, where each point in the contour line can be represented by a sequence of coordinate points.

[0038] In one example, training can be performed using a training image and a forgery cue map. Specifically, since the forgery cue map explicitly indicates the forgery regions in the training image, the forgery feature similarity values ​​between pixels in the forgery cue map can be used to indicate whether the forgery detection model accurately predicts the forgery feature similarity values ​​between corresponding pixels in the training image. This allows adjustment of the forgery detection model's parameters to improve the preset accuracy of the forgery feature similarity values. Specifically, the forgery detection model is trained using the training image and the corresponding forgery cue map, which can be performed using the following steps A1-A3:

[0039] Step A1: Extract features from each pixel in the training image to obtain the image features corresponding to each pixel in the training image;

[0040] Step A2: Using the forgery detection model, perform forgery feature similarity matching between image features corresponding to any two pixels in the training image to obtain the forgery feature similarity value between any two pixels in the training image.

[0041] Step A3: Train the forgery detection model based on the forgery feature similarity value between any two pixels in the training image and the target feature similarity value between any two pixels in the forgery clue image.

[0042] The target feature similarity value is the similarity between the forged features of two pixels indicated by the forged clue map. Specifically, the method for measuring target feature similarity can be the same as that for measuring forged feature similarity, such as Euclidean distance, cosine similarity, or Pearson correlation coefficient. Since any two pixels in the forged clue map have three possibilities: both are forged pixels, both are non-forged pixels, or one is forged and the other is non-forged, and the target feature similarity values ​​corresponding to both being forged and both being non-forged pixels are the same, when using cosine similarity to measure target feature similarity, the target feature similarity value between any two pixels in the forged clue map will either be 1 (corresponding to either both being forged or both being non-forged pixels) or 0 (corresponding to either one being forged and the other non-forged pixel).

[0043] In one implementation, step A3 can be executed as follows: steps B1-B4:

[0044] Step B1: Determine the first target pixel in the forgery clue map corresponding to the first training pixel corresponding to the forgery feature similarity value between any two pixels in the training image, and the second target pixel in the forgery clue map corresponding to the second training pixel.

[0045] Step B2: If the contents represented by the first target pixel and the second target pixel are the same, the first similarity value is used as the target feature similarity value corresponding to the fake feature similarity value between any two pixels in the training image.

[0046] Step B3: If the contents represented by the first target pixel and the second target pixel are inconsistent, the second similarity value is used as the target feature similarity value corresponding to the fake feature similarity value between any two pixels in the training image.

[0047] Step B4: Train the forgery detection model based on the forgery feature similarity value between any two pixels in the training image and the target feature similarity value corresponding to the forgery feature similarity value between any two pixels in the training image.

[0048] The first similarity value is the forgery feature similarity value between two pixels when both pixels in the forgery clue image are forged or neither is forged, and the second similarity value is the forgery feature similarity value between two pixels when one pixel in the forgery clue image is forged or the other is not forged.

[0049] In one example, the target feature similarity value between any two pixels in the forgery clue image can be determined directly based on whether the content represented by the corresponding pixels is consistent. For instance, when using 0 and 1 to represent the forgery clue image, if both pixels represent 0 or both represent 1, the target feature similarity value between the two pixels can be determined as 1 (first similarity value); if one pixel represents 0 or the other represents 1, the target feature similarity value between the two pixels can be determined as 0 (second similarity value). Furthermore, the forgery detection model can be trained based on the forgery feature similarity value between any two pixels in the training image and the target feature similarity value (i.e., the first similarity value or the second similarity value) corresponding to the forgery feature similarity value between any two pixels in the training image.

[0050] In the embodiments of this specification, when determining the forgery feature similarity value between any two pixels in the image to be detected, the corresponding pixels in the forgery clue map of the two pixels are first determined. Based on whether the corresponding pixels are the same, the target feature similarity value corresponding to the forgery feature similarity value between the two pixels is directly determined, which improves the speed of model training.

[0051] Specifically, when the similarity value of the forged features output by the forgery detection model is inconsistent with the similarity value of the target features, the parameters of the forgery detection model need to be adjusted so that the similarity value of the forged features output by the forgery detection model is as close as possible to the similarity value of the target features.

[0052] In the embodiments of this specification, the forgery detection model is trained based on the training image and the corresponding forgery cue map. Through the above process, the forgery cue map can be used to conveniently improve the preset accuracy of the forgery detection model for the similarity value of forgery features, thereby effectively reducing the difficulty of image forgery detection and improving the practicality and generalization ability of image forgery detection.

[0053] In one implementation, step S106 can be executed as follows: steps C1-C2:

[0054] Step C1: The forgery feature similarity value between each pixel in the image to be detected and all pixels in the image to be detected is used as the forgery probability value in the forgery probability map corresponding to each pixel in the image to be detected.

[0055] Step C2: Adjust the forgery probability value corresponding to each pixel in the forgery probability map so that the forgery probability values ​​corresponding to at least a second preset proportion of pixels in different forgery probability maps are similar.

[0056] Specifically, the forgery feature similarity value can be directly used as the forgery probability value in the forgery probability map. In this case, for each pixel in the image to be detected, the forgery probability maps may indicate opposite forgery regions. For example, for a forged pixel and an unforged pixel, in the forgery probability map corresponding to the forged pixel, the forgery feature similarity value is the forgery feature similarity value between the forged pixel and the unforged pixel, which is close to 1; while in the forgery probability map corresponding to the unforged pixel, the forgery feature similarity value is the forgery feature similarity value between the forged pixel and the unforged pixel, which is close to 0. In this case, the forgery regions indicated by the forgery probability maps corresponding to the forged and unforged pixels are exactly opposite, and this needs to be adjusted so that the forgery regions indicated by the forgery probability maps corresponding to each pixel in the image to be detected are close.

[0057] In one example, a second preset ratio can be set to constrain the proportion of the forged region indicated by the forgery probability map corresponding to each pixel in the image to be detected in the image to be detected, so that the forgery probability of most pixels in the forgery probability map corresponding to each pixel in the adjusted forgery probability map is similar.

[0058] In the embodiments of this specification, the forgery feature similarity value is used as the forgery probability value, and the forgery probability map corresponding to each pixel in the image to be detected is adjusted to ensure that the forgery regions indicated by all forgery probability maps are consistent. This process achieves accurate determination of the forgery probability map through forgery feature similarity, thereby improving the accuracy of forgery regions in the image to be detected.

[0059] In one implementation, adjusting the forgery probability value corresponding to each pixel in the forgery probability map can be performed as follows: D1-D2:

[0060] Step D1: Determine the target forgery probability map in the forgery probability map corresponding to each pixel in the image to be detected;

[0061] Step D2: When the forgery probability values ​​of pixels in the forgery probability map that are lower than the second preset ratio in the target forgery probability map are similar, update the forgery probability of each pixel in the forgery probability map according to the forgery probability of each pixel in the forgery probability map and the preset correction value.

[0062] The target forgery probability map is used to adjust the forgery region in the forgery probability map corresponding to each pixel in the image to be detected. After determining the forgery probability map, it can be adjusted based on the target forgery probability map to identify forgery regions that are inconsistent with those indicated by the target forgery probability map.

[0063] Specifically, a preset correction value can be set to adjust the forgery probability in the forgery probability map to be adjusted. For example, in the target forgery probability map, the forgery probability of the forgery area is close to 1. For a forgery probability map where the forgery probability of the forgery area is close to 0, the preset correction value can be set to 1. By subtracting the forgery probability corresponding to each pixel in the forgery probability map from the preset correction value, the updated forgery probability in the forgery probability map can be obtained, and the updated forgery probability is close to 1.

[0064] In the embodiments of this specification, the forgery probability map is adjusted based on the target forgery probability map, thereby achieving consistency of the forgery region indicated by the forgery probability map corresponding to each pixel in the image to be detected, and thus improving the accuracy of the forgery region in the image to be detected determined according to the forgery probability map.

[0065] In one implementation, step S108 can be executed as follows: steps E1-E4:

[0066] Step E1: Determine the forgery ratio corresponding to the forgery probability map based on the proportion of pixels in the image to be detected whose forgery probability value exceeds the preset probability in the forgery probability map.

[0067] Step E2: Take the average value of the forgery ratio of the forgery probability map corresponding to each pixel as the forgery ratio of the image to be detected.

[0068] Step E3: Add the forgery probability values ​​of the same pixel in the forgery probability map corresponding to each pixel in the image to be detected, and obtain the sum of the forgery probability values ​​corresponding to each pixel in the image to be detected.

[0069] Step E4: Starting from the pixel with the highest sum of forgery probability values ​​in the image to be detected, determine the forged pixels in the image to be detected that correspond to the forgery ratio according to the sorting of the sum of forgery probability values, and determine the area where the forged pixels in the image to be detected are located as the forgery area in the image to be detected.

[0070] The forgery ratio refers to the proportion of forged regions in the forgery probability map or the image to be detected. In one example, the forgery ratio can be determined by the proportion of forged pixels among all pixels. Specifically, the forgery ratio of the forgery probability map can be determined based on the proportion of pixels whose forgery probability value exceeds a preset probability. Since the image to be detected corresponds to multiple forgery probability maps, the forgery probability of the forgery probability map corresponding to each pixel in the image to be detected can be averaged to determine the forgery ratio of the image to be detected.

[0071] After determining the forgery probability of the image to be detected, all forgery probability maps corresponding to the image can be superimposed. The forgery probability value of each pixel in the image is the sum of the forgery probabilities of the corresponding pixels in all forgery probability maps. Furthermore, starting with the pixel with the highest forgery probability in the image, the pixels to be forged can be determined according to the sorting result of the forgery probabilities of all pixels in the image. The proportion of the forged pixels among all pixels in the image is the forgery ratio of the image. Further, the region where the forged pixels are located can be defined as the forgery region in the image.

[0072] Since forgery clue diagrams can intuitively represent forged areas within the area to be detected, in one example, a forgery clue diagram corresponding to the image to be detected can be output to reduce the difficulty for image reviewers in verifying whether an image has been forged.

[0073] In the embodiments of this specification, the forgery ratio of the image to be detected is determined by using the forgery probability map corresponding to each pixel, and then the forgery region in the image to be detected is determined based on the forgery ratio. This process achieves accurate determination of the forgery region through the forgery probability map corresponding to each pixel, thereby improving the accuracy of image forgery detection.

[0074] Figure 3 A schematic flowchart illustrating the training process of a forgery detection model for face image forgery detection is presented, such as... Figure 3 As shown, the training process of the forgery detection model is executed as follows: F1-F3:

[0075] Step F1: Training data generation. Specifically, obtain the fake face image and create a fake cue map based on the fake face image.

[0076] Step F2: Image Feature Representation. Specifically, high-dimensional features are extracted from both the fake and non-fake regions in the fake face image.

[0077] Step F3: Comparison Learning. Specifically, a forgery detection model (i.e., the CNN in the figure) is used to learn the feature consistency between different regions (i.e., forged regions and non-forged regions), so that the forgery feature similarity value of any two pixels output by the forgery detection model is consistent with the target feature similarity value corresponding to the forgery clue map.

[0078] It should be noted that the image forgery detection method provided in the embodiments of this specification can be executed by an image forgery detection device or a control module within that device for executing the image forgery detection method. This specification describes the image forgery detection device provided in the embodiments of this specification by using an image forgery detection device to execute the image forgery detection method as an example.

[0079] Figure 4 This is a schematic diagram of an image forgery detection device according to an embodiment of the present invention. Figure 4 As shown, the image forgery detection device 400 includes:

[0080] The feature extraction module 410 is used to extract features from each pixel in the image to be detected, so as to obtain the image features corresponding to each pixel in the image to be detected.

[0081] The feature matching module 420 is used to input the image features corresponding to each pixel in the image to be detected into the forgery detection model, perform forgery feature similarity matching between the image features corresponding to any two pixels in the image to be detected, and obtain the forgery feature similarity value between any two pixels in the image to be detected.

[0082] The probability map determination module 430 is used to determine the forgery probability map of the image to be detected for each pixel in the image to be detected based on the forgery feature similarity value.

[0083] The forgery region determination module 440 is used to determine the forgery region in the image to be detected based on the proportion of pixels in the forgery probability map whose forgery probability value exceeds the preset probability when the proportion of pixels whose forgery probability value exceeds the preset probability exceeds the first preset proportion.

[0084] In one embodiment, the forgery detection model is trained using training images and corresponding forgery clue maps, where forgery regions in the forgery clue maps are annotated pixel by pixel.

[0085] In one embodiment, the forgery detection model is trained using training images and corresponding forgery clue maps, including:

[0086] Feature extraction is performed on each pixel in the training image to obtain the image features corresponding to each pixel in the training image;

[0087] By using a forgery detection model, forgery feature similarity matching is performed between image features corresponding to any two pixels in the training image to obtain the forgery feature similarity value between any two pixels in the training image.

[0088] The forgery detection model is trained based on the forgery feature similarity value between any two pixels in the training image and the target feature similarity value between any two pixels in the forgery clue image.

[0089] In one embodiment, a forgery detection model is trained based on the forgery feature similarity value between any two pixels in the training image and the target feature similarity value between any two pixels in the forgery clue map, including:

[0090] Determine the first target pixel in the forgery clue map corresponding to the first training pixel corresponding to the forgery feature similarity value between any two pixels in the training image, and the second target pixel in the forgery clue map corresponding to the second training pixel;

[0091] If the content represented by the first target pixel and the second target pixel is the same, the first similarity value is used as the target feature similarity value corresponding to the fake feature similarity value between any two pixels in the training image;

[0092] If the contents represented by the first target pixel and the second target pixel are inconsistent, the second similarity value is used as the target feature similarity value corresponding to the fake feature similarity value between any two pixels in the training image;

[0093] The forgery detection model is trained based on the forgery feature similarity value between any two pixels in the training image and the target feature similarity value corresponding to the forgery feature similarity value between any two pixels in the training image.

[0094] In one embodiment, the forgery clue image is a mask image representing forgery regions in the training image.

[0095] In one embodiment, the probability map determination module 430 includes:

[0096] The probability value determination unit is used to take the forgery feature similarity value between each pixel in the image to be detected and all pixels in the image to be detected as the forgery probability value in the forgery probability map corresponding to each pixel in the image to be detected.

[0097] The adjustment unit is used to adjust the forgery probability value corresponding to each pixel in the forgery probability map, so that the forgery probability values ​​corresponding to at least a second preset proportion of pixels in different forgery probability maps are similar.

[0098] In one embodiment, the adjustment unit is used for:

[0099] In the forgery probability map corresponding to each pixel in the image to be detected, determine the target forgery probability map;

[0100] When the forgery probability values ​​of pixels in the forgery probability map that are lower than the second preset ratio in the target forgery probability map are similar, the forgery probability of each pixel in the forgery probability map is updated according to the forgery probability of each pixel in the forgery probability map and the preset correction value.

[0101] In one embodiment, the forgery region determination module 440 includes:

[0102] The first determining unit is used to determine the forgery ratio corresponding to the forgery probability map based on the proportion of pixels in the image to be detected whose forgery probability value exceeds a preset probability in the forgery probability map.

[0103] The second determining unit takes the average value of the forgery ratio of the forgery probability map corresponding to each pixel as the forgery ratio of the image to be detected.

[0104] The probability value determination unit is used to add the forgery probability values ​​corresponding to the same pixel in the forgery probability map corresponding to each pixel in the image to be detected, so as to obtain the forgery probability value corresponding to each pixel in the image to be detected.

[0105] The forgery region determination unit is used to start from the pixel with the highest forgery probability value in the image to be detected, and determine the forged pixels in the image to be detected that correspond to the forgery ratio according to the sorting of forgery probability values, and determine the region where the forged pixels in the image to be detected are located as the forgery region in the image to be detected.

[0106] The image forgery detection device in the embodiments of this specification can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. The embodiments in this specification do not impose specific limitations.

[0107] The image forgery detection device in the embodiments of this specification can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this specification does not specifically limit the embodiments to this type of device.

[0108] The image forgery detection apparatus provided in the embodiments of this specification can achieve... Figure 1 The various processes implemented in the method embodiments are not described in detail here to avoid repetition.

[0109] Based on the same idea, one or more embodiments of this specification also provide an electronic device, such as... Figure 5As shown. Electronic devices can vary considerably due to differences in configuration or performance, and may include one or more processors 501 and memory 502. Memory 502 may store one or more application programs or data. Memory 502 may be temporary or persistent storage. The application programs stored in memory 502 may include one or more modules (not shown), each module may include a series of computer-executable instructions for the electronic device. Furthermore, processor 501 may be configured to communicate with memory 502 and execute the series of computer-executable instructions in memory 502 on the electronic device. The electronic device may also include one or more power supplies 503, one or more wired or wireless network interfaces 504, one or more input / output interfaces 505, and one or more keyboards 506.

[0110] Specifically, in this embodiment, the electronic device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the electronic device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0111] Feature extraction is performed on each pixel in the image to be detected to obtain the image features corresponding to each pixel in the image to be detected;

[0112] The image features corresponding to each pixel in the image to be detected are input into the forgery detection model. The forgery feature similarity between the image features corresponding to any two pixels in the image to be detected is matched to obtain the forgery feature similarity value between any two pixels in the image to be detected.

[0113] Based on the forgery feature similarity value, determine the forgery probability map of the image to be detected corresponding to each pixel in the image to be detected;

[0114] If the proportion of pixels with a forgery probability value exceeding a preset probability in the forgery probability map exceeds a first preset proportion, the forged region in the image to be detected is determined based on the proportion of pixels with a forgery probability value exceeding the preset probability in the forgery probability map.

[0115] One or more embodiments of this specification also provide a storage medium storing one or more computer programs, the one or more computer programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform various processes of the above-described image forgery detection method embodiments, and specifically for performing:

[0116] Feature extraction is performed on each pixel in the image to be detected to obtain the image features corresponding to each pixel in the image to be detected;

[0117] The image features corresponding to each pixel in the image to be detected are input into the forgery detection model. The forgery feature similarity between the image features corresponding to any two pixels in the image to be detected is matched to obtain the forgery feature similarity value between any two pixels in the image to be detected.

[0118] Based on the forgery feature similarity value, determine the forgery probability map of the image to be detected corresponding to each pixel in the image to be detected;

[0119] If the proportion of pixels with a forgery probability value exceeding a preset probability in the forgery probability map exceeds a first preset proportion, the forged region in the image to be detected is determined based on the proportion of pixels with a forgery probability value exceeding the preset probability in the forgery probability map.

[0120] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described storage medium embodiment is basically similar to the method embodiment, so the description is relatively simple; relevant parts can be referred to the description of the method embodiment.

[0121] The methods, apparatus, modules, or units described in the above embodiments can be implemented by a computer chip or entity, or by a product with a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0122] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0123] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This specification describes one or more embodiments of methods, apparatus (systems), and computer program products according to embodiments of this specification with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0128] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0129] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0130] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. This specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0132] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0133] The above description is merely one or more embodiments of this specification and is not intended to limit this application. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for image forgery detection, comprising: extracting features of each pixel point in a to-be-detected image to obtain image features corresponding to each pixel point in the to-be-detected image; inputting the image features corresponding to each pixel point in the to-be-detected image into a forgery detection model to perform forgery feature similarity matching between image features corresponding to any two pixel points in the to-be-detected image, and obtain a forgery feature similarity value between any two pixel points in the to-be-detected image; determining a forgery probability map of the to-be-detected image corresponding to each pixel point in the to-be-detected image according to the forgery feature similarity value; in a case where a proportion of pixel points with a forgery probability value exceeding a preset probability in the forgery probability map exceeds a first preset proportion, determining a forgery proportion corresponding to the forgery probability map according to the proportion of pixel points with a forgery probability value exceeding a preset probability in the to-be-detected image in the forgery probability map; taking an average value of the forgery proportion of the forgery probability map corresponding to each pixel point as a forgery proportion corresponding to the to-be-detected image; adding forgery probability values corresponding to the same pixel point in the forgery probability map corresponding to each pixel point in the to-be-detected image to obtain a forgery probability value corresponding to each pixel point in the to-be-detected image; starting from a pixel point with the largest forgery probability value in the to-be-detected image, determining a forged pixel point corresponding to the forgery proportion of the to-be-detected image according to the order of the forgery probability value in the to-be-detected image, and determining a forged region in the to-be-detected image as a region in which the forged pixel point in the to-be-detected image is located.

2. The method of claim 1, wherein the forgery detection model is trained by a training image and a forgery clue map corresponding to the training image, and the forged region in the forgery clue map is processed by pixel-by-pixel labeling.

3. The method of claim 2, wherein the forgery detection model is trained by a training image and a forgery clue map corresponding to the training image, comprising: extracting features of each pixel point in the training image to obtain image features corresponding to each pixel point in the training image; performing forgery feature similarity matching between image features corresponding to any two pixel points in the training image by the forgery detection model to obtain a forgery feature similarity value between any two pixel points in the training image; training the forgery detection model based on the forgery feature similarity value between any two pixel points in the training image and a target feature similarity value between any two pixel points in the forgery clue map.

4. The method of claim 3, wherein the training of the forgery detection model based on the forgery feature similarity value between any two pixel points in the training image and the target feature similarity value between any two pixel points in the forgery clue map comprises: determining a first target pixel point in the forgery clue map corresponding to a first training pixel point corresponding to the forgery feature similarity value between any two pixel points in the training image, and a second target pixel point in the forgery clue map corresponding to a second training pixel point. in a case where the content represented by the first target pixel point and the second target pixel point is consistent, taking a first similarity value as a target feature similarity value corresponding to a forgery feature similarity value between any two pixel points in the training image, the first similarity value being a forgery feature similarity value between two pixel points in a case where both of the two pixel points are forged or none of the two pixel points is forged in the forgery clue map; in a case where the content represented by the first target pixel point and the second target pixel point is inconsistent, taking a second similarity value as a target feature similarity value corresponding to a forgery feature similarity value between any two pixel points in the training image, the second similarity value being a forgery feature similarity value between two pixel points in a case where one of the two pixel points is forged or none of the two pixel points is forged in the forgery clue map; training the forgery detection model based on the forgery feature similarity values between any two pixel points in the training image and the target feature similarity values corresponding to the forgery feature similarity values between any two pixel points in the training image.

5. The method of claim 2, wherein the forgery clue map is a mask image representing a forged region in the training image.

6. The method of claim 1, wherein the determining, according to the forgery feature similarity values, of a forgery probability map corresponding to the to-be-detected image for each pixel point in the to-be-detected image comprises: taking a forgery feature similarity value between each pixel point in the to-be-detected image and all pixel points in the to-be-detected image as a forgery probability value in a forgery probability map corresponding to each pixel point in the to-be-detected image; and adjusting the forgery probability value corresponding to each pixel point in the forgery probability map, so that forgery probability values corresponding to at least a second preset proportion of pixel points in different forgery probability maps are similar.

7. The method of claim 6, wherein the adjusting the forgery probability value corresponding to each pixel point in the forgery probability map comprises: determining a target forgery probability map in the forgery probability map corresponding to each pixel point in the to-be-detected image; and in a case where the forgery probability values corresponding to pixel points in the forgery probability map and the target forgery probability map that are lower than the second preset proportion are similar, updating the forgery probability corresponding to each pixel point in the forgery probability map according to the forgery probability corresponding to each pixel point in the forgery probability map and a preset correction value.

8. An apparatus for image forgery detection, comprising: a feature extraction module configured to perform feature extraction on each pixel point in a to-be-detected image to obtain an image feature corresponding to each pixel point in the to-be-detected image; a feature matching module configured to input the image feature corresponding to each pixel point in the to-be-detected image into a forgery detection model to perform forgery feature similarity matching between image features corresponding to any two pixel points in the to-be-detected image, and obtain a forgery feature similarity value between any two pixel points in the to-be-detected image; a probability map determination module configured to determine, according to the forgery feature similarity value, a forgery probability map corresponding to the to-be-detected image for each pixel point in the to-be-detected image. In a case where a proportion of pixel points with a forgery probability value exceeding a preset probability in the forgery probability map exceeds a first preset proportion, a proportion of forgery corresponding to the forgery probability map is determined according to the proportion of pixel points with a forgery probability value exceeding a preset probability in the forgery probability map in the to-be-detected image; an average value of the forgery proportions of the forgery probability map corresponding to each pixel point is taken as a forgery proportion corresponding to the to-be-detected image; a forgery probability value corresponding to each pixel point in the to-be-detected image is obtained by adding the forgery probability values corresponding to the same pixel point in the forgery probability maps of each pixel point in the to-be-detected image; starting from a pixel point with a maximum forgery probability value in the to-be-detected image, a pixel point that is forged in the to-be-detected image corresponding to the forgery proportion of the to-be-detected image is determined according to the sorting of the forgery probability values, and a region in which the pixel point that is forged is located in the to-be-detected image is determined as a forgery region in the to-be-detected image.

9. An electronic device comprising: a processor, and a memory arranged to store computer-executable instructions that, when executed, are capable of causing the processor to: perform feature extraction on each pixel point in a to-be-detected image to obtain an image feature corresponding to each pixel point in the to-be-detected image; input the image feature corresponding to each pixel point in the to-be-detected image into a forgery detection model to perform forgery feature similarity matching between image features corresponding to any two pixel points in the to-be-detected image to obtain a forgery feature similarity value between any two pixel points in the to-be-detected image; determine a forgery probability map of the to-be-detected image corresponding to each pixel point in the to-be-detected image according to the forgery feature similarity value; in a case where a proportion of pixel points with a forgery probability value exceeding a preset probability in the forgery probability map exceeds a first preset proportion, determine a proportion of forgery corresponding to the forgery probability map according to the proportion of pixel points with a forgery probability value exceeding a preset probability in the forgery probability map in the to-be-detected image; take an average value of the forgery proportions of the forgery probability map corresponding to each pixel point as a forgery proportion corresponding to the to-be-detected image; obtain a forgery probability value corresponding to each pixel point in the to-be-detected image by adding the forgery probability values corresponding to the same pixel point in the forgery probability maps of each pixel point in the to-be-detected image; starting from a pixel point with a maximum forgery probability value in the to-be-detected image, determine a pixel point that is forged in the to-be-detected image corresponding to the forgery proportion of the to-be-detected image according to the sorting of the forgery probability values, and determine a region in which the pixel point that is forged is located in the to-be-detected image as a forgery region in the to-be-detected image.

Images